--- id: "edwards-2022-instructional-models" title: "Instructional Models for Course-Based Research Experience (CRE) Teaching" authors: - "David I. Hanauer" - "Mark J. Graham" - "Rachel J. Arnold" - "Mary A. Ayuk" - "Mitchell F. Balish" - "Andrea R. Beyer" - "Kristen A. Butela" - "Christine A. Byrum" - "Catherine P. Chia" - "Hui-Min Chung" - "Kari L. Clase" - "Stephanie Conant" - "Roy J. Coomans" - "Tom D’Elia" - "Jason Diaz" - "Arturo Diaz" - "Jean A. Doty" - "Nicholas P. Edgington" - "Dustin C. Edwards" - "Elvira Eivazova" - "Christine B. Emmons" - "Kayla M. Fast" - "Emily J. Fisher" - "Christine L. Fleischacker" - "Gregory D. Frederick" - "Amanda C. Freise" - "Maria D. Gainey" - "Chris R. Gissendanner" - "Urszula P. Golebiewska" - "Nancy A. Guild" - "Heather L. Hendrickson" - "Christopher D. Herren" - "Margaret S. Hopson-Fernandes" - "Lee E. Hughes" - "Deborah Jacobs-Sera" - "Allison A. Johnson" - "Bridgette L. Kirkpatrick" - "Karen K. Klyczek" - "Ann P. Koga" - "Hari Kotturi" - "Janine LeBlanc-Straceski" - "Julia Y. Lee-Soety" - "Justin E. Leonard" - "Matthew D. Mastropaolo" - "Evan C. Merkhofer" - "Scott F. Michael" - "Jon C. Mitchell" - "Swarna Mohan" - "Denise L. Monti" - "Christos Noutsos" - "Imade Y. Nsa" - "Nick T. Peters" - "Ruth Plymale" - "Richard S. Pollenz" - "Megan L. Porter" - "Claire A. Rinehart" - "German Rosas-Acosta" - "Joseph F. Ross" - "Michael R. Rubin" - "Anne E. Scherer" - "Stephanie C. Schroeder" - "Christopher D. Shaffer" - "Amy B. Sprenkle" - "C. Nicole Sunnen" - "Sarah J. Swerdlow" - "Deborah Tobiason" - "Sara S. Tolsma" - "Philippos K. Tsourkas" - "Robert E. Ward" - "Vassie C. Ware" - "Marcie H. Warner" - "Jacqueline M. Washington" - "Kristi M. Westover" - "Simon J. White" - "JoAnn L. Whitefleet-Smith" - "Daniel C. Williams" - "Michael J. Wolyniak" - "Jill H. Zeilstra-Ryalls" - "David J. Asai" - "Graham F. Hatfull" - "Viknesh Sivanathan" venue: "CBE—Life Sciences Education" year: 2022 date: "2022-03" doi: "10.1187/cbe.21-03-0057" url: "/research/publications/10-1187-cbe-21-03-0057/" pdf: "/research/publications/10-1187-cbe-21-03-0057/dustin-edwards-10-1187-cbe-21-03-0057.pdf" pmc: "https://pmc.ncbi.nlm.nih.gov/articles/PMC9250372/" openAccess: true license: "cc-by-nc-sa" citedBy: 25 citedBySource: "OpenAlex, read 2026-09-12" --- # Instructional Models for Course-Based Research Experience (CRE) Teaching Three models for how instructors teach research courses. ## Abstract A report on research that explicates three models of pedagogical practice that underpin and characterize inquiry instruction in a course-based research experience. ## Full text Machine-extracted from the PDF linked above. It carries the artifacts that come with reading a typeset two-column page: running heads, figure captions in the flow of the prose, and words broken across line ends. The abstract above is the registry's deposit and is the authoritative text. CBE—Life Sciences Education • 21:ar8, 1–14, Spring 2022 21:ar8, 1 ARTICLE David I. Hanauer,1* Mark J. Graham,2 Rachel J. Arnold,3 Mary A. Ayuk,4 Mitchell F. Balish,5 Andrea R. Beyer,6 Kristen A. Butela,7 Christine A. Byrum,8 Catherine P. Chia,9 Hui-Min Chung,10 Kari L. Clase,11 Stephanie Conant,12 Roy J. Coomans,13 Tom D’Elia,14 Jason Diaz,15 Arturo Diaz,16 Jean A. Doty,17 Nicholas P. Edgington,18 Dustin C. Edwards,19 Elvira Eivazova,20 Christine B. Emmons,21 Kayla M. Fast,22 Emily J. Fisher,23 Christine L. Fleischacker,24 Gregory D. Frederick,25 Amanda C. Freise,26 Maria D. Gainey,27 Chris R. Gissendanner,28 Urszula P. Golebiewska,29 Nancy A. Guild,30 Heather L. Hendrickson,31 Christopher D. Herren,32 Margaret S. Hopson-Fernandes33 Lee E. Hughes,34 Deborah Jacobs-Sera,7 Allison A. Johnson,35 Bridgette L. Kirkpatrick,36 Karen K. Klyczek,37 Ann P. Koga,38 Hari Kotturi,39 Janine LeBlanc-Straceski,40 Julia Y. Lee-Soety,41 Justin E. Leonard,42 Matthew D. Mastropaolo,43 Evan C. Merkhofer,44 Scott F. Michael,45 Jon C. Mitchell,46 Swarna Mohan,47 Denise L. Monti,48 Christos Noutsos,49 Imade Y. Nsa,50 Nick T. Peters,51 Ruth Plymale,52 Richard S. Pollenz,53 Megan L. Porter,54 Claire A. Rinehart,55 German Rosas-Acosta,56 Joseph F. Ross,57 Michael R. Rubin,58 Anne E. Scherer,59 Stephanie C. Schroeder,60 Christopher D. Shaffer,61 Amy B. Sprenkle,62 C. Nicole Sunnen,63 Sarah J. Swerdlow,64 Deborah Tobiason,65 Sara S. Tolsma,66 Philippos K. Tsourkas,67 Robert E. Ward,68 Vassie C. Ware,69 Marcie H. Warner,7 Jacqueline M. Washington,70 Kristi M. Westover,71 Simon J. White,72 JoAnn L. Whitefleet-Smith,73 Daniel C. Williams,74 Michael J. Wolyniak,75 Jill H. Zeilstra-Ryalls,76 David J. Asai,77 Graham F. Hatfull,7 and Viknesh Sivanathan77* 1Department of English, Indiana University of Pennsylvania, Indiana, PA 15705; 2Department of Ecology and Evolutionary Biology, Yale University, New Haven, CT 06511; 3Salish Sea Research Center, Northwest Indian College, Bellingham, WA 98229; 4Biology, Howard University, Washington, DC 20059; 5Department of Microbiology, Miami University, Oxford, OH 45056; 6Department of Biology, Virginia State University, Petersburg, VA 23806; 7Department of Biological Sciences, University of Pittsburgh, Pittsburgh, PA 15260; 8Department of Biology, College of Charleston, Charleston, SC 29424; 9School of Biological Sciences, University of Nebraska-Lincoln, Lincoln, NE 68588; 10Biology, University of West Florida, Pensacola, FL 32514; 11Agricultural and Biological Engineering, Purdue University, West Lafayette, IN 47907; 12Department of Biology, University of Detroit Mercy, Detroit, MI 48221; 13Department of Biology, North Carolina A&T State University, Greensboro, NC 27411; 14Department of Biology, Indian River State College, Fort Pierce, FL 34981; 15Integrated Science, Business, and Technology, La Salle University, Philadelphia, PA 19141; 16Department of Biology, La Sierra University, Riverside, CA 92505; 17Division of Natural Sciences, University of Maine at Farmington, Farmington, ME 04938; 18Department of Biology, Southern Connecticut State University, New Haven, CT 06515; 19Biological Sciences, Tarleton State University, Stephenville, TX 76402; 20Biology Department, Columbia State Community College, Columbia, Tennessee 38401; 21Biology, Truckee Meadows Community College, Reno, NV 89512; 22Biological and Environmental Sciences, University of West Alabama, Livingston, AL 35470; 23Department of Biology, Johns Hopkins University, Baltimore, MD 21218; 24Biology, University of Mary, Bismarck, ND 58501; 25Department of Biology and Kinesiology, LeTourneau University, Longview, TX 75602; 26Microbiology, Immunology, and Molecular Genetics, UCLA, Los Angeles, CA 90025; 27Chemistry and Physics, Western Carolina University, Cullowhee, NC 28723; 28School of Sciences, University of Louisiana Monroe, Monroe, LA 71209; 29Biological Sciences and Geology, Queensborough Community College, Bayside, NY 11364; 30Molecular Cellular and Developmental Biology (MCDB), University of Colorado, Boulder, CO 80309; 31School of Natural and Computational Science, Massey University, Auckland 0632, New Zealand; 32Division of Biology, Kansas State University, Manhattan, KS 66506; 33Biology, Hillsborough Community College, Tampa, FL 33635; 34Biological Sciences, University of North Texas, Denton, TX 76203; Instructional Models for Course-Based Research Experience (CRE) Teaching James Hewlett, Monitoring Editor Submitted Mar 5, 2021; Revised Aug 30, 2021; Accepted Oct 22, 2021 DOI:10.1187/cbe.21-03-0057 *Address correspondence to: David I. Hanauer (hanauer@pitt.edu) or Viknesh Sivanathan (sivanathanv@hhmi.org). © 2022 D. I. Hanauer et al. CBE—Life Sciences Education © 2022 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial-Share Alike 4.0 International Creative Commons License (http://creativecommons.org/licenses/ by-nc-sa/4.0). “ASCB®” and “The American Society for Cell Biology®” are registered trademarks of The American Society for Cell Biology. CBE Life Sci Educ March 1, 2022 21:ar8 21:ar8, 2 CBE—Life Sciences Education • 21:ar8, Spring 2022D. I. Hanauer et al. 35Center for the Study of Biological Complexity, Virginia Commonwealth University, Richmond, VA 23284; 36Math and Natural Sciences, Collin College, Plano, TX 75074; 37Biology Department, University of Wisconsin-River Falls, River Falls, WI 54022; 38Department of Biology, College of Idaho, Caldwell, ID 83605; 39Department of Biology, University of Central Oklahoma, Edmond, OK 73034; 40Department of Biology, Merrimack College, North Andover, MA 01845; 41Department of Biology, Saint Joseph’s University, Philadelphia, PA 19131; 42Natural Sciences, Mitchell Community College, Statesville, NC 28677; 43Mathematics and Sciences, School of Arts and Sciences, Neumann University, Aston, PA 19014; 44Natural Sciences, Mount Saint Mary College, Newburgh, NY 12508; 45Biological Sciences, Florida Gulf Coast University, Fort Myers, FL 33965; 46Department of Science and Mathematics, Northern State University, Aberdeen, SD 57401; 47College of Computer, Mathematical, and Natural Sciences, University of Maryland College Park, College Park, MD 20742; 48Department of Biology, University of Alabama at Birmingham, Birmingham, AL 35223; 49Biological Sciences, SUNY Old Westbury, Old Westbury, NY 11568; 50Microbiology, University of Lagos, Lagos 101017, Nigeria; 51Plant Pathology and Microbiology, Iowa State University, Ames, IA 50011; 52Department of Biology, Ouachita Baptist University, Arkadelphia, AR 71998; 53Cell Biology, Microbiology and Molecular Biology, University of South Florida, Tampa, FL 33620; 54School of Life Sciences, University of Hawai’i at Mānoa, Honolulu, HI 96822; 55Department of Biology, Western Kentucky University, Bowling Green, KY 42101; 56Department of Biological Sciences, University of Texas at El Paso (UTEP), El Paso, TX 79968; 57Department of Biology, Xavier University of Louisiana, New Orleans, LA 70125; 58Department of Biology, University of Puerto Rico at Cayey, Cayey, PR 00736; 59Biology, George Mason University, Fairfax, VA 22030; 60Biological Sciences, Webster University, St. Louis, MO 63119; 61Department of Biology, Washington University in St. Louis, St. Louis, MO 63130; 62Biology, Salem State University, Salem, MA 01970; 63Biological Sciences, University of the Sciences, Philadelphia, PA 19104; 64Biology, Thiel College, Greenville, PA 16125; 65Biology, Carthage College, Kenosha, WI 53140; 66Department of Biology, Northwestern College, Orange City, IA 51041; 67School of Life Sciences, University of Nevada, Las Vegas, Las Vegas, NV 89154; 68Biology, Case Western Reserve University, Cleveland, OH 44106; 69Biological Sciences, Lehigh University, Bethlehem, PA 18015; 70Dept of Natural Sciences, Nyack College, New York, NY 10004; 71Biology, Winthrop University, Rock Hill, SC 29732; 72Molecular and Cell Biology, University of Connecticut, Storrs, CT 06269; 73Biology & Biotechnology, Worcester Polytechnic Institute, Worcester, MA 01609; 74Department of Biology, Coastal Carolina University, Conway, SC 29528; 75Biology, Hampden-Sydney College, Hampden-Sydney, VA 23943; 76Biological Sciences, Bowling Green State University, Bowling Green, OH 43402; 77Science Education, Howard Hughes Medical Institute, Chevy Chase, MD 20815 INTRODUCTION The movement to the course-based research experience (CRE), as an improvement to the traditional introductory laboratory course, has been ongoing for the last decade (American Associ- ation of University Women, 2010; American Association for the Advancement of Science, 2011; National Research Council [NRC], 2012; President’s Council of Advisors on Science and ABSTRACT The course-based research experience (CRE) with its documented educational benefits is increasingly being implemented in science, technology, engineering, and mathematics education. This article reports on a study that was done over a period of 3 years to expli- cate the instructional processes involved in teaching an undergraduate CRE. One hundred and two instructors from the established and large multi-institutional SEA-PHAGES pro- gram were surveyed for their understanding of the aims and practices of CRE teaching. This was followed by large-scale feedback sessions with the cohort of instructors at the annual SEA Faculty Meeting and subsequently with a small focus group of expert CRE in- structors. Using a qualitative content analysis approach, the survey data were analyzed for the aims of inquiry instruction and pedagogical practices used to achieve these goals. The results characterize CRE inquiry teaching as involving three instructional models: 1) being a scientist and generating data; 2) teaching procedural knowledge; and 3) fostering proj- ect ownership. Each of these models is explicated and visualized in terms of the specific pedagogical practices and their relationships. The models present a complex picture of the ways in which CRE instruction is conducted on a daily basis and can inform instructors and institutions new to CRE teaching. Technology [PCAST], 2012). Studies have shown that this edu- cational approach, when compared with a traditional labora- tory, increases inclusivity and persistence in science for a range of students with different demographic descriptors (Russell et al., 2007; Jordan et al., 2014; Hanauer et al., 2017; Hernan- dez et al., 2018) while reaching similar or better outcomes in CBE—Life Sciences Education • 21:ar8, Spring 2022 21:ar8, 3Instructional Models terms of the development of procedural and content knowledge in science (Russell and Weaver, 2011; Jordan et al., 2014; Wol- kow et al., 2014). While there is broad agreement that CREs are a valuable alternative to the traditional laboratory, the instructional com- ponents of this form of inquiry teaching are not fully transpar- ent and, to date, have not been adequately described. The Council on Undergraduate Research has recognized this situa- tion in their recent MIRIC (Mentoring the Integration of Research into the Classroom) initiative designed to provide instructors with the “significant level of training” required to implement and execute a CRE (CUR, 2020). To further support the development and implementation of CREs, a more detailed description of the instructional processes involved in teaching a CRE is needed. It is this need that the current paper addresses. The primary goal of this study is to develop a set of instruc- tional models that can be used by science, technology, engineer- ing, and mathematics (STEM) instructors to inform their in-lab CRE teaching. The concept of an instructional model as used here refers to organized sets of specific instructional practices used by CRE instructors to achieve particular CRE-related stu- dent educational outcomes. As such, the study presented here delineated a set of specific instructional practices used by active CRE instructors and modeled the relations between the use of these practices and specified outcomes. The resultant educa- tional models should provide clarification of the ways in which the aims of CRE can be effectively achieved by instructors. Traditional Laboratory The traditional laboratory course has a long history of instruc- tion and established methods of implementation. Its aim is to provide a structured approach to the learning of well-defined research skills and procedures. This is achieved through the pro- vision of explicit direction for experiments that have predeter- mined and prescribed outcomes (Tamir, 1977; Tamir and Lunetta, 1978; Domin, 1999; Weaver et al., 2008; Auchincloss et al., 2014). The instructor presents the procedure and clarifies and exemplifies the processes involved, and the student follows both the verbal instructions and the descriptions that appear in the course manual (Tamir, 1977). Results of the experiments are already known, which means that students are not actually exposed to the mistakes and failures in science that are quite central to understanding the nature of the scientific process (Auchincloss et al., 2014). Within the context of the traditional laboratory course, it is possible for students to work with some of the components of scientific thinking, including observation, measurement, and documentation (Tamir and Lunetta, 1978). However, neither the student nor the instructor is faced with the unknown or the unpredictable aspects of science as part of the educational experience (Tamir, 1977; Tamir and Lunetta, 1978; Domin, 1999; Weaver et al., 2008; Auchincloss et al., 2014). From the institution’s and the instructor’s perspectives, the traditional laboratory course does have some advantages. The course design is well controlled and suits itself to the institu- tional demands of clear objectives, planned timing, and explicit assessment. This approach allows a relatively large number of students to participate in a laboratory course with limited expenditures for time, space, equipment, and personnel (Domin, 1999). Students see advantages in this design, because they have exact guidelines on what they are supposed to be doing and the outcomes that are expected (Hanauer et al., 2018). Students consider the traditional laboratory course as helpful in developing foundational knowledge of skills and pro- cedures and as involving low levels of pressure (Hanauer et al., 2018). However, the traditional laboratory course has been crit- icized because of its naïve exemplification of science, including avoidance of the failures and uncertainties of scientific work (Bencze and Hodson, 1999; Rahm et al., 2003; NRC, 2005) and because it may not promote persistence in the sciences (PCAST, 2012; Graham et al., 2013; Hanauer et al., 2017; Hernandez et al., 2018). Course-Based Research Experience While there are several different types of potential laboratory course designs, an important alternative to the traditional labo- ratory course is the CRE, which has authentic scientific research as its central component (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; PCAST, 2012; Graham et al., 2013; Auchincloss et al., 2014; Hernandez et al., 2018). Authen- tic scientific research, as conceptualized within the CRE litera- ture, refers to the requirement that the research and associated scientific output produced by the student have value and signif- icance beyond the confines of the course itself (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014; Brownell and Kloser, 2015; Rowland et al., 2016, Shortlidge et al., 2017). The data and analyses produced within the context of the CRE are used and disseminated to a wider group of scientists and researchers and are part of an ongoing scientific research agenda (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014). This aspect of a CRE has direct ramifications for the way this type of educa- tional model is implemented in laboratory courses. As with other scientific endeavors, results are not guaranteed and tim- ing is not always predictable and may not fit into the timelines of the semester system (Auchincloss et al., 2014). Furthermore, students may face uncertainty and frustration from the pres- ence of unexpected results and ambiguity (Auchincloss et al., 2014). The continuing stages following an unexpected result may be beyond what is on the syllabus of the course, involving new protocols and scientific processes. There is far more of the unknown within a CRE than in a traditional laboratory course (Auchincloss et al., 2014). Furthermore, because the data and analyses are going to be used beyond the classroom, there is a need for quality control over the scientific output. This adds extra levels of responsibility for the instructor and students involved. These features and the uncertainty of the of the pro- tocols and processes make teaching a CRE different from teach- ing a traditional laboratory course (Auchincloss et al., 2014). To address the need for enhanced clarity concerning CRE teaching, we worked with instructors who teach the Science Education Alliance–Phage Hunters Advancing Genomics and Evolutionary Science (SEA-PHAGES) CRE for undergraduate students. As with other CREs, SEA-PHAGES is designed to pro- duce usable scientific outputs by teaching, directing, and men- toring students in the context of a laboratory course. The bene- fit of studying the SEA-PHAGES CRE is the advantages offered by its scale and administrative support, which are not found in many other CREs. First, SEA-PHAGES involves 293 instructors 21:ar8, 4 CBE—Life Sciences Education • 21:ar8, Spring 2022D. I. Hanauer et al. from 146 colleges and universities, all of whom deal with a shared set of scientific questions involving the discovery and genomic analysis of bacteriophages. Accordingly, the program provides access to a substantial number of instructors deeply involved in undergraduate education. Second, this is an estab- lished CRE with a history of continuous instruction for the last 11 years. As such, there is extensive expertise and experience in inquiry teaching across the instructors. And third, educational research within the SEA-PHAGES program has established that this program provides a quality education for a range of differ- ent student populations and is a promoter of persistence in the sciences (Hanauer et al., 2017). While the SEA-PHAGES program is on a far larger scale than most CREs and has a supportive administrative structure not found in many other CREs, the central educational features of SEA-PHAGES are those of a CRE. The program is designed to facilitate usable scientific outputs by teaching, directing, and mentoring undergraduate students. The SEA-PHAGES program is a two-semester research experience that involves undergrad- uate students in the identification and characterization of novel bacteriophage viruses. The first semester involves bacterio- phage isolation and DNA purification, which is followed by a second semester involving genome annotation and bioinfor- matic analyses. The student and faculty outcomes of this CRE involve submission of viral genome sequences to project and public databases such as PhagesDB (https://phagesdb.org) and GenBank (www.ncbi.nlm.nih.gov/genbank), making this out- put directly accessible to a wide range of researchers for the advancement of science and relevant for therapeutic applica- tion (Dedrick et al., 2019). Building upon the large number of SEA-PHAGES instructors with CRE teaching experience in the SEA-PHAGES program, we constructed instructional models that characterize CRE instruction. METHODS Overview For the current study, the following systematic qualitative approach was employed to investigate, develop, and represent the instructional models of inquiry teaching in a CRE setting. 1. Defining Instructional Practices and Educational Aims for CRE Instruction: The aim of the first stage of this study was to define the aims and pedagogical practices of CRE instruc- tion. A survey with open-ended questions asking about the main educational aims of instructors and how these were achieved pedagogically was sent to SEA-PHAGES instruc- tors. Based on systematic approaches to qualitative data (Neuendorf, 2017), the verbal responses were coded for aims of CRE teaching and the instructional practices used by instructors. 2. Instructional Model Development: The aim of this stage was to model the instructional processes of CRE teaching based on systematic visualization methods (Miles et al., 2013; Neisser, 1967). Following the coding of the verbal data into instructional practices and aims in the first stage of this proj- ect, instructional practices were related to specific instruc- tional aims. The relationships between instructional practices were visualized as a series of connecting lines within a specific model of instruction. The result was a net- work of the relations found between the instructional prac- tices to achieve a particular instructional CRE aim. Both statistical relationships (frequencies of co-occurrence) and verbal explanations (explicit connections specified in the verbal data) inform the construction of the models. Through this process, three distinct models of CRE instructional aims emerged. 3. Community Validation of Instructional Models: The aim of this stage of the project was to evaluate, modify, and validate the emergent, exploratory models of CRE instruction defined in the previous stage. This aim was achieved through a pro- cess of large-scale community-member checking. In line with qualitative approaches to research, this approach involved informant feedback on proposed interpretations of qualitative data (Creswell and Miller, 2000). At the yearly Science Education Alliance (SEA) faculty meeting, each of the three models was presented in lecture format to all the instructors assembled at the faculty meeting. The instructors were then organized into small groups of four to six partici- pants, given copies of the models, and asked to revise them according to their understandings of their own instructional practices. Revisions suggested by each group were either handwritten on the copies of the models that had been dis- tributed or were made orally in front of all the assembled instructors at the faculty meeting. All copies of model modi- fications and comments from the groups were collected, and the oral session was fully recorded. Following the member checking session, the comments, revisions, and suggestions made both on the models and orally were analyzed, and revisions were made to the three models. 4. Expert CRE Instructor Validation of the Proposed Models: The aim of this stage was to validate with expert CRE instructors the revision of the instructional models made following ini- tial large-scale member checking. A group of expert CRE instructors representing a range of institution types, gen- ders, and ethnicities participated in a focus group. The revised models were presented, and oral responses were col- lected from the participants in the focus group. The session was video-recorded, and following this session, the models were revised and finalized. Participants Participation and completion of the initial survey involved 102 SEA-PHAGES instructors, which was an 85% response rate for the survey for instructors who participated in the student assessment outcomes process in 2018. Table 1 summarizes the demographic data collected on the respondents to the initial survey. As can be seen in Table 1, there is a range of faculty ranks represented in this data set, with a predominance of ten- ured faculty (63.4%). The sample of instructors predominantly identified as ethnically White (87.3%) and had a majority of female participants (56.4%). The proportions found within the SEA-PHAGES program are comparable to those of national averages with a predominance of white instructors (Heilig et al., 2019). The majority of the respondents had at least 11 years of science teaching (68.6%) including CRE and regular teaching. The respondents to the survey were relatively new to the SEA-PHAGES program, with 50.5% having been in the pro- gram for fewer than 3 years. SEA-PHAGES instructors who were invited to the annual SEA faculty meeting participated in CBE—Life Sciences Education • 21:ar8, Spring 2022 21:ar8, 5Instructional Models the large-group feedback session. There were 96 participants at the SEA faculty meeting from a range of institution types, including research universities, 4-year schools, and community colleges. Travel and housing costs for the participants were paid for by the Howard Hughes Medical Institute. Approximately 90 participants were in the session representing the different insti- tutions participating in the program. Seven instructors were selected for the expert validation focus group discussion. Two criteria were used for selecting the instructors for the focus group: instructors with student outcomes on the Persistence in the Sciences (PITS) survey consistently in the highest 10% and representation of a range a range of institution types (including community colleges, 4-year schools, and research universities), genders (four women and three men), and ethnicities (one Afri- can American, one Hispanic, and five White). The PITS assess- ment survey includes six psychosocial outcomes (Project Own- ership Content, Project Ownership Emotion, Self-Efficacy, Science Identity, Scientific Community Values, and Network- ing) of a CRE research experience and has been shown to be reliable at differentiating CREs from other laboratory courses (Hanauer et al., 2016). As such, having student outcomes con- sistently in the upper 10% on the PITS survey is a measure of the ability of these instructors in teaching a CRE. Materials A survey with quantitative and qualitative items was developed to collect data on instructor teaching practices and beliefs. Quantitative items were developed for this survey and consisted of the following instructor psychosocial variables: Self-Efficacy, Ownership of Educational Approach, Trust Relations, and Com- munity Belonging. The Self-Efficacy as an Instructor scales were modified from Hanauer et al. (2016) and the Trust Relations scales from Cavanagh et al. (2018). The Ownership of Educa- tional Approach and Community Belonging scales were devel- oped specifically for this survey. Because the quantitative data involved both modified and new scales, a psychometric analysis of dimensionality was conducted. Following acceptable out- comes for the factor analysis of the dimensionality of each of these variables, scores were averaged for high-loading items on each variable (see Table 3 later in the article for full scales, fac- tor loadings, and descriptive data). The qualitative variables address the instructors’ aims and activities. All responses on the survey to the qualitative data were written answers. The prompts for the qualitative data consisted of the following: Please take a few moments to think about your instruction in the SEA-PHAGES program. In the space below, please specify what you think are the main educational aims of the SEA- PHAGES program. What for you are the main educational aims of the SEA-PHAGES course? In the space below, please explain how you achieve your edu- cational aims in the SEA-PHAGES. For the large-group member checking, once the small groups had been organized, the following prompts were provided for the evaluation of each of the models: What does the model succeed in capturing? What relevant activities does the model miss? What additional relations need to be added? Is the model valuable in explicating instruction in the SEA-PHAGES program? Participants were instructed to summarize their revisions and comments on the photocopied models provided. For the final focus group, each of the three models was presented in turn, and participants were asked to respond with any com- ments, modifications, and corrections they thought necessary. Procedure A survey link was sent to all SEA-PHAGES instructors who par- ticipated in the student outcomes assessment process using the online survey service Qualtrics (n = 120). The response rate from instructors was 85% (102 participants). Following online consent to research participation, participants responded to the survey. Feedback from SEA-PHAGES instructors on the emer- gent models was solicited at the annual SEA faculty meeting. All instructors at the SEA faculty meeting were invited to a TABLE 1. Demographic information on survey participants Category Frequency Percentage Rank Part-time adjunct professor 3 3 Full-time adjunct professor 17 16.8 Full-time clinical professor 6 5.9 Tenure-track assistant professor 11 10.9 Tenured assistant professor 3 3 Tenured associate professor 24 23.8 Tenured full professor 37 36.6 Gender Male 44 43.6 Female 57 56.4 Ethnic identification Asian 4 3.9 Black or African American 3 2.9 Hispanic or Latino 5 4.9 White 89 87.3 Other 1 0.9 Years of teaching 1–2 3 2.9 3–5 12 11.8 6–10 17 16.7 11–15 20 19.6 16–20 15 14.7 21+ 35 34.3 Years in SEA-PHAGES 1 16 15.5 2 21 20.4 3 15 14.6 4 7 6.8 5 4 3.9 6 8 7.8 7 7 6.8 8 7 6.8 9 8 7.8 10 7 6.8 11 3 2.9 21:ar8, 6 CBE—Life Sciences Education • 21:ar8, Spring 2022D. I. Hanauer et al. group session. During the first 20 minutes of this session, the methodology and the analyzed models of instruction were pre- sented to the whole group. Instructors were then divided into groups of four to six people and given a copy of the three mod- els that had been presented. They were instructed to discuss the models as a group and address the prompt questions. Responses were written on the photocopies and collected at the end of the session. Each group presented an oral response to the models to the whole group and responded to questions and comments. The full session was video-recorded. For the final, expert focus group session, the modified versions of the three models were presented and discussed by the participants. The session was video-recorded. Data were collected according to the ethical guidelines of Indiana University of Pennsylvania Institutional Review Board (IRB no. 18-062). Analytical Procedures The analysis of the quantitative items on the survey followed standard statistical procedures. Descriptive statistics were used to understand the overall characteristics of the instructors in terms of the variables of Self-Efficacy as an Instructor, Owner- ship over the Educational Approach, Trust Relations with Stu- dents, and the Degree of SEA-PHAGES Community Belonging. Using standard qualitative analytical approaches, the verbal data on the survey were analyzed in the following stages: 1. Content Coding of Pedagogical Practices: The initial stage of analysis consisted of constructing a codebook of instructor responses to the open-ended items on the survey. Each of the instructors’ statements was analyzed in terms of the peda- gogical practices specified and the aims that they were try- ing to achieve. Through a cyclical process of reading and note-taking, a provisional list of specific pedagogical prac- tices and pedagogical aims was defined. Before the entire data set was coded, 20% of the utterances were coded by two researchers independently with an 87% agreement rate. All cases of disagreement were evaluated and either assigned to existing codes or a new code was developed. A second round of coding produced a 94% agreement rate, which was considered reliable for coding the data set for codes dealing with specific pedagogical practices. The list of specific peda- gogical practice codes can be found in Table 2. 2. TURF (Total Unduplicated Reach and Frequency) Analysis of Aim: The frequency of occurrence for aims by instructor was calculated. A TURF analysis was performed to clarify the optimal agreement on the aims of the CRE across instructors (Howell, 2016). The TURF algorithm identifies the optimal specified aims accepted by the largest number of participant instructors, which specifies which combination of aims is accepted by the largest number of instructors. This analysis produced a combination of three aims that were accepted by 95% of the instructors. The three aims were: 1) being a sci- entist and generating data; 2) teaching procedural knowl- edge; and 3) fostering project ownership. These three educational aims had the highest frequency of being men- tioned across all instructors and, as such, were considered to best capture the educational aims underpinning the program as specified by instructors. 3. Pedagogical Practice Code Clustering: Once the data set had been analyzed for codes of specific practices, the codes used by each instructor were recorded. The sets of codes were organized initially by instructor and subsequently by the main educational aims specified by that instructor. In this way, codes were clustered by both instructor and aim. 4. Co-occurrence Code Analysis: The codes were analyzed within the verbal statements of each instructor for the co-occur- rence of codes of specific pedagogical practices within an aim. The analysis was done in relation to the collection of statements from each instructor so as to not create artificial connections between codes. The analytical process of instructor-specific, co-occurrence coding created groupings of specific pedagogical practice codes found within each of the three aims specified in the TURF analysis. Specific peda- gogical practices that had a similar pedagogical aim were grouped together into more general pedagogical practices, and high frequencies of co-occurrence in relation to particu- lar educational aims were specified. Table 2 presents these groupings. 5. Model Specification: The result of previous stages of analysis consisted of the specification of three main educational aims of the program and a set of co-occurring pedagogical prac- tices that facilitated this pedagogical aim. Three instruc- tional models were defined based on this data. Related, co-occurring sequences of pedagogical practices were placed in relation to one another, and the full set of relations was graphically represented. Logical connections were made between the pedagogical practices, and instructor descrip- tions were rechecked. The result of this process was the spec- ification of three models of instruction. 6. Member Checking: Instructor feedback and validation (mem- ber checking) was conducted to validate the emergent mod- els from the analysis of the survey data. As specified in the METHODS above, the instructional models were presented to the collected audience of SEA-PHAGES instructors at the annual SEA faculty meeting, and through group work, spe- cific modifications to the models were provided in written, visual, and oral form. All comments and suggestions from the instructor groups were analyzed. Repeated comments by at least two groups were considered important to address in revising the model, and revisions accompanied by a strong rationale were considered for inclusion. The following prac- tices were added at this stage (Student Presentation, Scien- tific Output, Future Educational and Career Opportunities, and Ethical Understanding). The models were revised and updated with these additions. 7. Finalization of the Models: As a final stage of member check- ing, the models were presented to a focus group of expert instructors. The revised instruction models were presented and discussed with this group. Changes were not found to be necessary at this stage, and the models were finalized. RESULTS Survey Participant Diversity and Alignment To ensure validity and suitability of building instructional mod- els with a range of instructors from the SEA-PHAGES program, we evaluated the degree to which these instructors were aligned and felt comfortable with the educational approach used in the program. On the initial survey to instructors, data were col- lected on the psychosocial variables of Self-Efficacy as an CBE—Life Sciences Education • 21:ar8, Spring 2022 21:ar8, 7Instructional Models TABLE 2. Codebook of pedagogical practices and associated educational aims Pedagogical practice Definition Specific practices Associated educational aim Provide scientific content A pedagogical practice in which back- ground scientific information is provided to the student • Verbal explanation of scientific background for the technique being used • Reading of primary literature • Reference to lecture material • Procedure development Explicit discussion of a CRE A pedagogical practice in which the specific characteristics of a CRE (authentic research experience) are explained to the student • Differentiation of CRE classes from traditional labs • Focus on the importance of generating authentic data • Positioning students as researchers and scientists • Being a scientist and generating data Facing ambiguity A pedagogical practice in which students are made to face failure, uncertainty, and ambiguity in their scientific work • Allowing students to struggle with ambiguity and uncertainty • Explaining that failure is a central part of science and is to be expected • Holding back from providing quick answers • Being a scientist and generating data Modeling scientific thinking A pedagogical practice in which the instructor models verbally for the students the processes of thinking as a scientist in relation to the issues or problems they present in their ongoing research • Thinking through problems with students • Modeling the design, planning, and methodological components of an experiment • Critically thinking through and discussing the questions and issues posed by student work • Procedure development • Being a scientist and generating data Protocols and training A pedagogical practice in which the instructor presents, trains, and supervises students following a new protocol • Provision and reading of the protocol • Protocol practice and training • Supervision of the following of a protocol • Procedure development • Fostering ownership Mentorship A pedagogical practice in which the instructor or a peer mentors an individual student during the research process by providing individual guidance and problem solving • Mentoring of individual students by instructor or teaching assistant • Peer mentoring and problem solving • Provision of specific individualized guidance in writing • Being a scientist and generating data Peer collaboration A pedagogical practice that involves groups of students working together on their research • Team or small-group assignments • Shared responsibility across the team • Shared discussion and support for success and challenges • Fostering ownership Encouraging independence in students A pedagogical practice designed to enhance the ability of students to work independently as researchers • Directing students to think independently • Asking students to problem solve themselves • Positioning students as the decision makers in their scientific work • Holding back from providing answers • Being a scientist and generating data • Fostering ownership Fostering personal responsibility A pedagogical practice designed to enhance students’ understanding of their personal responsibility as researchers • Assigning specific work to be completed • Explaining the responsibility of authentic scientific data • Monitoring regular reporting of progress • Requiring clear and consistent documentation of research • Promoting personal reflection on the research that has been conducted • Fostering ownership Encouraging enthusiasm A pedagogical practice designed to recognize success and encourage positive enthusiasm toward research • Facilitating and recognizing the joy of discovery • Instructor enthusiasm • Course recognition of successes • Being a scientist and generating data (Continues) 21:ar8, 8 CBE—Life Sciences Education • 21:ar8, Spring 2022D. I. Hanauer et al. Pedagogical practice Definition Specific practices Associated educational aim Documentation of scientific practice A pedagogical practice involving the documentation of the research of the students • Lab notebook documentation • Notebook discussion • Notebook checking • Procedure development Student presentation A pedagogical practice involved in creating situations in which the students present their ongoing research • Lab notebook meeting with other students present • Poster session for the presentation of outcomes • Conference participation • Being a scientist and generating data • Fostering ownership Scientific output This code relates to the actual scientific output of an authentic research activity. • Specific products of research (such as an isolated phage, electron microscopy, annotated genome) • Procedure development Future educational and career opportunities This code relates to the discussion of short- and long-term future educational and career opportunities. • Discussion of graduate school, internships, and related science careers • Facilitating future options for scientific development through instructor network • Fostering ownership Ethical understanding This code relates to the discussion of ethical issues during course lab time with students • Discussion of ethics of research • Discussion of integrity in research • Fostering ownership Encouraging perseverance This code relates to the encouragement a faculty member provided during lab work • Helping students overcome short-term frustrations • Providing support during difficult periods • Being a scientist and generating data Instructor, Ownership over the Educational Approach, Trust Relations with Students, and the Degree of Community Belong- ing. As can be seen in Table 3, the instructors who completed the survey have very high levels of belief in their abilities as CRE instructors, strongly agree with the educational approach, have positive trust relations with their students, and have a sense of belonging to the educational community of the SEA- PHAGES program. For evaluation of the representativeness of the set of instruc- tors who participated in constructing the models of CRE instruc- tion, the outcomes of their students on the PITS survey (Hanauer et al., 2016) was compared with the program-wide and multiyear data set of student outcomes. The z-score com- parisons between the means of the multiyear sample (n = 22,492) and student outcomes from the instructors who partic- ipated in the survey (n = 5564) are between 0 and 0.06, indicat- ing that the student outcomes for the participant instructors are very close to the averages for the SEA-PHAGES program. Taken together, these comparisons suggest that instructors who participated in this study are valid informants for building models of instruction, because they are confident in their instructional abilities, are aligned with the program, and can produce student outcomes very close to program outcomes. Aims and Pedagogical Practices of SEA-PHAGES CRE Instruction The primary goal of this study was to describe the instructional practices of the SEA-PHAGES program. To accomplish this, we first determined the main educational aims of the SEA-PHAGES program as understood and accepted by the largest number of its instructors. Using TURF analysis of instructor responses to an open-ended survey, three aims that had the highest fre- quency of mention across all instructors and were shared by 95% of the instructors were identified. They consist of 1) being a scientist and generating data; 2) procedural knowledge; and 3) fostering ownership. These three aims are considered to best capture the underpinning aims of the SEA-PHAGES program as defined by its instructors. To then uncover the set of pedagogical practices that instruc- tors use to achieve their educational aims, we analyzed survey responses for the co-occurrence of pedagogical practices by both instructor and educational aim. Related, co-occurring ped- agogical practices were then placed in relation to one another, and the full set of relations for each aim was graphically repre- sented as a model of CRE instruction. Logical connections were drawn as lines between the pedagogical practices within each model, and these emergent models and connections were then checked, modified, and validated through an iterative process with SEA-PHAGES instructors. The three resulting models of CRE instruction and the relationships between pedagogical practices within each model are presented and described in the following sections. Model 1: Being a Scientist and Generating Data The pedagogical aim of supporting students to be a scientist and generate data is at the core of authentic research experi- ences as a STEM educational approach. As this educational approach is potentially different from other educational approaches to which many students may be accustomed, the pedagogical practices that support this aim are therefore cen- tered around equipping students with the mindset and tools to engage productively with authentic research. These pedagogi- cal practices, as described here and in Figure 1, can be grouped into three discernible stages. TABLE 2. Continued CBE—Life Sciences Education • 21:ar8, Spring 2022 21:ar8, 9Instructional Models TABLE 3. Descriptive statistics for instructor psychosocial variables Instructor variables (7-point scale; 7 = strongly agree) Item factor loadings Mean SD Median Minimum Maximum Self-efficacy as an instructor I know how to explain scientific procedures relevant to this course. I know how to explain scientific concepts relevant to this course. I have adequate levels of scientific knowledge to be able to confidently teach this laboratory course. I know appropriate teaching strategies and approaches in order to teach this laboratory course. −0.89 −0.82 −0.73 −0.68 6.68 0.45 7 5.25 7 Ownership over educational approach I think that students are positively engaged in this course. I believe that the way this course is designed is most suitable for the aims of this course. I am confident that the design of this course is of value to the student’s science education. This course reflects what I learned about effective scientific inquiry teaching. 0 0.99 0.88 0.81 0.81 6.62 0.44 6.75 5 7 Trust relations with students It’s important to me to understand what my students’ educational goals are. I truly care about my students’ educational welfare. I’m confident that my students would say that I “get” them. 0 0.68 0.61 0.43 6.35 0.47 6.37 5.13 7 Degree of community belonging I feel that I belong in the SEA-PHAGES community. I see myself as a phage-biologist I believe that my work contributes to the SEA-PHAGES community. The SEA-PHAGES community supports my pedagogical work. Membership in the SEA-PHAGES community has added value to my professional identity. 0 0.78 0.57 0.58 0.58 0.62 5.9 0.91 6.14 2 7 FIGURE 1. Being a scientist and generating data. Model presents co-occurrence relationships between coded verbal responses to survey items followed by member checking for validity. Lines represent co-occurring verbal codes within instructor statements concerning the instructional aim of being a scientist and generating data. The size of the box represents the number of related instructional practices. The model reveals the central role of ambiguity and uncertainty in CRE instruction and role of a series of instructor activities to alleviate, contextualize, and address this aspect of the CRE laboratory. 21:ar8, 10 CBE—Life Sciences Education • 21:ar8, Spring 2022D. I. Hanauer et al. The first stage involves positioning students to be active par- ticipants in a research community. The model describes two pedagogical practices for doing so. The first involves equipping students with an understanding of their role and position as student-researchers within the context of a larger scientific community. This is achieved through explicit discussion with students about the importance of what they are doing, the ways in which their work interacts with the scientific community, and the resulting high bar required for the quality of their work (Table 2, “Explicit discussion of a CRE”). The second approach involves providing students with the primary tools to begin conducting research. Here, students are trained to implement research protocols (Table 2, “Protocols and training”). Importantly, the learning and usage of specific protocols occur in response to the needs of their research at a given juncture. Thus, the learning of a protocol is not an abstract activity for future reference (as in some lab course designs), but rather is tied to a specific scientific output that the student needs to generate at a particular time during the course. These two pedagogical practices—explicit discussion and protocols and training—interact to support a deep understanding of the protocol and the need to perform the protocol appropriately so that students understand that the protocol is designed to pro- duce authentic scientific data that will be used by other scien- tists. This is exemplified in the following instructor statements: “At the beginning of the semester, I explain the goals (as I see them) clearly and communicate the cutting-edge nature of this enterprise” and “I run my classes as ‘research groups,’ so that nearly everything we do is in service to increasing our under- standing or moving forward of the research.” As student-researchers begin to engage with research proto- cols, we enter the second and central stage of this instructional model, which deals directly with students facing ambiguous or unanticipated research outcomes inherent to authentic research (Table 2, “Facing ambiguity”). Instructors allow for situations in which a student is unsure of how to continue or faces an unde- sired research outcome, both natural aspects of science. How- ever, given that this nature of science is not necessarily known or appreciated by a student-researcher and can therefore be dif- ficult to contend with, the instructor contextualizes these out- comes and provides students the means to manage them. As seen in Figure 1, SEA-PHAGES instructors describe two types of approaches for this. One approach involves a series of psychological dispositions and mindsets that include encourag- ing independence of thinking, perseverance, and engagement with and enthusiasm for their research. These are captured in the following quotes by instructors: “At the start of each lab class I ask the students to tell me what they are doing today, to encourage independence, rather than dependence on instruc- tors” and “I praise and encourage every success, as well as their efforts so that they do not become discouraged.” The second approach to help students manage ambiguity involves supportive and instructional frameworks. Here, instructors can promote peer collaborations so that students can share their expertise, skills, and knowledge to advance their research together, or the instructor can provide direct guidance and discussion for an individual student (Table 2, “Mentor- ship”). The latter is done by modeling the practice of scientific thinking (Table 2, “Modeling scientific thinking”). Modeling sci- entific thinking in the context of resolving an issue faced by students can promote their internalization of scientific thinking and thereby not only provide them the means to solve future issues but also the agency to move their ongoing research for- ward. Once students are able to manage the ambiguity they face, instructors can then engage students in the third stage of this model, in which scientific outputs that are valuable to the scientific community are actually produced. The specific scien- tific outputs will depend on the actual scientific inquiry being conducted. In the case of the SEA-PHAGES program, these include outcomes such as genome annotations submitted to GenBank. Model 2: Teaching Procedural Knowledge The pedagogical aim of teaching procedural knowledge is at the core of actualizing authentic research, because without proce- dural knowledge, science cannot be conducted. The pedagogi- cal practices of this second model bolster an aspect of the first model—the teaching of protocols—so that a student can engage with protocols as a responsible, problem-solving, and deci- sion-making student-researcher. The pedagogical practices for this aim, as described here and in Figure 2, can also be grouped into three discernible stages. The first stage involves the provision of scientific background for the experimental technique being taught (Table 2, Content Information). Importantly, this precedes teaching a student to use a protocol. The aim of this scaffolding is for students to be able to contextualize a procedure when it is taught and thereby avoid blindly following a protocol without really understanding what is happening from a scientific perspective. This is achieved through explicit instruction, discussion, and the reading of pri- mary literature, and is exemplified by the following instructor statements: “I encourage my students to think through the methods for finding bacteriophages before I ever give them the protocols. This enhances their active learning by encouraging discussion and thought, rather than simply following instruc- tions” and “My students also read primary scientific literature. I found a nice and relatively easy to read paper on phage therapy being used to treat coral disease. They follow the same basic TABLE 4. Mean, SD, and z-score for multiyear (2015–2020) SEA-PHAGES student outcomes (n = 22,492) and survey participant instructor student outcomes (n = 5564) Project ownership content Project ownership emotion Self-efficacy Science identity Scientific community values Networking Multiyear SEA-PHAGES student outcomes (2015–2020) 3.68 (1.04) 3.55 (1.11) 3.92 (1.06) 3.65 (1.14) 4.84 (1.41) 3.4 (1.19) Student outcomes from instructor participants 3.62 (1.13) 3.48 (1.19) 3.85 (1.15) 3.61 (1.24) 4.76 (1.53) 3.4 (1.26) z-score comparison 0.05 0.05 0.06 0.03 0.05 0.0 CBE—Life Sciences Education • 21:ar8, Spring 2022 21:ar8, 11Instructional Models protocols as the students carry out, so they’re not as intimidated by the Materials and Methods section.” Once the student has sufficient content knowledge to under- stand a protocol, we enter the second stage, in which a student is taught to both use a protocol (Table 2, “Protocols and train- ing”) and think as a scientist (Table 2, “Modeling scientific thinking”). The interaction between these three pedagogical practices—teaching content information, teaching protocols, and modeling scientific thinking—positions a student to be an independent researcher; a student will not only carry out a pro- tocol, but will also be able to interpret the results, even when the results are ambiguous or unexpected; to problem solve; and to be the decision maker in relation the research that is being con- ducted. This positioning of students as independent researchers is exemplified by the way one instructor describes his interac- tions with students: “When they have questions, especially questions like, ‘What does this (result) mean?’ or ‘So now what do we do?’ I try to respond with my own questions, rather than answers. I try to get them to do the thinking, rather than me.” In conjunction with promoting independence, being taught to properly document their research positions students to be responsible for their research (Table 2, “Documentation of sci- entific practice”). The usage of the notebook is described by one instructor in the following way: “We strive to help students understand that a notebook is far more than a collection of pro- tocol steps. That it should, instead, tell the entire story of their experiment—explain their purpose and how the experiment works, record any modifications to the protocol, and thorough recording of primary data and their analyses.” This documenta- tion process is therefore also an additional opportunity for stu- dents to exercise scientific thinking and to converse with the instructor and other peer student-researchers. In the third stage, scientific output is generated that, as with other aspects of learning procedural knowledge, is contextual- ized in relation to the protocol, the science that underpins it, and the way a scientist would interpret the results. Model 3: Fostering Project Ownership Project ownership—the sense of the personal significance of the research being conducted—has been shown to be a feature that differentiates authentic research experiences from traditional laboratory courses (Hanauer and Dolan, 2014) and to promote the persistence of students in the sciences. The pedagogical practices for fostering project ownership are centered around promoting in students a sense of agency and personal responsi- bility to their research, to themselves, and to the larger scientific community. These pedagogical practices, as described here and in Figure 3, can be grouped into three different stages. The first stage of fostering project ownership begins with teaching a scientific protocol (Table 2, “Protocols and train- ing”). This starting point is not arbitrary, because an actual abil- ity to perform the science itself is required in order to be involved and engaged with, and to have ownership over, the research. This is principally an issue of self-efficacy and is exem- plified by these instructor statements: “By having each student isolate a phage, characterize it, name it and see an image via electron microscopy, students get the feeling of ownership and contribution to the greater scientific community” and “Beyond all else, instilling beginning students with the confidence that they too can ‘do science’ establishes trust and mutual respect that leads to project ownership and success.” Once a student has the means to actively engage in research, we enter the second stage, in which project ownership is fos- tered by making the students personally responsible for their research (Table 2, “Fostering personal responsibility”). One instructor stated the following: “I give the students as much control and ownership of their experiment as possible, and make them responsible for it as well. I tell them how to do the FIGURE 2. A model of procedural knowledge instruction. Model presents co-occurrence relationships between coded verbal responses to survey items followed by member checking for validity. Lines represent co-occurring verbal codes within instructor statements concern- ing the aim of procedural knowledge instruction. The size of the box represents the number of related instructional practices. The model reveals the broader context within which protocols are taught in a CRE, specifying the importance of scientific background, modeling scientific thinking, and documentation in developing student understanding and use of scientific protocols. 21:ar8, 12 CBE—Life Sciences Education • 21:ar8, Spring 2022D. I. Hanauer et al. protocols, of course, and provide the materials, but then, essen- tially, we provide lab space for them for fourteen weeks, all day, every day, only interrupted by the scheduled meeting times of the class.” This responsibility is reinforced through several other pedagogical approaches. In one approach, the student’s respon- sibility is grounded through an understanding of the ethical aspects of conducting research such as the importance of integ- rity, honesty, and transparency (Table 2, “Ethical understand- ing”). In another approach, the student’s personal responsibility is bolstered through the explicit positioning of the student as an independent thinker (Table 2, “Encouraging independence in students”). Allowing and promoting the process of independent thought, problem solving, and decision making in relation to their research, gives students agency to be personally responsi- ble for their research. One instructor stated: “I encourage them to evaluate their results, to discuss them with their peers, and then to present them to me. I don’t ever tell them what they should do next—instead I help them to work through the pros and cons of the various options before them, and encourage them to test out their ideas. When the results are in, I encourage them to evaluate them critically, and then to repeat the thought process.” As students become more personally responsible for their own research, we then enter the third stage, in which a number of pedagogical practices are implemented to support students as they manage that responsibility. With regard to the more anx- iety-ridden aspects of having personal responsibility over one’s research, the instructor can help by emphasizing the positive aspects of doing research (Table 2, Encouraging Engagement and Enthusiasm). One instructor stated: “I take time to get to know my students and provide them a tremendous amount of support and encouragement. I bring enthusiasm as well as com- passion to my class/lab.” Instructors can also promote team- work (Table 2, “Peer collaboration”) that requires shared and reciprocal responsibility of (or by) the team members. Personal responsibility can be further enhanced by relating the ongoing research process to future outcomes for a student’s own career. In the medium term, this can be in the form of presenting per- sonal research to other people in the form of a final course poster session, data publication, research article participation, or conference participation (Table 2, “Student presentation”). In the longer term, participation in a research experience can provide important insight, skills, and experience that may be important for future career goals, such as graduate school and a science-related career (Table 2, “Future educational and career opportunities”). For students, having responsibility over their research also involves having responsibility over their edu- cations and futures. Summary of Relations between CRE Instructional Models The research presented here describes CRE instruction in terms of three interrelated models. The first model—being a scientist and generating data—is focused on a set of instructional prac- tices that support students while they handle the ambiguity and uncertainty of the scientific process in order to produce scientif- ically valuable output. The second and third models develop the first model by explicating the teaching of protocols in order to generate data and fostering underpinning aspects of being a FIGURE 3. A model of fostering project ownership. Model presents co-occurrence relationships between coded verbal responses to survey items followed by member checking for validity. Lines represent co-occurring verbal codes within instructor statements concern- ing the aim of fostering project ownership. The size of the box represents the number of related instructional practices. The model situates the development of project ownership within the learning of specific protocols followed by a series of instructor practices that encourage and support personal responsibility, peer collaboration, and shared research presentation. CBE—Life Sciences Education • 21:ar8, Spring 2022 21:ar8, 13Instructional Models scientist, respectively. While all three models can be seen as distinct instructional models fulfilling specific CRE-related edu- cational aims, all three are connected to the larger aim of help- ing students to perform as novice scientists who produce usable scientific output. DISCUSSION AND CONCLUSIONS Prior research on CREs has tended to focus on the definitional aspects of a CRE (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014; Hernandez et al., 2018) and student outcomes (Russell et al., 2007; Jordan et al., 2014; Hanauer et al., 2017; Hernandez et al., 2018). Less has been said about the character- istics of the pedagogy used in this educational approach. The aim of this paper is to characterize the pedagogy of CRE teach- ing through the modeling of instructors’ statements about how they teach. The results of this study suggest that three interre- lated instructional models describe the specific components of CRE inquiry teaching. These models describe how to facilitate the process of being a scientist and generating scientific data and ways of teaching procedural knowledge and fostering proj- ect ownership. A summary of the models and the data presented in this study suggests three underpinning features that characterize CRE pedagogy. The first feature is inherent to the design of a CRE and consists of the centrality of an active research agenda that directs the content and outcomes of the course. When the implementation of scientific research with needed scientific out- puts directs the work of student-researchers, then many aspects of the laboratory course need to adapt to the procedural vari- ability of authentic science. There is a degree of uncertainty related to timing, outcomes, and course development that needs to be addressed. The second feature that characterizes this form of pedagogy emerges from the first. Because the aim is to contribute to an active research agenda, and this causes a degree of uncertainty, a range of additional pedagogical sup- portive practices need to be implemented by the instructor. Instructors need to provide encouragement, foster persever- ance, support peer work, model scientific thinking, and mentor students as they work through the research they are doing. These interventions are ongoing and timed in response to con- tingencies that emerge in the scientific research itself. The third feature is tied to both the first and the second features summa- rized here and consists of a change in the relationship between the instructor and the student. Because both the student-re- searcher and instructor are involved in the same research agenda, their relationship is characterized by shared interest in producing scientifically valuable outputs. As such, instructors encourage and support students’ personal responsibility, inde- pendence, and ownership over the research and, at the same time, situate students’ work within their own research and that of the scientific community. Together, the students and the instructor move the field of science forward. The pedagogy presented here reveals differences in the rela- tions between instructor, student, and course work in a CRE compared with a traditional laboratory course. The traditional laboratory course is defined by a structured environment for both student and instructor, in which the expectations of course timing, course content, and procedural outcomes are predeter- mined (Tamir, 1977; Tamir and Lunetta, 1978; Domin, 1999; Weaver et al., 2008) and the pedagogy involves explicit instruc- tion, demonstration, and hands-on implementation of a set of defined experiments with defined outcomes (Tamir, 1977). As a result, the core set of relations between instructor and student are those of expert and novice, and while the student’s work might be consequential from an educational perspective, it has no scientific value in itself. This is very different from what is described for CRE pedagogy in this study. The instructor is a mentor and a guide; the student is an independent responsible researcher; the research is real and reported to the scientific community. Teaching in a CRE requires a different set of posi- tions for the instructor, student, and scientific output. We hypothesize that these different positions are central to the pos- itive outcomes observed for students participating in CREs. The pedagogy presented here has implications for instruc- tors and institutions new to CREs. CRE pedagogy has specific qualities that may make it different from the pedagogy of other undergraduate science courses. The CRE approach counters more traditional beliefs in a structured knowledge hierarchy within which scientific research can only be con- ducted at higher educational levels. These beliefs often inform the design of early-career laboratory courses and cre- ate labs designed to help students learn procedures in a con- trolled way without facing the difficulties and uncertainties of the authentic scientific process. CRE pedagogy, on the other hand, parallels the pedagogy of graduate science edu- cation, in that both are rooted within the context of doing science and involve cognitive, conceptual, and emotional facilitation to manage the uncertainties of the scientific pro- cess and to be productive as a scientist. Undergraduate sci- ence educators are likely already familiar with this form of pedagogy from their own graduate educations, with a subset additionally engaged in teaching graduate students. Key, then, to supporting instructors as they adopt and implement CRE pedagogy, are an institutional or departmental mindset and culture that promote the integration of undergraduate teaching with research, so that the scientific process is both authentic and a fundamental component of undergraduate science education, and that encourage instructors to lever- age their graduate training experience and their researcher identity to inform their teaching. The models and data pre- sented here provide a framework to support discussions regarding how to best teach a CRE. There are limitations to the current study. The core data set used to produce the visualizations and descriptions of the instructional models consisted of written responses to open- ended questions. While a systematic approach was used to ana- lyze these responses, verbal data are polysemantic and can be understood in a variety of ways. To address this, the current study used two levels of member checking and feedback provi- sion from the instructors active in teaching a large-scale CRE. However, these instructors did come from one program—the SEA-PHAGES program—and this also might be a limiting factor of what is described in this study. The nature of this program does have diversity in terms of the student bodies taught and institutions involved, but all participants came from the same program, and this could have influenced what is described in the models. The models are still exploratory, and further research will be needed to show the relationships between these models and student outcomes. 21:ar8, 14 CBE—Life Sciences Education • 21:ar8, Spring 2022D. I. Hanauer et al. The instruction models presented in this report attempt to capture what is specific to CRE inquiry teaching. They pres- ent a complex picture of the ways in which instruction is conducted on a daily basis as understood by instructors who teach in a CRE. Being a CRE instructor requires a flexible richness in addressing individual students and the work they are doing. It requires the ability to honestly interact as a sci- entist with the work conducted, and it requires allowing a student to function independently while providing the right amount of support for that independence. More than any- thing else, being a CRE instructor requires seeing the stu- dents as researchers and facilitating the process so that the students see themselves as scientists and their work as part of the scientific community. ACKNOWLEDGMENTS We thank all SEA-PHAGES instructors who participated in the survey. This work was supported by Howard Hughes Medical Institute grants 52008198 and GT12052 to D.I.H. and grants 54308198 and GT12053 to G.F.H. REFERENCES American Association for the Advancement of Science. (2011). Vision and change in undergraduate biology education: A call to action. Washington, DC. American Association of University Women. (2010). Why so few? Women in science, technology, engineering and mathematics. Washington, DC. Auchincloss, L. C., Laursen, S. L., Branchaw, J. L., Eagan, K., Graham, M., Hanauer, D. I., … & Dolan, E. L. (2014). Assessment of course-based un- dergraduate research experiences: A meeting report. CBE—Life Sciences Education, 13, 29–40. Bencze, L., & Hodson, D. (1999). Changing practice by changing practice toward more authentic science and science curriculum development. Journal of Research in Science Teaching, 36, 521–539. Brownell, S., & Kloser, M. J. (2015). Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology. Studies in Higher Education, 40, 525–544. Cavanagh, A. J., Chen, X., Bathgate, M., Frederick, J., Hanauer, D. I., & Graham, M. J. (2018). Trust, growth mindset, and student commitment to active learning in a college science course. CBE—Life Sciences Education, 17(1), 1–8 Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative research. Theory into Practice, 39(3), 124–130. Council on Undergraduate Research. (2020). Divisions—Biology Mentoring the Integration of Research into the Classroom (MIRIC): An initiative of the CUR Biology Division. Retrieved May 8. 2020, from www.cur.org/ who/governance/divisions/biology/miric Dedrick, R. M., Guerrero-Bustamante, C. A., Garlena, R. A., Russell, D. A., Ford, K., Harris, K., ... & Schooley, R. T. (2019). Engineered bacteriophages for treatment of a patient with a disseminated drug-resistant Mycobacteri- um abscessus. Nature Medicine, 25(5), 730–733. Domin, D. (1999). A review of laboratory instruction styles. Journal of Chemical Education, 76, 543–547. Graham, M., Frederick, J., Byars-Winston, A., Hunter, A.-B., & Handelsman, J. (2013). Increasing persistence of college students in STEM. Science, 341, 1455–1456. Hanauer, D. I., & Dolan, E. (2014). The Project Ownership Survey: Measuring differences in scientific-inquiry experiences. CBE—Life Sciences Educa- tion, 13, 149–158. Hanauer, D. I., Frederick, J., Fotinakes, B., & Strobel, S. A. (2012). Linguistic analysis of project ownership for undergraduate research experiences. CBE—Life Sciences Education, 11, 378–385. Hanauer, D. I., Graham, M. J., Betancur, L., Bobrownicki, A., Cresawn, S. G., Garlena, R. A., … & Hatfull, G. F. (2017). An inclusive research education community (iREC): Impact of the SEA-PHAGES program on research outcomes and student learning. Proceedings of the National Academy of Sciences USA, 114, 13531–13536. Hanauer, D. I., Graham, M., & Hatfull, G. F. (2016). A measure of student persistence in the sciences (PITS). CBE—Life Sciences Education, 15, ar54. Hanauer, D. I., Jacobs-Sera, D., Pedulla, M. L., Cresawn, S. G., Hendrix, R. W., & Hatfull, G. F. (2006). Teaching scientific inquiry. Science, 314, 1880– 1881. Heilig, J. V., Flores, I. W., Barros-Souza, A. E., Barry, J. C., & Barcelo Monroy, S. (2019). Considering the ethnoracial and gender diversity of faculty in United States college and university communities. STCL Hispanic Journal of Law and Policy, 2–31. Hanauer, D. I., Nicholes, J., Liao, F., Beasley, A., & Henter, H. (2018). Short- term research experience (SRE) in the traditional lab: Qualitative and quantitative data on outcomes. CBE—Life Sciences Education, 17(4), 1–14. Hernandez, P. R., Woodcock, A., Estrada, M., & Wesley Schultz, P. (2018). Undergraduate research experiences broaden diversity in the scientific workforce. BioScience, 68, 204–211. Howell, J. (2016). A simple introduction to TURF analysis (Sawtooth Software Research Paper Series). Retrieved October 7, 2020, from www .sawtoothsoftware.com/support/technical-papers/turf-related-papers/ a-simple-introduction-to-turf-analysis–201 Jordan, T. C., Burnett, S. H., Carson, S., Caruso, S. M., Clase, K., DeJong, R. J. … & Hatfull, G. F. (2014). A broadly implementable research course in phage discovery and genomics for first-year undergraduate stu- dents. mBio, 5, e01051–13. Miles, M. B., Huberman, A. M., & Saldana, J. (2013). Qualitative data analysis: A methods sourcebook (3rd ed.). Los Angeles: Sage. Neisser, U. (1967). Cognitive psychology. New York: Appleton-Century-Crofts. National Research Council (NRC). (2005). America’s lab report: Investiga- tions in high school science. Washington, DC: National Academies Press. National Research Council. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. Washington, DC: National Academies Press. Neuendorf, K. A. (2017). The content analysis guidebook (2nd ed.). Los Ange- les: Sage. President’s Council of Advisors on Science and Technology. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington, DC: U.S. Government Office of Science and Technology. Rahm, J., Miller, H. C., Hartley, L., & Moore, J. C. (2003). The value of an emergent notion of authenticity: Examples from two student/teacher scientist partnership programs. Journal of Research in Science Teaching, 40, 737–756. Rowland, S., Pedwell, R., Lawrie, G., Lovie-Toon, J., & Hung, Y. (2016). Do we need to design course-based undergraduate research experiences for authenticity? CBE—Life Sciences Education, 15, ar79. Russell, C. B., & Weaver, G. C. (2011). A comparative study of traditional, inquiry-based, and research-based laboratory curricula: Impacts on understanding of the nature of science. Chemistry Education Research and Practice, 12, 57–67. Russell, S. H., Hancock, M. P., & McCullough, T. (2007). The pipeline: Benefits of undergraduate research experiences. Science, 31, 548–549. Shortlidge, E. E., Bangera, G., & Brownell, S. E. (2017). Each to their own CURE: Faculty who teach course-based undergraduate research experiences report why you too should teach a CURE. Journal of Micro- biology and Biology Education, 18, 1–11. Tamir, P. (1977). How are the laboratories used?Journal of Research in Science Teaching, 14, 311–316. Tamir, P., & Lunetta, V. N. (1978). An analysis of laboratory inquiries in the BSCS Yellow Version. American Biology Teacher, 40(6), 353–357. Weaver, G. C., Russell, C. B., & Wink, D. J. (2008). Inquiry-based and re- search-based laboratory pedagogies in undergraduate science. Nature Chemical Biology, 4, 577–580. Wolkow, T. D., Durrenberger, L. T., Maynard, M. A., Harrall, K. K., & Hines, L. M. (2014). A comprehensive faculty, staff, and student training program en- hances student perceptions of a course-based research experience at a two-year institution. CBE—Life Sciences Education, 13, 724–737.