Center for Teaching and Learning

Home / Inclusive Teaching / Evidence-Based Teaching Practices

Evidence-Based Teaching Practices

Nine Evidence-Based Teaching Practices to Combat Systemic Inequities in Hybrid or Online Courses

Brandeis’ distinctive leadership role in higher education requires that we apply the principles of access and social justice upon which Brandeis was founded in the present educational context that is partially or completely online.

Equity gaps are more pronounced in an online educational environment than in a face-to-face classroom. [Gavassa et al. 2019, Howard et al. 2019, Verschelden 2017, Xu et al. 2014].

Members of our Brandeis teaching community can provide equitable opportunities for all our students to engage by adopting evidence-based practices that reduce equity gaps in a partially or completely online educational context.

Nine of those practices are listed here, followed by an annotated bibliography with full citations.

Expand All

1. Brandeis principles of equity and access demand highly structured hybrid courses

Visible, clear structure in a hybrid course improves exam performance for traditionally underrepresented students, closing the achievement gap even while in-person contact hours are reduced.
Alignment of learning goals, class activities, assignments and assessments can reduce achievement gaps.
Structure students' learning to build their intellectual skills in a sequence. Include lower-order learning opportunities (understand, analyze) as well as higher-order learning (evaluate, create).
Clarity of expectations and instructions can reduce achievement gaps.
Students benefit from understanding how we structure their skill development and knowledge acquisition, and how we offer feedback.
Short recorded lessons (15 min. max) with prior focus questions and immediately subsequent test/quiz questions (low stakes, formative) reinforce learning more than later studying.
"As Brandeis seeks to foster a campus community made stronger by diversity in all forms, it must create more-structured opportunities for students to engage across difference — including differences in socioeconomic background, interests, identity, politics and beliefs."

(Anderson/Krathwohl 2000, Babb 2018, Brandeis 2020, Bransford 2000, ch 2, Froyd 2008, Gavassa 2019, Singer-Freeman 2019, Tate 2004, Tileston 2004, Verschelden 2017, Weiman 2014, Winkelmes 2019)

2. Brandeis student/faculty engagement and student/student collaborative learning are essential in a partially online learning/teaching context

Synchronous online community-building sessions at the beginning of the course support Brandeis student engagement/community/belonging more easily than asynchronous early sessions.
Small working/study/discussion groups with TAs or faculty (organized around time zones) can encourage Brandeis student engagement, collaboration and community. Groups can share online notes or projects with the whole class.
- (Hybrid Instruction and IT Support Working Group 2020)
Establishing a social contract with students at the beginning of the course is essential in online and hybrid environments.
- (Joel Cutcher-Gershenfeld -Heller)
Together, these two types of connectivity — vertical and horizontal — define the Brandeis academic value proposition:
1. The mix of cutting-edge research, scholarship and creative work on the one hand and a deep commitment to undergraduate liberal arts education on the other fosters a unique learning environment for Brandeis students. It allows for a degree of "vertical connectivity" not found at other Research-1 universities or liberal arts colleges.
2. The university's collaborative ethos also extends to faculty across the curriculum, creating an unusual degree of "horizontal connectivity." These connections allow faculty, staff and students to transcend programmatic, departmental and school affiliations in their academic pursuits — a kind of collaboration that creates dynamic learning and research opportunities. Such connectivity has led to the introduction of innovative cross-school majors and programs, and provides an institutional nimbleness as the Brandeis curriculum evolves to meet the changing needs of students.
  - (The Framework for the Future 1/2020)
Brandeis... will... recognize and count the mentoring of undergraduate research as part of faculty members' teaching assignments... consistent with the Brandeis academic value proposition — with smaller classes and greater opportunities for collaborative work.
- (Framework… 1/2020)
Frequent faculty feedback with faculty/student interaction can encourage students.
- (Babb 2018, Chickering 1987, Cummings 2020, Ishiyama 2002)
Student-faculty contacts are an essential part of successful learning and teaching.
- (Ambrose 2012 Appendix E, Babb 2018, Wieman 2014)
Identify and engage preconceptions: meet students where they are.
- (Bransford 2000, Hattie 2015, Mazur, Eddie, Winkelmes 2019)
Peer instruction increases understanding efficiently and encourages positive attribution activities, and knowing this will enhance students' peer learning experiences.
- (Ambrose 2010, Babb 2018, Froyd 2008, Light, Weiman 2014, Winkelmes 2019)
In STEM courses, average examination scores improved by about 6% in active learning sections, and that students in classes with traditional lecturing were 1.5 times more likely to fail than were students in classes with active learning.
- (Freeman 2014)
Active learning appears effective across all class sizes — although the greatest effects are in small (n ≤ 50) classes.
- (Freeman 2014)
Brandeis will increase collaborative learning opportunities for students.
- (Framework… 1/2020)

3. Assessment must be equitable.

Equitable assessments are characterized by: alignment with course knowledge and skills, clarity of processes and criteria for grading, testing environments that decrease stereotype threat, inclusive content and focus on applicability of tested knowledge and skills.

Varied assessment measures for students' learning can enhance equity. Short-answer, low-stakes quizzes after each lesson, opportunities to do work, receive feedback and revise are essential to successful learning.
- (Fiske/Light 1990)
Immediately after a lecture, taking a low-stakes short answer test improved long-term final recall more than studying a lecture summary or taking a multiple choice test.
- (Butler 2007)
Frequent faculty feedback with faculty/student interaction encourages students.
- (Babb 2018, Chickering 1987, Cummings 2020, Froyd 2008, Ishiyama 2002, Wieman 2014)

4. Inclusive, accessible, culturally aware instruction reduces stereotype threats and intersectional inequities in learning experiences.

(Babb 2018, Chickering 1987, Ishiyama 2002, Singer-Freeman 2019, Verschelden 2017, Wieman 2014)

Diversity and inclusion are "not solely about race and socioeconomics but are also about our growing international student body, our transgender and nonbinary communities, and our students with disabilities."
- (Brandeis 2020)
…the university was grounded by a deep ethical core, motivated by "a highly sensitive social consciousness, the concern for the underdog and the resistance to any kind of discrimination or privilege."
- (President Sachar, quoted in Brandeis 2020)
Universal design principles offer accessibility for students with disabilities and learning differences as well as learning benefits for all students.
- (Ableser and Moore 2020)

5. Communicating high expectations, encouragement and support correlates with higher student achievement

Teachers' expectations, encouragement and support toward students influences students' attitude toward online learning and their performance.
Sharing a growth mindset with students (intelligence is malleable and not fixed) is an effective way to convey encouragement and increase students' self-efficacy and success.
Syllabus language should convey clarity, a path to success and approachability of the instructor.
- (Niari 2016, Froyd 2008, Ishiyama 2002, Yeager/Dweck 2012)

6. Communicating with students about the purposes, tasks and criteria for their learning before they begin working reduces achievement gaps, supports student persistence, and helps mitigate the negative effects of the online classroom for underserved students.

Explicating the applicability and relevance of academic work increases students' potential to succeed and maximizes cognitive bandwidth.
Students benefit from understanding how we structure their skill development and knowledge acquisition and how we offer feedback.
- (Anderson/Krathwohl 2000, Bransford 2000 ch 2, Howard 2019, Singer-Freeman 2019, Verschelden 2017, Winkelmes 2019, Wieman 2014)

7. Teaching students about learning skills and structuring metacognition exercises for them is as important for student’s mastery as teaching about the course content.

Metacognition improves learning. It includes: 1) planning, 2) implementing/monitoring, 3) reflecting/revising.
- (Nilson 2013, Tanner 2012)
Weekly metacognitive reflection heightens awareness of learning equitably so students clearly understand the value of what they are learning and how they can apply it beyond the course.
- (Ambrose 2010, Butler 2007, Fiske/Light 1990, Froyd 2008, Mosteller, Singer-Freeman 2019, Smith 2014, Wieman 2014, Winkelmes 2019)
Making the discussion of metacognitive knowledge part of the everyday discourse of the classroom helps foster a language for students to talk about their own cognition and learning.
- (Tanner 2012)
Connect to students' prior understanding to eliminate misconceptions that confuse learning and metacognition.
- (Ambrose 2010, Bransford 2000, Froyd 2008, Yeager/Dweck 2012, Weiman 2014)
Teach students to regularly monitor and adjust their approaches to learning.
- (Ambrose 2010)

8. Explicitly inform students about what success looks like near the start of a series of lessons to reduce equity gaps. (Hattie 2015)

Considering multiple examples of criteria in practice, before students begin working, helps students to begin their work at the same starting line of understanding about expectations
- (Winkelmes 2019)
When faculty communicate transparently in conversation with students about the purposes, tasks and criteria (with multiple examples) for their upcoming work before students begin the work, achievement gaps decrease while students' academic confidence, sense of belonging in college, metacognitive awareness of their learning and persistence all increase.
- (Winkelmes 2019)

9. One of the most statistically significant contributors to student success is collaboration among faculty who examine the impact of their teaching and who research ways to improve teaching and learning.

"… we seek to... foster the development and dissemination of new forms of scholarship and approaches to teaching."
- (Brandeis 2020)
Teachers working together to know and evaluate their impact on student learning is among the very largest predictors of student success. (Online learning analytics enable this sharing in an online context.)
- (Froyd 2008, Hattie 2015, Weiman 2014)

A Brief, Annotated Bibliography

Ablesar, J. and Moore, C. (2018). Universal Design for Learning and Digital Accessibility: Compatible Partners or a Conflicted Marriage? Educause Review.
- Universal design for learning (UDL) and digital accessibility both seek to increase learning access and reduce barriers for students. As such, their primary goals are compatible and are widely acknowledged as crucial to ensuring equity in education.
Ambrose, S.A., Bridges, M.W., DiPietro, M., Lovett, M.C., & Norman, M.K. (2010). The Jossey-Bass higher and adult education series. How learning works: Seven research-based principles for smart teaching. Jossey-Bass. (ch. 5, Feedback; Appdx E)
- Uncover and connect to students' prior understanding/knowledge.
- Structure students' learning in a sequence: lesson followed by application followed by feedback followed by revision.
- Incorporate small tests/quizzes/activities where students frequently retrieve and review knowledge.
- Scaffold into a sequence of activities the learning exercises to help students develop not only the component skills and knowledge necessary to perform complex tasks, they must also practice combining and integrating them to develop greater fluency.
- Explain the value of each learning activity before students engage in it.
- Students' motivation and persistence depends upon their clear recognition of the value of each learning activity, their awareness of what and how they are learning, their understanding of how to usefully apply their learning.
- Teach students to regularly monitor and adjust their approaches to learning.
Anderson, L, Krathwohl, D. (2000). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. [Bloom. B. (1956). Taxonomy of educational objectives. New York: David McKay.]
- Structure students' learning to build their intellectual skills in a sequence. Include lower-order learning opportunities (understand, analyze, synthesize) as well as higher-order learning (evaluate, create).
Babb, S. ; Stewart, C. ; Johnson, R. (2018). Applying the seven principles for good practice in undergraduate education to blended learning environments. Online Course Management: Concepts, Methodologies, Tools, and Applications, Vol. 3, pp.1102-1124. [Chickering, Arthur W.; Gamson, Zelda F. Seven Principles for Good Practice in Undergraduate Education. AAHE Journal (March 1987): 3-7.]
- … these principles have been repeatedly tested in online and traditional courses, and shown to be effective at meeting learning outcomes. In order to apply the seven principles to hybrid courses, instructors should: 1) encourage contact and communication between themselves and the students; 2) provide multiple opportunities for interactions between students; 3) create well-designed student-presented projects and assignments that require participation, engagement, and feedback; 4) provide prompt feedback on both assignments and inquiries; 5) establish both deadlines and expectations for time spent on learning assessments; 6) communicate high expectations for the course through a well-designed syllabus, challenging assignments, and praise for excellent performance; and 7) allow students autonomy in assignment topics.
Brandeis University (2020). The Framework for the Future (Final Report). January, 2020. Retrieved from Brandeis University website.
Bransford J, Brown A, Cocking R, Donovan SM, Pellegrino J (2000). How People Learn: Brain, Mind, Experience, And School. Expanded ed. Washington, D.C.: National Academy Press, 2000.
Bruff, Derek. (2019). Intentional Tech: Principles to Guide the Use of Educational Technology in College Teaching.
- Learning goals should guide how teachers use technology. For example, technologies can support connection in social learning communities or help teachers to conduct real-time formative assessment to see how students are learning and inform adjustments to teaching. Ensure students have equitable access to a technological tool before adopting it for class use.
Butler, A.C., & Roediger III, H.L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19(4-5), 514-527. [Smith, Megan A, and Jeffrey D Karpicke. "Retrieval Practice with Short-answer, Multiple-choice, and Hybrid Tests." Memory 22.7 (2014): 784-802.]
- Short, low-stakes testing after a lecture or lesson can improve knowledge retention. Which kind of test is most effective? Immediately after a lecture, taking a low-stakes short answer test improved long-term final recall more than studying a lecture summary or taking a multiple choice test did.
Du, J., Zhao, M., Xu, J., Lei, S. (2016). African American female students in online collaborative learning activities: The role of identity, emotion, and peer support. Computers in Human Behavior, vol 63, pp, 948-958.
- This qualitative study examined African American female students’ online collaborative learning.
- African American women perceived peer supports as a give-and-take process. African American women viewed group member role as a formation of identity. “Frustration” was the common response to peer participation and interaction.
Dziuban, C., Graham, C.R., Moskal, P.D. et al. (2018). Blended learning: the new normal and emerging technologies. International Journal of Educational Technology in Higher Education 15, 3 (2018).
Freeman, S., Eddy, S. L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H., & Wenderoth, M.P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415.
- We meta-analyzed 225 studies that reported data on examination scores or failure rates when comparing student performance in undergraduate science, technology, engineering, and mathematics (STEM) courses under traditional lecturing versus active learning. Average examination scores improved by about 6% in active learning sections, and that students in classes with traditional lecturing were 1.5 times more likely to fail than were students in classes with active learning. ... active learning appears effective across all class sizes — although the greatest effects are in small (n ≤ 50) classes.
Friday, E., Friday-Stroud, S.S., Green, A.L., Hill, A.Y. (2006). A multi-semester comparison of student performance between multiple traditional and online sections of two management courses. Journal of Behavioral and Applied Management, 8(1), 66-81.
- This multi-semester (eight semesters), multi-course study compared student performance in undergraduate online and traditional sections of "Organization and Management" with sample sizes of 380 and 213, respectively. … this study found no statistically significant difference in student performance between online and traditional classes in both management courses after examining eight semesters of data. However, this study found gender differences with both management courses
Froyd J (2008). White Paper on Promising Practices in Under- graduate STEM Education, Commissioned Papers, Washington, DC.
- Offers research-based promising practices, including: explicate learning goals, organize students in small groups and in cohorts, organize a course by questions or problems, provide formative feedback to students, engage students in active learning exercises in groups, involve students in research, teachers research STEM learning and teaching practices together.
Gavassa, S., Benabentos, R., Kravec, M., Collins, T., Eddy, S. (2019). Closing the Achievement Gap in a Large Introductory Course by Balancing Reduced In-Person Contact with Increased Course Structure. CBE - Life Sciences Education, Vol.18(1).
- We designed, taught, and assessed a fully online course (100% online) and a hybrid-and-flipped course (50% online 50% face-to-face) and compared those formats with a lecture-based face-to-face course. The three formats also varied in the degree of structure; the hybrid course was the most structured and the face-to-face course was the least structured. All three courses were taught by the same instructor in a large Hispanic-serving research university. We found that exam scores for all students were lowest in the face-to-face course. Hispanic and Black students had higher scores in the hybrid format compared with online and face-to-face, while white students had the highest performance in the online format. We conclude that a hybrid course format with high structure can improve exam performance for traditionally underrepresented students, closing the achievement gap even while in-person contact hours are reduced.
Hattie, J. (2015). The Applicability of Visible Learning to Higher Education. Scholarship of Teaching and Learning in Psychology. American Psychological Association. Vol. 1, No. 1, 79 –91.
- The Visible Learning research is based on a synthesis of 1,200 meta-analyses relating to influences on achievement. This article focuses specifically on the evidence and implications for higher education teachers. As nearly every intervention can show some evidence of success, we need to ask not "What works?" but "What works best" and seek comparisons between different ways of influencing student learning. The major implications relate to teachers who work with others to seek evidence of their impact on students, who inform students early what success looks like especially about surface and deep learning, who provide appropriate levels of challenge and feedback, and who have aligned their claims about success, assessment and teaching.
Howard, Tiffiany, Mary-Ann Winkelmes, and Marya Shegog. “Transparency Teaching in the Virtual Classroom: Assessing the Opportunities and Challenges of Integrating Transparency Teaching Methods with Online Learning.” Journal of Political Science Education, June 2019.
- The existing literature on transparency teaching in higher education reveals that the adoption of transparent-oriented assignments improves the learning outcomes for underserved students at the introductory level, and decreases the rate of attrition among the student population with the highest risk of dropping out. The goal of this study is to establish that transparent teaching methods help mitigate the negative effects of the virtual classroom for underserved students.
Hybrid Instruction and IT Support Working Group (2020). Virtual meeting 5/6/2020, Brandeis University Zoom platform, online)
Ishiyama, J., & Hartlaub, S. (2002). Does the Wording of Syllabi Affect Student Course Assessment in Introductory Political Science Classes? PS: Political Science & Politics, 35(3), 567-570.
- Syllabus language should convey clarity, a path to success, and approachability of the instructor.
Lave, J., & Gomes, A. (2019). The Long Life of Learning in Practice. Learning and Everyday Life: Access, Participation, and Changing Practice (pp. 1-9). Cambridge: Cambridge University Press.
- Considers the politics of knowledge that inform theories of learning, and to reconceive learning in/as transformation, and as itself always a cultural/historical practice
Light, R. (1990). Harvard Assessment Seminars. 2 vols. [Fiske, E. (March 5, 1990). How to Learn in College: Group Study, Many Tests. p. A1.]
- Peer learning in groups, frequent low-stakes quizzes, opportunities to do work, receive feedback and revise are essential to successful learning.
Niari, M. (2016). The Pygmalion Effect in Distance Learning. European Journal of Open, Distance and E-Learning, 19 (1). [Rosenthal, R. ; Dweck, C.]
Teachers' expectations, encouragement and support toward students influences students’ attitude toward online learning and their performance. Sharing a growth mindset with students (intelligence is malleable and not fixed) is an effective way to convey encouragement and increase students' self-efficacy and success.
Nilson, L. B. (2013). Creating self-regulated learners: Strategies to strengthen students' self-awareness and learning skills. Sterling, VA: Stylus.
- Teaching students about learning skills and structuring metacognition exercises is as important for student’s learning as teaching about the course content.
NRC (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering, Washington, DC: National Academies Press.
Singer-Freeman, K., Hobbs, H., Robinson, C. (2019). Theoretical Matrix of Culturally Relevant Assessment. Assessment Update: Progress, Trends, and Practices in Higher Education, vol. 31, n. 4, pp. 1-3.
- To achieve educational equity, we must examine the equity of assessments. Equitable assessments are characterized by: alignment with course knowledge and skills, clarity of processes and criteria for grading, testing environments that decrease stereotype threat, inclusive content, focus on applicability of tested knowledge and skills.
Soffer, T., Nachmias, R. (March 2018). Effectiveness of learning in online academic courses compared with face-to-face courses in higher education. Journal of Computer Assisted Learning.
- This study examined the effectiveness of 3 online courses compared with the same 3 courses in a face‐to‐face (F2F) format, which had the same characteristics (e.g., the same instructor and final exam content and place). Students in the online courses reported better understanding of the course structure, better communication with the course staff, watching the videos lessons more, and higher engagement and satisfaction. Students in the F2F courses reported better contribution of the learning content. Students' final grades were higher in the online courses, and no differences were found in the completion rate. The findings suggest that in many of the examined effectiveness aspects, online courses are as effective as, or more effective than, F2F courses.
Tanner, K. (2012). Promoting Student Metacognition. CBE Life Sci Educ. 2012 Summer; 11(2): 113–120.
- Making the discussion of metacognitive knowledge part of the everyday discourse of the classroom helps foster a language for students to talk about their own cognition and learning.
Tanyel, F., Griffin, J. (2014). A 10-year comparison of outcomes and persistence rates in online versus face-to-face courses. B> Quest, 1-22.
- The purpose of the research reported on in this article was to compare student outcomes for online versus face-to-face sections of courses matched by course number and instructor for a ten-year period following the introduction of online courses at a small-sized, southeastern regional state university. Results indicated a +12% difference in the percent of students receiving credit for the course and +.15 higher average course GPA (on a 4.0 scale) favoring the F2F format. Longitudinal analyses indicated that as online sections of courses were offered in more disciplines by more instructors to more students, the differences in GPA became apparent. These results are discussed in terms of the potential unintended effects of taking an online version of a course on the hour and GPA continuation requirements for keeping state scholarships.
Tate, M. L. (2012). Sit and Get Won't Grow Dendrites: 20 Professional Learning Strategies That Engage the Adult Brain (2nd ed.). Thousand Oaks, CA: Corwin Press.
- Structure active learning exercises that engage students regularly in applying each newly acquired lesson.
Verschelden, C. (2017). Bandwidth Recovery: Helping Students Reclaim Cognitive Resources Lost to Poverty, Racism, and Social Marginalization. Sterling, VA: Stylus. [Verlazzo, L. (May 2019) Author interview: Enhancing the Mental Bandwidth of Students. Education Week Teacher.]
- Equitable educational practice requires reducing the impact on students' learning of structural, intersectional social inequalities that crowd their bandwidth for focusing, and accommodating all learners.
Weiman, C. and Gilbert, S. (2014). The Teaching Practices Inventory: A New Tool for Characterizing College and University Teaching in Mathematics and Science CBE — Life Sciences Education Vol. 13, 552–56
- A validated rubric of STEM teaching practices that benefit STEM students' learning includes: structured organization, reducing cognitive load by sequencing/scaffolding preparation and practice, supporting students' motivation, structured opportunities to practice and get feedback, reflecting on learning, self-evaluation of learning strategies, teachers collaborate and share teaching practices.
Winkelmes, M.A. et al. (2019). Transparent design in higher education teaching and learning: A guide to implementing the transparency framework institution-wide to improve learning and retention. Sterling, VA: Stylus. (ch. 1, pp. 17-33).
- When faculty communicate in conversation with students about the purposes, tasks and criteria (with multiple examples) for their upcoming work, achievement gaps decrease while students’ academic confidence, sense of belonging in college, metacognitive awareness of their learning and persistence all increase.
Xu, Di, and Shanna S Jaggars. "Performance Gaps between Online and Face-to-Face Courses: Differences across Types of Students and Academic Subject Areas." The Journal of Higher Education 85.5 (2014): 633-59.
- Using a dataset containing nearly 500,000 courses taken by over 40,000 community and technical college students in Washington State, this study examines the performance gap between online and face-to-face courses and how the size of that gap differs across student subgroups and academic subject areas. While all types of students in the study suffered decrements in performance in online courses, those with the strongest declines were males, younger students, Black students and students with lower grade point averages.

Resources

For additional scholarship of teaching and learning (SoTL), see:

Evidence-Based Teaching Practices

Nine Evidence-Based Teaching Practices to Combat Systemic Inequities in Hybrid or Online Courses

A Brief, Annotated Bibliography

Resources

Compiled by Mary-Ann Winkelmes and Brad Wheeler