Quick Tip: Wise Feedback

The article “Breaking the Cycle of Mistrust: Wise Interventions to Provide Critical Feedback Across the Racial Divide,” by Yeager et al. https://www.apa.org/pubs/journals/releases/xge-a0033906.pdf , introduces the concept of wise feedback; a strategy that helps instructors frame feedback in a way that communicates that students can meet high expectations and gives concrete direction for how to meet the expectations. 

The research comprised three studies focusing on middle and high school students. In the first study, students were divided into two groups: one receiving “wise feedback” and the other serving as a control. For the wise feedback group, comments on their essay drafts were paired with a note that said, “I’m giving you these comments because I have very high expectations, and I know that you can reach them.” In contrast, the control group’s note read, “I’m giving you these comments so that you’ll have feedback on your paper.” Students who received wise feedback were more likely to act on the suggestions and produced higher-quality revisions. This approach was particularly impactful for students from diverse backgrounds.

Adapting this to programming, the following steps can help provide effective feedback (from UC Berkley’s Greater Good in Education Center https://ggie.berkeley.edu/practice/giving-wise-feedback):

  • When reviewing a developer’s code or project, include the following in your feedback:

    • Start with specific actionable feedback:

      • “Your implementation meets the basic functionality, but I’ve added comments suggesting optimizations for improving performance and readability.”

      • “While your API design fulfills the requirements, the comments I left highlight ways to make the endpoints more efficient and user-friendly.”

    • State high expectations:

      • “This project is a step toward building production-ready code, which requires adherence to best practices for maintainability and scalability.”

      • “Writing clean and modular code here will prepare you for working on large, collaborative codebases in professional settings.”

    • Express confidence in the programmer’s ability to succeed:

      • “Based on your previous work, I’m confident you’ll be able to implement the suggested changes effectively.”

      • “Your problem-solving skills from earlier tasks show that you have what it takes to refine this code to meet higher standards.”

  • Additional tips to consider when using wise feedback in programming:

    • Use this type of feedback for tasks that represent meaningful learning opportunities and require the developer’s best effort, such as debugging complex issues or designing scalable solutions.

    • Incorporate this practice into a broader culture of trust, where developers feel valued and supported by their peers and mentors.

    • Avoid overpraising incomplete or substandard work, as this can unintentionally reinforce the belief that their efforts are not being taken seriously or that expectations are low.

    • While this approach benefits all developers, it is particularly impactful for those new to the field, who may struggle with imposter syndrome or worry about being judged unfairly based on stereotypes or biases.

This research emphasizes the importance of trust in the student-teacher relationship. It demonstrates that wise feedback, by directly addressing and mitigating mistrust, can significantly improve academic outcomes for underrepresented students. The findings have broad implications for educational practice by suggesting that through the lens of social-psychological theory, the study underscores how early interventions can create lasting positive effects on trust, motivation, and performance. Furthermore, the Science of “Wise Interventions” explores how social-psychological approaches can address educational disparities, reinforcing the effectiveness of strategies likewise feedback in fostering positive student outcomes: https://studentexperiencenetwork.org/wp-content/uploads/2018/08/The-Science-of-Wise-Interventions.pdf 

Integrating metacognitive and student engagement strategies into your practice

The article, How Changes in K-12 Schooling Hampered the Preparation of College Students” Published last month in the Chronicle of Higher Education,  explores how reforms in K-12 education, combined with broader societal changes and disruptions from the COVID-19 pandemic, and teacher shortages have affected the academic habits and preparedness of today’s college students, particularly Generation Z. When examining how test-based accountability has impacted curriculum design, policies such as “No Child Left Behind” have prioritized short-term performance over long-term learning.

Teaching became narrowly focused on test-taking skills, such as answering multiple-choice questions and writing formulaic essays, leaving students unprepared for broader academic tasks. “The [K-12] curriculum has been narrowed to mirror standardized tests. Students work on short passages to develop specific skills. Every class has a lesson, and every lesson has an assessment plan and an outcome. So when students arrive at college, they are conditioned to expect that same level of guidance. Ethan Hutt [an associate professor of education at the University of North Carolina at Chapel Hill], would like to give an assignment that goes like this: Pick a topic, write an essay, and say something interesting. But he finds he can’t — not even with his doctoral students”.

The article argues that, as a result, Students struggle with tasks like reading long texts and writing analytical essays because of a curriculum that favored short passages and responses. The shift away from holistic learning resulted in “atomized” education, where students are conditioned to expect detailed rubrics and granular guidance for assignments. Professors find that students enter college unprepared for the skills that are expected of them. Students often struggle with open-ended assignments, critical thinking, and engaging deeply with academic material. The article concludes that while K-12 reforms and the pandemic have played significant roles in shaping current students’ academic habits, broader systemic and cultural factors must also be addressed. Higher education inherits these challenges and must adapt to better support students’ academic and social transitions.

Based on the content of this article it may be helpful to integrate student engagement and metacognitive practices into first and second year classes such as integrating the Student Engagement Roster into your classroom practice:

SER Roster: The early feedback tool that allows faculty members to communicate with students about how they are doing in a class and make recommendations to improve or deepen their learning. SER also allows instructors to meet two faculty reporting obligations at IUB: Attendance verification and early evaluation grades. More information on those topics is available in the Help and Resources section.

Next week (1/20/2025) Submit SER with Attendance/Participation Feedback for:

  • Anyone not attending

  • Anyone already struggling to turn in work

  • Anyone doing a great job participating

Metacognitive Strategies

 

Think Aloud Exercises:

Anytime you can talk out loud (‘think aloud’) about how you view a document or a picture or programming an assignment, or share your thinking processes with students you are helping them become more metacognitive in their own approaches to the subject.  Once you have modeled for them how you would solve a problem or interpret a piece of writing, have students work in pairs to talk out loud as to how they are thinking about an assignment piece of homework or an assignment.

Concept Mapping: 

Learning benefits can be derived from instructor-led or student-constructed concept maps of the connections and key ideas from a course or class. It is best that the instructor demonstrate how to design a concept map of a class or course before students are asked to do the same. Show students how the readings, videos, assignments and activities are connected to the course learning outcomes and other courses.

Design a brief or detailed concept map of the course or sub-components of the course and share with students. Then later on in course students can form small groups and build a concept map as a review activity before a mid-term or as a review of a portion of the course. Students can do for homework or they can do in class and share with each other explaining the interrelationships between each component. Ask the students to draw all the ‘cross-links’ and label them as they see the components connecting fully or partially.

Guided Notes

Guided notes are structured outlines provided by an instructor that include key concepts, ideas, or topics, with spaces for students to fill in additional information during the lecture or while engaging with course content.

  • Lecture Notes: An instructor provides an outline of the lecture with prompts for students to add definitions, examples, or connections.

  • Reading Notes: A guided framework for analyzing a reading or resource, directing students’ attention to specific questions or themes.

  • Problem-Solving Frameworks: In mathematics or sciences, guided notes might include partial steps of a problem-solving process, leaving students to complete the solution.

This technique aligns with various teaching strategies, including scaffolding and universal design for learning (UDL), as it helps to bridge the gap between independent learning and instructor support.

More strategies and resources available here.

Teaching and Learning Conversation

Below are the extended notes and presentation from our conversation around Decoding the Disciplines with David Pace


Decoding Student Learning
📚

Decoding student learning is a framework for understanding where students are struggling and how to address those struggles. It’s a way of asking questions about your situation and systematically exploring it to find out what is needed.

What is Decoding Student Learning?

Decoding student learning is a strategy for identifying the bottlenecks in student learning and developing targeted interventions to address those bottlenecks.

The 7 Questions of Decoding Student Learning

The decoding student learning framework is based on 7 questions that help you understand where students are struggling and how to address those struggles. The questions are:

  • Where is student learning being bought? Where are the bottlenecks in student learning?

  • What exactly do students have to do to succeed? What are the specific skills or knowledge that students need to succeed?

  • What are the mental moves that students need to make? What are the cognitive processes that students need to use to succeed?

  • How do I model this for students? How can I show students how to do the things they need to do to succeed?

  • How do I provide feedback to students? How can I give students feedback that will help them improve?

  • How do I assess student learning? How can I measure student learning to see if they are succeeding?

  • How do I adjust my instruction? How can I adjust my teaching to better meet the needs of my students?

The Interview Technique

The interview technique is a method for gathering information about student learning. It involves asking students to describe a task or problem that they are struggling with, and then asking them to elaborate on their thought process.

Step

Description

1. Identify a task or problem

Ask students to describe a task or problem that they are struggling with.

2. Ask students to elaborate

Ask students to elaborate on their thought process and how they approach the task or problem.

3. Break down the process

Break down the process into smaller steps and identify the specific skills or knowledge that students need to succeed.

4. Model the process

Show students how to do the things they need to do to succeed.

5. Provide feedback

Give students feedback that will help them improve.

Example of the Interview Technique

For example, if you are teaching a history class and students are struggling to read and analyze primary sources, you might ask them to describe how they approach reading a primary source. They might say something like, “I just read the first sentence and try to remember the important things.” You could then ask them to elaborate on how they know what the important things are, and how they connect the information to the broader historical context.

Student Response

Follow-up Question

“I just read the first sentence and try to remember the important things.”

“How do you know what the important things are?”

“I look for keywords and try to connect them to the broader historical context.”

“How do you connect the information to the broader historical context?”

By using the interview technique, you can gain a deeper understanding of how students are thinking and what they need to succeed.

📚 Breaking Down Complex Tasks into Manageable Pieces

Breaking down complex tasks into smaller, manageable pieces is essential for effective learning. This process helps to identify the specific skills or knowledge that students need to acquire, making it easier to provide targeted support and feedback.

Identifying the Pieces

To break down a complex task, identify the individual components that make up the task. For example, when teaching students how to write a history paper, break down the task into smaller pieces such as:

  • Deciding on a topic

  • Conducting research

  • Analyzing evidence

  • Writing the paper

Providing Opportunities for Practice and Feedback

Once the individual components of the task have been identified, provide students with opportunities to practice each component and receive feedback. This can be done through a variety of methods, including:

  • Guided practice exercises

  • Peer review

  • Self-assessment

Assessing Student Progress

Assessing student progress is an ongoing process that involves evaluating student performance on each component of the task. This can be done through a variety of methods, including:

  • Formative assessments (e.g., quizzes, class discussions)

  • Summative assessments (e.g., exams, final projects)

The Importance of Repetition and Iteration

Repetition and iteration are essential components of the learning process. By repeating and refining their skills and knowledge, students can develop a deeper understanding of the material and improve their performance.

Bloom’s Taxonomy

“Bloom’s Taxonomy is a framework for categorizing educational learning objectives into levels of complexity and specificity. The taxonomy is divided into six levels: Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating.”

Level

Description

Remembering

Recall previously learned information

Understanding

Demonstrate comprehension of learned information

Applying

Use learned information to solve problems or complete tasks

Analyzing

Break down complex information into smaller parts

Evaluating

Make judgments about the value or quality of information

Creating

Generate new ideas or products

The Importance of Community and Collaboration

Learning is a social process that involves collaboration and community. By working together with peers and instructors, students can develop a deeper understanding of the material and improve their performance.

Upcoming Conference

There will be a conference in two weeks that will provide training and workshops on the topics discussed in this lecture. The conference will feature a variety of tracks, including a paper track, a workshop track, and a training track. https://decoding.webflow.io/events/decoding-conference-2024

📚 Decoding in Education 📚

Decoding is a teaching approach that helps students move up the pyramid of learning, from basic understanding to evaluation, analysis, and creation.

The Pyramid of Learning

Level

Description

Remembering

Recall previously learned information

Understanding

Demonstrate comprehension of learned information

Applying

Use learned information to solve problems

Analyzing

Break down complex information into smaller parts

Evaluating

Make judgments about the value or quality of information

Creating

Generate new ideas or products

The Importance of Decoding

Decoding helps students develop a deeper understanding of the material and move beyond basic recall. It also helps teachers identify areas where students need more guidance and support.

The Role of Decoding in Informatics Courses

In informatics courses, decoding is particularly important because it helps students develop problem-solving skills and think critically about complex information.

The Impact of Decoding on Student Learning

Decoding can have a significant impact on student learning, including:

  • Improved understanding of complex concepts

  • Increased ability to analyze and evaluate information

  • Enhanced problem-solving skills

  • Greater confidence in their ability to learn

The Relationship Between Decoding and AI

Research has shown that in order to effectively use AI tools, students need to have a strong foundation in decoding and problem-solving skills. This is because AI tools require students to ask good questions and provide clear instructions in order to produce accurate results.

The Challenge of Academic Misconduct

One of the challenges of using decoding in education is the potential for academic misconduct. When students are given the opportunity to use AI tools to complete assignments, they may be tempted to use these tools to cheat.

The Role of Emotional Resistance in Learning

Emotional resistance can play a significant role in the learning process. By breaking down complex tasks into smaller, manageable pieces, students can develop a sense of motivation and confidence, which can help to overcome emotional resistance.

Strategies for Overcoming Emotional Resistance

Emotional resistance can be a significant barrier to student learning. Strategies for overcoming emotional resistance include:

  • Breaking down complex tasks into smaller, more manageable steps

  • Providing clear instructions and expectations

  • Encouraging students to ask questions and seek help when needed

  • Creating a supportive and inclusive learning environment

The Importance of Making Steps Visible

Making steps visible is an important part of the decoding process. This involves breaking down complex tasks into smaller, more manageable steps and providing clear instructions and expectations.

The Role of Reflection in Decoding

Reflection is an important part of the decoding process. It involves taking the time to think about what you have learned and how you can apply it in different contexts.

The Relationship Between Decoding and Student Motivation

Decoding can have a significant impact on student motivation. When students are able to break down complex tasks into smaller, more manageable steps, they are more likely to feel a sense of accomplishment and motivation.

The Challenge of Logical and Mental Bottlenecks

Logical and mental bottlenecks can be a significant challenge in decoding. Strategies for overcoming these bottlenecks include:

  • Breaking down complex tasks into smaller, more manageable steps

  • Providing clear instructions and expectations

  • Encouraging students to ask questions and seek help when needed

  • Creating a supportive and inclusive learning environment

The Importance of Identifying Bottlenecks

Identifying bottlenecks is an important part of the decoding process. This involves taking the time to think about where students are getting stuck and how to provide additional support and guidance.

The Role of Algorithmic Thinking in Decoding

Algorithmic thinking is an important part of the decoding process. It involves breaking down complex tasks into smaller, more manageable steps and providing clear instructions and expectations.

Example of Algorithmic Thinking

For example, when writing a program, a student might need to think about the following steps:

  • What is the starting point of the program?

  • What is the goal of the program?

  • What are the steps needed to achieve the goal?

  • How will the program be implemented?

By breaking down the task into smaller, more manageable steps, the student is able to think more clearly and critically about the problem.

The Importance of Writing Down Steps

Writing down steps is an important part of the decoding process. This involves taking the time to think about the steps needed to complete a task and writing them down in a clear and concise manner.

Example of Writing Down Steps

For example, when writing a program, a student might write down the following steps:

  • Define the problem

  • Identify the inputs and outputs

  • Determine the algorithm

  • Implement the algorithm

  • Test the program

By writing down the steps, the student is able to think more clearly and critically about the problem and ensure that they are completing all of the necessary steps.## 📝 Understanding Student Learning and Development 📚

Metacognitive Skills and Learning Strategies

Metacognitive skills are essential for students to develop in order to become successful learners. These skills involve thinking about one’s own learning and understanding, and being able to adjust learning strategies accordingly.

“Metacognitive skills are the processes that help us plan, monitor, and evaluate our own learning. They are the ‘thinking about thinking’ skills that enable us to reflect on our own learning and make adjustments as needed.”

Some common metacognitive skills include:

  • Planning and goal-setting

  • Monitoring progress and adjusting learning strategies

  • Evaluating one’s own learning and understanding

  • Reflecting on learning experiences and identifying areas for improvement

The Importance of Teaching Metacognitive Skills

Teaching metacognitive skills is crucial for helping students develop a growth mindset and become more effective learners. By explicitly teaching these skills, educators can help students develop a deeper understanding of their own learning and become more self-directed.

“Teaching metacognitive skills is not just about teaching students how to learn, but also about teaching them how to think about their own learning. By doing so, we can help students develop a more nuanced understanding of their own strengths and weaknesses, and become more effective learners as a result.”

Strategies for Teaching Metacognitive Skills

There are several strategies that educators can use to teach metacognitive skills, including:

  • Modeling metacognitive thinking and behaviors

  • Providing opportunities for students to reflect on their own learning

  • Encouraging students to set goals and develop plans for achieving them

  • Teaching students how to evaluate their own learning and understanding

The Role of Improv Techniques in Teaching Metacognitive Skills

Improv techniques can be a useful tool for teaching metacognitive skills, particularly in the context of writing and communication. By using improv techniques, educators can help students develop a more flexible and adaptable approach to learning, and become more comfortable with taking risks and trying new things.

“Improv techniques can help students develop a more growth-oriented mindset, and become more comfortable with uncertainty and ambiguity. By using these techniques, educators can help students develop a more nuanced understanding of their own learning and become more effective communicators as a result.”

Assessing Student Learning and Development

Assessing student learning and development is a critical component of teaching metacognitive skills. By using a variety of assessment strategies, educators can help students develop a more nuanced understanding of their own strengths and weaknesses, and become more effective learners as a result.

Assessment Strategy

Description

Self-assessment

Students reflect on their own learning and understanding, and identify areas for improvement.

Peer assessment

Students provide feedback and guidance to one another, and help each other develop a more nuanced understanding of their own learning.

Formative assessment

Educators use a variety of strategies to assess student learning and understanding, and provide feedback and guidance to help students improve.

📚 Disciplinary Literacies and Accreditation 📚

Disciplinary Literacies refer to the specific skills and knowledge required to be successful in a particular field or discipline.

“Disciplinary literacies are the skills and knowledge that are specific to a particular discipline, and are required to be successful in that field.”

📊 ABET Accreditation 📊

ABET (Accreditation Board for Engineering and Technology) is an international organization that sets standards for engineering programs. To be accredited, programs must meet certain criteria, including:

Criteria

Description

Technical Skills

Students must demonstrate technical skills, such as programming and device creation.

Communication Skills

Students must demonstrate communication skills, such as teamwork and presentation.

Problem-Solving Skills

Students must demonstrate problem-solving skills, such as analysis and critical thinking.

📝 Learning Outcomes and Resume Building 📝

Learning Outcomes are specific skills and knowledge that students are expected to acquire in a course or program.

“Learning outcomes are the specific skills and knowledge that students are expected to acquire in a course or program.”

Resume Building is the process of highlighting skills and experiences on a resume to demonstrate value to potential employers.

  • Break down learning outcomes into specific skills and experiences that can be highlighted on a resume.

  • Use language from job postings and industry reports to describe skills and experiences.

📊 Example: Statistics Course 📊

A statistics course can be designed to teach students specific skills, such as data analysis and problem-solving.

Skill

Description

Data Analysis

Students must demonstrate the ability to analyze data and draw conclusions.

Problem-Solving

Students must demonstrate the ability to apply statistical methods to solve real-world problems.

💡 Real-World Applications 💡

Real-World Applications are examples of how skills and knowledge can be applied in real-world settings.

“Real-world applications are examples of how skills and knowledge can be applied in real-world settings.”

  • Use case studies and guest speakers to illustrate real-world applications of skills and knowledge.

  • Encourage students to reflect on how they can apply skills and knowledge in their own careers.

📈 Networking and Recognition 📈

Networking is the process of building relationships with others in a field or industry.

“Networking is the process of building relationships with others in a field or industry.”

  • Use networking opportunities, such as conferences and meetings, to build relationships with others in a field or industry.

  • Recognize and reward students and faculty for their contributions to a field or industry.

📊 Example: Recommendation System 📊

A recommendation system can be used to recognize and reward students and faculty for their contributions to a field or industry.

Criteria

Description

Quality of Recommendation

The quality of the recommendation is based on the expertise and reputation of the recommender.

Distance from Recommender

The distance from the recommender is based on the relationship between the recommender and the person being recommended.

The framework discussed in the lecture is a method for teaching and learning that focuses on breaking down complex tasks into smaller, manageable steps. This approach is useful for identifying bottlenecks and assumptions in the learning process.

What is the Hidden Curriculum? 🤔

The hidden curriculum refers to the unwritten rules and expectations that students need to understand in order to be successful in a class. This can include social interactions, academic underpinnings, and other implicit knowledge that is not explicitly taught.

In the context of the lecture, the hidden curriculum is related to the academic underpinnings that students need to know in order to be successful in a class. This includes the underlying concepts and skills that are necessary for understanding the material.

The 7 Steps 📝

The 7 steps are a key part of the framework, and are used to break down complex tasks into smaller, manageable steps. These steps are:

  • Identify the task or goal

  • Break down the task into smaller steps

  • Identify the key concepts and skills needed for each step

  • Determine the order in which the steps should be taught

  • Develop a plan for teaching each step

  • Implement the plan and assess student learning

  • Refine the plan based on student feedback and assessment data

Benefits of the Framework 📈

The framework has several benefits, including:

  • Helps to identify bottlenecks and assumptions in the learning process

  • Allows for a more iterative approach to teaching and learning

  • Helps to develop a plan for teaching each step of a complex task

  • Allows for refinement of the plan based on student feedback and assessment data

  • Can be used to identify areas where students are struggling, and to develop targeted interventions

Example: Web Design Course 📊

In a web design course, the framework can be used to break down the task of building a website into smaller, manageable steps. For example:

  • Identify the task: Build a website

  • Break down the task into smaller steps:

    • Design the layout

    • Write the HTML and CSS code

    • Add images and other media

    • Test and debug the site

  • Identify the key concepts and skills needed for each step:

    • Design principles

    • HTML and CSS syntax

    • Image editing software

    • Debugging techniques

  • Determine the order in which the steps should be taught:

    • Design principles

    • HTML and CSS syntax

    • Image editing software

    • Debugging techniques

  • Develop a plan for teaching each step:

    • Lecture and discussion on design principles

    • Hands-on exercise on HTML and CSS syntax

    • Tutorial on image editing software

    • Group project on debugging techniques

By using the framework, the instructor can develop a plan for teaching each step of the task, and can refine the plan based on student feedback and assessment data.## 📚 The Hidden Curriculum 📚

The hidden curriculum refers to the unwritten rules and expectations that govern behavior in academic settings. It encompasses the cultural capital, social norms, and unspoken assumptions that students are expected to understand and navigate in order to succeed.

“Cultural capital” refers to the social assets, such as knowledge, skills, and connections, that provide individuals with an advantage in navigating social and academic environments.

🤔 Decoding the Hidden Curriculum 🤔

Decoding the hidden curriculum involves identifying and making explicit the unwritten rules and expectations that govern academic behavior. This process can help students navigate academic environments more effectively and reduce feelings of confusion and uncertainty.

Decision Points in the Hidden Curriculum

Decision Point

Description

Is this essential to know now, or can I just wait to be clear?

Students must decide whether to seek clarification on a concept or wait until later.

Is it okay to raise my hand in this class?

Students must decide whether to ask questions in class or seek help outside of class.

Is it okay to ask a question after class?

Students must decide whether to approach the professor after class to ask questions.

📝 The Importance of Advising 📝

Advising is a critical component of the hidden curriculum. Advisors can help students navigate academic environments, make informed decisions, and develop the skills and knowledge needed to succeed.

“Advising is teaching” – a theoretical concept that emphasizes the importance of advising in academic settings.

📊 Research on the Hidden Curriculum 📊

Research on the hidden curriculum has shown that students who are aware of the unwritten rules and expectations that govern academic behavior are more likely to succeed. Studies have also highlighted the importance of advising and mentoring in helping students navigate academic environments.

Program to Decode the Hidden Curriculum

Program Component

Description

Interviews with students and faculty

Identify bottlenecks and areas of confusion in the hidden curriculum.

Development of common language

Create a shared understanding of the hidden curriculum among students and faculty.

Identification of key decision points

Determine the critical decision points in the hidden curriculum that students must navigate.

Sample Questions for Anonymous and Mid-semester Feedback Surveys

We have discussed the importance of getting midsemester feedback from students. Mid-semester feedback lets instructors “check the pulse” of their class partway through the semester. By surveying students, especially after a chance to evaluate evidence of learning, instructors and students alike can gain practical and actionable insights about what is working as well as recommendations for improving learning and teaching.

In addition to the general questions provided in this blog post, you may want to ask your students more specific questions about your course goals, inclusive teaching practice, classroom dynamics, pacing, and content. Curating your own questions enables you to gather targeted student feedback and be responsive to that feedback through adjustments to the course to benefit students and their learning for the rest of the semester. The following questions were developed by Poorvu Center for Teaching and Learning and the University Center for Teaching and Learning.

General Questions (pre-populated in Canvas) 

  • What is helping your learning in this class?

  • What is hindering your learning in this class?

  • What could the instructor change to improve your learning experience in this class?

  • What could you do differently to improve your learning experience in this class?

Inclusive Teaching Questions 

  • Does the instructor make you feel that you belong in the class?  What suggestion do you have for promoting a greater sense of belonging?

  • How comfortable do you feel participating in the class? Is there anything that the instructor could do to encourage more equitable participation among all of the students?

  • What does the instructor do to foster mutual respect among students?  What else could the instructor do?

  • Does the instructor teach in a manner that supports students like you to succeed? In what ways?

  • How accessible is the instructor outside of class to support you in your learning?

  • Are you able to access all course materials?  Do you have any suggestions for making materials more accessible?

Course Materials & Goals Questions 

  • Does the instructor clearly communicate course expectations?  Do you have any suggestions for improvement?

  • To what extent are the course materials valuable? Which of the materials–used in class or assigned out of class–have been most useful and least useful to your learning?

  • How has the professor’s use of materials (readings, slides, multi-media such as podcasts or films, etc.) helped you understand the subject of this course? What changes might help you learn better?

  • Is the volume of course material appropriate? Student-Learning Focused Questions

  • What are the most significant things you have learned thus far from taking this course?

  • How does the instructor help you understand what are the most important points within class? Could they do anything differently?

  • Do you feel as though your performance in the class is a reflection of your understanding of course material?  Why or why not?

  • Do you have any suggestions to improve the quality of feedback that you are getting on your understanding of course material?

Examples of questions to solicit targeted feedback on specific or new aspects of the course:

  • Which (Class activity, assessment, course material, technology tool) has been most beneficial to your learning and why?

Classroom Dynamics & Pacing Questions 

  • How does your instructor engage your attention? Are there other approaches that would make you feel engaged in the class?

  • What types of opportunities do you have to interact with the instructor and the other students in the course?  How are these interactions supporting your learning and improving your experience in the course?

  • How is the pacing of this course for you?  Do you have any suggestions for improvement?

Likert Questions If you teach a large course, reviewing qualitative questions from many students may not yield clearly defined feedback. Instead, consider using Likert scale questions, which can’t be used in the midsemester feedback tool on Canvas but can be used in various other survey options. When viewing your results, consider looking at the median and mode responses and a histogram of all of the responses, as to not allow outliers to skew your interpretation.

Below are some questions that could mostly be answered on a 1 to 5 Likert scale with 5 being strongly agree and 1 being strongly disagree.

Inclusive Teaching Questions 

  • The instructor makes me feel that I belong in the class.

  • I am able to easily get answers to questions about course material.

  • The instructor fosters mutual respect among students.

  • The instructor of the class is accessible for questions or concerns.

  • The course materials are all accessible.

Course Materials & Goals Questions 

  • The instructor communicates course expectations clearly.

  • I find the course materials to be valuable.

  • The volume of course material is appropriate.

Student-Learning Focused Questions 

  • The instructor helps me understand what the most important points within class are.

  • I feel as though my course performance is an accurate reflection of my understanding of course material.

  • I am getting enough feedback on my understanding of course material.

Classroom Dynamics & Pacing Questions 

  • I am engaged during class.

  • The level of interaction in the course is supporting my learning.

  • The pace of the course is: 1- much too fast, 2- a little too fast, 3- just right, 4- a little too slow, 5- much too slow

Examples of questions to solicit targeted feedback on specific or new aspects of the course:

  • Would you prefer more or less (discussion? lecture? small group work? presentations?)

  • Which (Class activity, assessment, course material, technology tool) contributed to your learning?

Evidence Based Research Supporting the Use of PlayPosit in classes

PlayPosit https://app.teaching.iu.edu/tools/playposit is an interactive teaching tool/application used to make interactive videos, also known as bulbs. The application can be integrated into the Canvas using your own videos or by extracting videos from other sources, such as YouTube or TED Talks, and convert them into an interactive, topic- and student-focused mini-lessons by adding questions at appropriate time-points in the video. Educators may also prepare video clips from longer recorded lectures and encourage student engagement by providing context or additional information through text and images on slides, inserting questions to check for understanding, including discussion and reflection questions, giving pre-recorded feedback as they see fit, and even incorporating polling. PlayPosit videos pause at certain intervals chosen by the instructor to give students an opportunity to respond.

PlayPosit settings allow for multiple playback options, including allowing students to rewind, fast forward or retake the activities. The instructor may also incorporate instant feedback after each question; so, students not only see whether they answered correctly but also understand the rationale. This feedback enables instructors to modify instructional activities midstream in light of their effectiveness, impact, and value. Because formative evaluations are designed to guide the teaching process – and are not used as outcome indicators – they are generally individualized evaluations that are under the control of the instructor and target specific instructional topics, issues or concerns.

van der Meij, H., & Bӧckmann, L. (2021). Effects of embedded questions in recorded lectures. Journal of Computing in Higher Education, 33(1), 235–254. https://doi.org/10.1007/s12528-020-09263-x

  • Researchers sought to examine the effectiveness of providing “open-ended embedded questions” in recorded video lessons to better prepare students for upcoming in-class topics.

  • Student surveys, user logs, and knowledge tests showed that learners “engaged significantly more with the embedded questions lecture” and showed “significantly higher” average scores on those topics. Researchers concluded that embedded questions “can increase the effectiveness of online video-recorded

     lectures.”

Lewandowski, H.J., Pollard, B., West, C.G. (2020). Using custom interactive video prelab activities in a large introductory lab course. 2019 Physics Education Research Conference Proceedings. https://doi.org/10.1119/perc.2019.pr.Lewandowski

  • As part of a broader redesign of a large introductory physics course at the University of Colorado Boulder, researchers developed pre-lab videos with embedded PlayPosit questions to help students better prepare for in-person lab activities.

  • Researchers found that 90% of students completed the pre-lab video modules, with the “vast majority of students spending a nontrivial amount of time engaging with each question.” 80% of students felt the activities prepared them well for in-person lab tasks.

Sherifi, D., Jia, Y., Hunt, T. J., & Ndanga, M. (2023). Evaluation of a PlayPosit guided group project’s impact on student engagement in an undergraduate course. Discover Education, 2(1), 32. https://doi.org/10.1007/s44217-023-00057-8

  • Students appreciated that PlayPosit activities were enjoyable and different from other learning resources. Eleven students expressed that PlayPosits were “enjoyable, “short”, “quick”, “not overbearing”, “not drawn out or boring”, “made learning fun”, “a unique way of learning”, and “kept students interested”.

  • PlayPosit increased the teaching presence of the professor by virtue of having multiple touch points with the material focused on relevant tasks.

  • PlayPosit contributed to better interaction of the students with the course content, and as per their comments, was helpful and beneficial, as well as interesting and attractive. Furthermore, students were more attentive to the other course videos and recordings.

Karpicke, J.D. (2012). Retrieval-based learning: Active retrieval promotes meaningful learning. Current Directions in Psychological Science, 21(3), 157–163. https://doi.org/10.1177/0963721412443552

  • Researchers compared the effects of different study strategies on student learning between three test groups in the same course: one studying concepts by rereading materials with no recall activities, one mostly rereading with some recall activities, and one reading just once with more recall activities.

  • While surveyed students believed rereading would be the most effective study strategy, those who simply reread the same materials with no recall activities performed poorest on assessments. Those who practiced just one retrieval activity “doubled long-term retention,” and those who practiced multiple retrieval activities showed even larger gains

For more information on how to integrate PlayPosit into your course, please visit https://app.teaching.iu.edu/tools/playposit