Book a consultation
Email Action Unread Streamline Icon: https://streamlinehq.com
Learning Science

The Impact of AI Feedback in Applied Learning

By Team Multiverse

|
Arrow Left Streamline Icon: https://streamlinehq.com
See all posts

Contents

  1. A Comparative Study: Traditional Coaching Methods vs AI-feedback
  2. Study Details

We know effective learning hinges on feedback (Hattie & Temperley, 2007). Yet, it is time consuming, requires the right expertise and needs to be timely in order to work. At Multiverse, we have investigated how to use AI to tackle these challenges and evaluate whether AI feedback is as effective as traditional models.

Learner feedback is traditionally viewed as a passive transmission of information from a teacher to a learner. However, the modern learning landscape emphasises a more engaging and responsive process centered around the learner, where the exchange of ideas is just as crucial as the information itself (Griffiths, Murdock-Perriera,& Eberhardt, 2023). The potential role of AI and particularly, ChatGPT, cannot be overemphasized in the development of this modern landscape. At Multiverse, we believe that these technologies could revolutionise work-based learning environments by offering effective feedback and positioning the learner as active participants in their feedback process.

A Comparative Study: Traditional Coaching Methods vs AI-feedback

In a recent study (Teasley, 2023), we explored the potential of ChatGPT for delivering meaningful feedback. We compared AI-driven feedback with traditional coaching methods, taking into account the learner’s acceptance and reactions to AI-assisted feedback. We also explored the interaction between both the AI and the learner (Neurerer, et. al, 2018). Although previous research has explored assisted feedback (Maier & Klotz, 2022), this work is at the forefront of using AI to deliver feedback in work-integrated learning environments.

Initial hypotheses were that apprentices would prefer coach feedback over AI-generated feedback. Surprisingly, we found that 70% of apprentices showed a preference for receiving both AI and coach feedback. ChatGPT offered a greater amount of feedback that encouraged self-regulation and autonomy in learners compared to human coaches, showcasing the reliability of AI in providing feedback. Furthermore, we found no significant difference in feedback effectiveness between ChatGPT and coaches, with ChatGPT's feedback slightly favored. Virtual rapport assessments indicated a moderately positive perception of ChatGPT's feedback for its human-like qualities and coherence. Qualitative feedback showed a preference for combining AI's specific and objective feedback with the personal touch and context understanding of human coaches.

Our study suggests that while AI can offer specific and objective feedback, the nuanced understanding and personal engagement provided by human feedback is irreplaceable, advocating for a complementary use of both AI and human feedback in educational contexts. At Multiverse, our exceptional coaches are at the heart of our learning experience. They offer personalized engagement through direct human interaction, which is enhanced by the use of AI technology. Our learners also have access to real-time AI feedback, whenever they need it, through our new on-demand AI tutor. This provides our learners with the tools and resources to reap the benefits from both human and AI feedback.

In summary, by utilizing the cognitive apprenticeship model and AI-enablement (Amankwatia, 2023) we can offer real-time coaching, adaptively scaffold support based on learner performance, and encourage reflective practice through dialogue, enhancing understanding and skill acquisition in a collaborative learning environment.

Study Details

Our study was a mixed methods design which used data from thirteen apprentices enrolled in a technology consulting degree programme. Naturally occurring coach feedback was compared with ChatGPT-generated feedback. This feedback was generated and coded against an Agentic Feedback taxonomy. Surveys measuring apprentice perceptions of feedback, acceptance, motivation, and virtual rapport were developed from the Feedback in Learning Scale (FLS; Jellicoe & Forsythe, 2019).

Survey data was compared and differences were tested for significance and effect sizes. Qualitative data was analysed for key themes and reported. Inter-coder reliability was calculated for feedback coding trials (overall agreement, 79.8%).

Overall, the study demonstrated that ChatGPT's feedback on digital apprenticeship assignments matched the agentic quality of coach feedback and suggests the potential for AI tools to enhance feedback in work-integrated learning by complementing human inputs with timely, specific, and effective feedback (Teasley, 2023).

References

Team Multiverse

Read more posts by this author

Privacy PolicyContact UsPress EnquiriesLevyTermsPoliciesPrivacy Settings

Multiverse • 2 Eastbourne Terrace • Floors 5+6 • London • W2 6LG | info@multiverse.io
© Multiverse 2024