Title: Promoting a Growth Mindset: Using Generative-AI to Uncover Themes in Student-Identified Areas of Growth

 

Introduction: The traditional block clerkship education structure doesn’t facilitate longitudinal assessment and feedback across the clerkship year to foster students’ development of a growth mindset. Addressing this challenge, our institution grouped clerkships into Thematic Clinical Blocks (TCBs) based on their potential for similarity in acquisition of, and continuity in, medical knowledge and practice. To facilitate students’ growth mindset, students had dedicated curricular time to review their numerical and narrative evaluations from individual clerkships within a TCB and write reflections identifying which clinical competencies they deemed areas of individual strength and growth. This study seeks to use the reflections to elucidate common themes between TCBs, so that the institution can better support students in achieving competency goals.

 

Aims: We used generative-Artificial Intelligence to investigate student-written reflections, aiming to identify which clinical competencies clerkship students prioritized as areas of growth.

 

Methods: Thematic analysis of 130 clerkship student reflections was conducted using Versa chat, a generative-AI web application akin to ChatGPT 3.5, designed by the UCSF AI Tiger Team. Reflection prompts invited students to reflect on strengths and growth areas in 2 of the school’s 7 ACGME-aligned competencies. We asked Versa to identify similarities and differences between themes in the reflections according to TCB, with emphasis on areas of growth. Investigators coded level of reflection using a validated rubric, with coder discrepancies reconciled using Versa.

 

Results: The most commonly cited competency growth areas across TCBs were Medical Knowledge and Patient Care, with varying justification between TCBs as determined by Versa chat. The application also derived specific methods used by students to improve competency, such as using question banks and seeking formal feedback. The majority of reflections (68.2%) were coded as the highest level of reflection.

 

Conclusions: Student-identified growth areas are consistent across clerkships, with similar underlying themes (e.g. improving knowledge base, communicating with patients). It is fitting that Medical Knowledge and Patient Care are the most commonly cited competencies given that these competencies are also the most broad in scope. Next steps would be to find ways that curriculum leaders can encourage students to focus on other competency areas as well. Given the reflection scores, it is evident that students are capable of critical reflection. In an effort to make reflection a more regular component of student learning, it may be beneficial to complete the reflection process on a more regular basis, e.g. after every individual clerkship in addition to an end-of-TCB reflection. One major limitation is the inability to control for the element of random chance that is built into the Versa algorithm for determining responses. Lastly, this study shows that Generative-AI serves as a promising tool for qualitative data analysis, yielding insight for institutions to provide targeted assistance to students.