Psychology21 papers
LLMs in qualitative psychology
Emerging applications of large language models in qualitative psychological research methodology as well as the critics.
ES
Evgeny Smirnov
All Reviews
PsychologyNiche
intermediate
Comparing the Efficacy and Efficiency of Human and Generative AI: Qualitative Thematic Analyses
Prescott, Maximo R. et al. (2024)
- Published
- Aug 2, 2024
- Journal
- JMIR AI · Vol. 3
- DOI
- 10.2196/54482
At a GlanceAI
ChatGPT and Bard produced similar themes far faster than humans, but weaker coding agreement supports hybrid qualitative analysis.
SummaryAI
This study tests whether generative AI can accelerate qualitative analysis needed to improve digital health interventions. On 40 SMS reminders for HIV medication adherence, ChatGPT and Bard recovered many human-generated inductive themes while completing analysis in about 20 minutes versus roughly 567 minutes for human coders. However, coding agreement with humans was only fair to moderate, and people better identified nuanced, interpretive themes. The findings support using LLMs to reduce workload while retaining human oversight rather than replacing qualitative researchers.
Method SnapshotAI
The study compares human coders with ChatGPT and Bard on inductive and deductive thematic analyses of SMS health-intervention messages.
BackgroundAI
Basic qualitative research methods, especially thematic analysis and intercoder reliability, plus familiarity with generative AI.
One of the first pros & cons analysis on the usage of LLMs in psychology + comparison
— ES