The human-AI agency rubric: a reflective instrument for human-AI collaboration in digital information problem solving

Authors Josien Boetje, Zindzi de Graaf, Stan van Ginkel, Matthijs Smakman, Johan Versendaal, Erik Barendsen
Publication date 2026
Research groups Smart Systems for Healthy Living, Digital Ethics
Type Report

Summary

Higher education students routinely engage in complex digital information tasks that require information problem-solving (IPS) competence. While generative AI (GenAI) tools can support IPS, they also introduce risks of overreliance and diminished agency. Yet, educators lack practical tools to guide students toward critical GenAI use. Existing IPS rubrics, developed for pre-AI contexts, mainly assess final products and offer limited insight into students' agency in their use of GenAI throughout the IPS process. This study developed and validated a formative rubric characterizing human-AI agency across IPS phases. Development followed five educational design research cycles: indicator derivation from think-aloud protocols with pre-service teachers and teacher educators; validation with experts; usability testing with pre-service teachers; inter-rater reliability assessment; and pattern identification. The resulting rubric contains 55 behavioral indicators across six IPS phases, operationalized via the DIPS model, and three agency levels, from substitution through complementation to augmentation. Pattern analysis identified nine prototypical human-AI agency patterns within tasks, including Offloader, Critical Verifier, or Creative Director. Results indicate that human-AI agency is context-and phase-dependent rather than a fixed characteristic: substitutive use is not inherently inferior to augmentative use. The rubric scaffolds a reflective cycle of awareness of current GenAI use, recognition of dilemmas each AI-use decision entails, and informed decision-making about whether, when, and how to use AI. It makes variation in human-AI agency visible, equipping students, teachers, and researchers with a shared vocabulary to open the conversations that matter: not whether AI was applied, but rather how deliberately and purposefully it was used.

Downloads en links

On this publication contributed

Language English
Key words Generative AI, information problem solving, agency, higher education, formative evaluation
Digital Object Identifier 10.2139/ssrn.6408618
Page range 1-44