Eason Chen, Sophia Judicke, Kayla Beigh, Xinyi Tang, Isabel Wang, Nina Yuan, Zimo Xiao, Chuangji Li, Siyi Li, Reed Luttmer, Shreya Singh, Maria Yampolsky, Naman P. Parikh, Yvonne Zhao, Meiyi Chen, Scarlett Huang, Anishka Mohanty, Gregory Johnson, John Mackey, Jionghao Lin · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.31481 · cited by 0 works (OpenAlex) · open-access PDF
우리는 AI 지원 시스템에 대한 놀라운 액세스가있는 학부 증거 기반 수학 과정에서 153 명의 학생을 연구했습니다. 학생들은 이 과정에서 일찍이 또는 나중에 AI 채팅봇과 홈 워크 피드백 인터페이스를받을 수 있도록 무작위로 배정되었습니다. 초기 액세스 그룹에만 액세스 할 수있는 기간 동안, 그들의 홈 워크 점수는 2.56 점이 높았지만 시험 성능은 그룹 사이에 크게 다르지 않았으며, AI 지원 홈 워크 성능이 향상된 AI는 후속 시험에서 감지 할 수있는 이득으로 번역되지 않았습니다. 모든 학생들이 액세스를받은 후, AI 채팅봇에 보낸 100 개의 추가 메시지는 후속 시험 채팅 활동을 통해 채팅 포인트가 적은 0.62 점과 관련이 있습니다
Machine translation (Meta M2M100); the English original follows.
We studied 153 students in an undergraduate proof-based mathematics course with staggered-access to an AI support system. Students were randomly assigned to receive an AI chatbot and a homework-feedback interface earlier or later in the course. During the period when only the early-access group had access, their homework scores were 2.56 points higher, but exam performance did not differ significantly between groups, suggesting that improved AI-supported homework performance did not translate into detectable gains on a subsequent exam. After all students received access, every additional 100 messages sent to the AI chatbot was associated with a 3.02-point lower score on the subsequent exam. A mediation analysis showed a significant indirect association linking lower baseline self-efficacy to greater chatbot use and, in turn, lower exam scores: a one-point decrease in self-efficacy corresponded to 0.62 fewer exam points through chatbot use. These findings raise concerns that students may use the chatbot to offload their cognitive reasoning. Use of the homework-feedback interface, which required students to submit their proof attempts before receiving feedback, was not negatively associated with exam performance. We recommend against using a chatbot for AI tutoring and suggest exploring a tutor embedded in the course activities.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Chatbot, TUTOR, Cognition, Computer science, Psychology, Mediation
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.31481, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange