Newest papers

Beyond Better Ideas: How Human-AI Collaboration Shapes Scientific Ideas and Human Perceptions

Qian Chen, Saatvik Pradhan, Janice Ahn, Berk Atil, Angkai Li, Shijie Lu, Nobin Sarwar, Michael Ulis, Ruyuan Wan, Wenpeng Yin · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30568 · cited by 0 works (OpenAlex) · open-access PDF

Korean

AI가 점점 더 효율적으로 과학 연구에서 협력자가되기 때문에 AI가 점점 더 효율적으로 평가되기 때문에, 그 영향은 아이디어를 향상시키는 것보다 더 커질 수 있습니다. 그럼에도 불구하고 기존 연구는 이 차원들을 고립적으로 검토했습니다. 우리는 인간-AI 및 인간-인간 연구 팀, 인간 및 LLM 과학적 아이디어 평가자, 협력 출처 분석 및 참가자 설문 조사에 대한 통합적 인 체험 연구를 제시합니다. i) AI 협력이 과학적 아이디어와 그들의 평가를 어떻게 재구성하는지, 그리고 ii) AI 협력은 AI에 대한 인간 평가 가치 아이디어를 어떻게 재구성하는지, 그러나 iterative 검토는 지속적으로 품질을 향상시킵니다. 둘째, AI 판사들은 아이디어에 대한 인간의 차이점을 근접적으로

Machine translation (Meta M2M100); the English original follows.

Abstract

As AI increasingly becomes a collaborator in scientific research, its impact may extend beyond improving ideas to reshaping how they are evaluated, attributed, and perceived by humans. Yet existing research has largely examined these dimensions in isolation. We present a unified empirical study of Human-AI scientific ideation, investigating two complementary perspectives: i) how AI collaboration shapes scientific ideas and their evaluation, and ii) how firsthand collaboration reshapes human perceptions of AI. Through a shared experimental setting involving Human-AI and Human-Human research teams, human and LLM evaluators, collaboration-provenance analysis, and participant surveys, we uncover several counterintuitive findings. First, AI collaboration shows no significant advantage in human-evaluated idea quality, while iterative revision consistently improves quality. Second, AI judges can approximate human scores yet disagree on idea rankings; notably, an AI judge favors AI-assisted proposals without a corresponding advantage in human evaluations. Third, collaboration provenance remains only partially detectable, revealing substantial overlap between Human-AI and Human-Human scientific outputs. Finally, firsthand AI collaboration is associated with greater perceived usefulness but smaller efficiency gains than expected, suggesting a nuanced recalibration of human expectations. Overall, our findings challenge the notion that AI's value in scientific ideation lies primarily in producing better ideas, revealing instead its broader influence on scientific creation, evaluation, attribution, and human perceptions.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Counterintuitive, Perception, Epistemology, Empirical research, Value (mathematics), Computer science

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange