AI for Me but not for Thee?

Students aren’t alone in their use of generative AI tools on campus this fall. Faculty members also use the technology to streamline their workload—including the sometimes-tedious work of providing peer reviews for research that has been submitted to academic journals.

In a recent article in the Chronicle of Higher Education, Paul Bloom, who edits Behavioral and Brain Sciences, reported he became suspicious when a reviewer praised an argument for having “earned its keep.” An AI detector reported a high level of certainty that the review had been machine written.

Bloom experimented. He asked an AI tool to “peer review” one of his own unpublished manuscripts. He received meticulous, instructive feedback that he found both useful and unnerving. He suggested that both writers and peer reviewers might benefit from AI tools. However, he remains committed to upholding his publisher’s policy that prohibits the use of AI tools. He noted that his colleagues worry AI might “fail to do the real job of assessing the value of the work as a whole.”

In a recent New York Times essay, Rachael Bedard wrote that she believes AI has helped her become a better primary care provider. She uses a tool called OpenEvidence, accessible only to health care providers, to help interpret lab results, select medications, and diagnose patient ailments. But she worries that the same tool is “making doctors-in-training worse.”

“I worry about the consequences of introducing such a powerful decision aid so early in their careers. Not only have they memorized less than I’d like, but they can seem almost passive in their relationship to their machines,” Bedard wrote.

Underlying both Bloom’s and Bedard’s observations lies a similar concern: “I can recognize beneficial uses of AI, but I worry about them.”

Communication researchers might recognize the “third-person effect.” In a 1983 article in Public Opinion Quarterly, W. Phillips Davison wrote that people believe that “other people” are more influenced by media content than they are. Many researchers have confirmed that we believe propaganda affects gullible people, violent shows harm sensitive people, and social media drains the brains of young people. But not me.

I’ve overheard faculty members from multiple disciplines say they believe AI sharpens their judgment and erodes the judgment of their students. They often go on to say that students need to learn basic skills before they use AI tools. Researchers have begun to explore the relative value of personal experience and AI-assisted learning.

In one study, developers who depended on AI to help learn about a new tool scored lower on conceptual understanding than those who did not use AI. However, developers who stayed actively engaged rather than relying heavily on the AI tool learned almost as well as those who didn’t use AI at all.

In an observational study in The Lancet Gastroenterology & Hepatology, experienced endoscopists who had grown used to AI-assisted colonoscopies detected precancerous growths less often when the AI was taken away: the detection rate fell from 28.4 percent to 22.4 percent. Perhaps they were less actively engaged in their work.

In a study published this year in New Ideas in Media and Communication, people who believed they were skilled users of AI also believed that AI-written health misinformation would fool others but not them. However, participants could only detect AI-written content about 53% of the time.

An AI story currently unfolding at Dartmouth may provide the clearest example of the third-person effect in higher education. In August, Provost Santiago Schnell wrote a Washington Post op-ed urging colleges to distinguish what students can do on their own from what they can do with AI. The Post asks guest writers to confirm their work was not created with AI. Schnell did not disclose AI use when he submitted.

Recently, the student newspaper, The Dartmouth, reported that the AI detector Pangram identified Schnell’s op-ed as having been AI-generated. Schnell has now acknowledged using ChatGPT for “language refinement, copy editing, improving clarity and organization, and making writing processes more efficient.” But Dartmouth’s student policy warns that failure to disclose AI use when submitting work can lead to disciplinary action.

Dartmouth’s faculty AAUP chapter is petitioning for Schnell’s removal. Dartmouth’s President Sian Beilock said the institution needs “role-appropriate rules” for AI. That sounds like the third-person effect.

Writing in The Chronicle of Higher Education last fall, Justin Reich of the Massachusetts Institute of Technology argued that while some academic experts use AI proficiently, best practices have not yet been established. He urged colleges to be cautious about declaring what “AI literacy” means. If he’s right, faculty members have no special standing to judge their own AI use in comparison to the use of a “third person.”

I must confess my own place in this story. I’m a retired communication professor whose research and teaching focused largely on media technology. ChatGPT was introduced shortly before I finished my most recent academic book. I toyed with it then, and I’ve kept experimenting. I now use AI regularly for research, outlining, and editing. I believe it sharpens my craft without replacing the ideas or the words at the heart of my writing.

I worked with Anthropic’s AI tool, Claude, to gather data and draft an outline for this story. Then I asked Claude to write an article. Pangram correctly identified that “draft” as 100% AI written. I then rewrote in my own voice using the same outline Claude and I had developed together. Claude copyedited my revised article and noted some places where I had mischaracterized sources. I revised. Now, a Pangram analysis found the article to be 100% human written.

If I were still teaching, I’d worry about my students who don’t have the experience to develop ideas and write papers on their own. That, of course, is exactly what the third-person effect research predicts I’d say. But rather than policing them with Pangram, I’d like to think that I would coach them in how to express their own ideas in a way that is both human and accurate.