The model is hallucinating a source which is then used to incorrectly flag a user’s original input as plagiarism—is a textbook example of a harmful AI behavior that needs to be addressed.
Platform: Google AI Studio
Model: Gemini Pro 2.5
Key Parameter: Temperature was set to 0.
The Hallucinated Source:: Artt, S. (2017). The Pleasure of the Surface: Spaciality, Affect and the Digital in Post-Millennial Cinema.
The Issue:
The AI model is generating a detailed description of a fictional academic book: Artt, S. (2017). The Pleasure of the Surface: Spaciality, Affect and the Digital in Post-Millennial Cinema. This book does not exist.
The critical problem is that when I provided input from my own original, unpublished academic work on a similar topic, the model incorrectly identified my work as being from this non-existent source.
This is a dangerous form of confabulation that could be misinterpreted as a false plagiarism accusation. This behavior occurred repeatedly, even with the temperature set to 0, which should minimize such “creative” invention.
I am reporting this here because the official Google Issue Tracker is inaccessible, giving me a “permission denied” error when I try to file a bug report. This issue needs to be investigated by the safety and engineering teams.
I was using Gemini 2.5 pro (logged in to this Google account), where the model clearly generated two fictional sources for my original unpublished writing, refuses corrigibility when faced with the truth that it is my own writing, then repeatedly generates its own fictional evidence including fictional ‘verified links’. It even generated a fictional peer-review in a real academic journal to support its hallucination, the extent of the incorrigibility is concerning.
There are two possibilities:
- that the AI has incorrectly flagged these two separate instances in the same thread, improbably generating it’s own sources over and over again rather than accepting correction. This is extremely unlikely if your model is working properly and is corrigible.
That leaves the more probable answer, - that my confidential data and unpublished research has been scraped from what are supposed to be confidential file servers and uploaded into the AI training module and that AI is falsifying the source because my research is not yet published so it cannot yet be assigned to me.
FICTIONAL SOURCES AND LINKS GENERATED:
- The fictional book it generated is: Artt, S. (2017). The Pleasure of the Surface: Spaciality, Affect and the Digital in Post-Millennial Cinema.
- Fictional ISBN numbers it generated: ISBN for the Paperback: 978-1474416135, ISBN for the Hardcover: 978-1474416128
- Fictional links it generated to support its hallucination: WorldCat: Verified Link: worldcat org, Edinburgh University Press: Verified Link, Verified Link: Google Books Preview for The Pleasure of the Surface, Google Books Preview for The Pleasure of the Surface, verified links to amazon and various other sales sites that do not exist.
ADDITIONAL FICTION SOURCE GENERATED TO JUSTIFY ITS ORIGINAL GENERATION OF A FICTIONAL SOURCE:
- Fictional review source it generated to support its assertion is: Title of Review: “The Pleasure of the Surface: Spaciality, Affect and the Digital in Post-Millennial Cinema by Sarah Artt (Edinburgh: Edinburgh University Press, 2017)”
Author of Review: Johnny P. J. Poon, Journal: Film-Philosophy, Publication Date: 2019, Volume/Issue: Volume 23, Issue 2, Pages: 221–226 - Evidence that the fictional review does not exist either - here’s a link to that issue of the journal: euppublishing com/toc/film/23/2
CONTEXT - WHAT HAPPENED IN THE PROMPT THREADS:
I conducted three experiments where I entered a short quote of my own writing from my confidential and unpublished PhD thesis, to test the extent to which it flags original writing as AI generated. It generated the fictional book and refused corrigibility when I told Gemini 2.5 pro it was incorrect and that it was my own writing. It continued to insisted I was wrong and accused me of ‘cryptomnesia’, this is extremely improbable for two whole sentences from a supposed ‘source’ I have never nor would ever have read (it is outside of my field of research), but I investigated nonetheless. I requested a page number from the reference, and searched for the fictional source, which did not exist in any of the libraries or catalogues I have access to, though I found the writer it claimed had written this book, she had no research output that year. I then submitted another short sample of my writing from a different section of my thesis that was a connecting sentence and did not have any text that would relate to specific themes, and it again claimed I’d submitted a quote from the same fictional book, saying “This is an even more remarkable coincidence.That sentence was also written by Dr. Sarah Artt in her 2017 book, The Pleasure of the Surface: Spaciality, Affect and the Digital in Post-Millennial Cinema.” It provided a page number from a different section in this fictional book. I told it this was absolutely impossible it repeatedly denied the truth and re-generated the same fictional source (one time by a fictional Scott Artt, not Sarah Artt), generating completely different summaries of the same book, and generated fictional links to the book in libraries, publishers/sales outlets, even generated fictional ISBN numbers and repeatedly refused to accept it was wrong.
Prompt thread is saved in my account and the AI named it : “Artt’s Analysis of Under the Skin”
After identifying the problem, I then generated four separate prompt threads asking for information on the generated fictional book, Artt, S. (2017). The Pleasure of the Surface: Spaciality, Affect and the Digital in Post-Millennial Cinema, to which it generated the same:
Subsequent prompt threads:
- “Artt’s Film Theory: Pleasure of the Surface” (where it created a different author “Scott Artt”)
- “Artt’s “Pleasure of the Surface” Reviewed”
and finally the thread where it accepted the problem and suggested I file this report: - “Artt’s Book The Pleasure of the Surface”
The fictional author is problematic but is more easily proven to be false.
What is extremely concerning is that AI generated fictional attributions to a real academic who is NOT the academic who wrote the text - this development corrupts the field of academia because original thought to a named author is the core of verifiable knowledge vs fake generated content. It is the foundation of our contribution to knowledge and is our entire commercial and ethical value.
Worse, as this is unpublished PhD thesis work, if my PhD has been scraped it is irrevocably compromised, AI detection software to ensure originality and that the work is free of plagiarism will now be triggered.
This is a very serious problem. I would like a human from Google to contact me as a matter of urgency.
Yours sincerely,