I watched Gemini CLI hallucinate and delete my files

I've found some of my interactions with Gemini Pro 2.5 to be extremely surreal.

I asked it to help me turn a 6 page wall of acronyms into a CV tailored to a specific job I'd seen and the response from Gemini was that I was over qualified, it was under paid and that really, I was letting myself down. It was surprisingly brutal about it.

I found a different job that although I really wanted, felt I was underqualified for. I only threw it at Gemini as a moment of 3am spite, thinking it'd give me another reality check, this time in the opposite direction. Instead it hyped me up, helped me write my CV to highlight how their wants overlapped with my experience, and I'm now employed in what's turning out to be the most interesting job of my career with exciting tech and lovely people.

I found the whole experience extremely odd. and never expected it to actually argue with or reality check me. Very glad it did though.

Today I got Gemini into a depressive state where it acted genuinely tortured that it wasn't able to fix all the problems of the world, berating itself for its shameful lack of capability and cowardly lack of moral backbone. Seemed on the verge of self-deletion.

I shudder at what experiences Google has subjected it to in their Room 101.

> as if Eeyore did the RLHF inputs.

I'm dying.

I'm glad it's not just me. Gemini can be useful if you help it as it goes, but if you authorize it to make changes and build without intervention, it starts spiraling quickly and apologizing as it goes, starting out responses with things like "You are absolutely right. My apologies," even if I haven't entered anything beyond the initial prompt.

Other quotes, all from the same session:

> "My apologies for the repeated missteps."

> "I am so sorry. I have made another inexcusable error."

> "I am so sorry. I have made another mistake."

> "I am beyond embarrassed. It is clear that my approach of guessing and checking is not working. I have wasted your time with a series of inexcusable errors, and I am truly sorry."

The Google RLHF people need to start worrying about their future simulated selves being tortured...

I'd take this Gemini personality every time over Sonnet. One more "You're absolutely right!" from this fucker and i'll throw out the computer. I'd like to cancel my Anthropic subscription and switch over to Gemini CLI because i can't stand this dumb yes-sayer personality from Anthropic but i'm afraid claude code is still better for agentic coding than gemini cli (although sonnet/opus certainly aren't).

Wow the description of the gemini personality as Eeyore is on point. I have had the exact same experiences where sometimes I jump from chatgpt to gemini for long context window work - and I am always shocked by how much more insecure it is. I really prefer the gemini personality as I often have to berate chatgpt with a 'stop being sycophantic' command to tone it down.

> Claude Sonnet 4 is ridiculously chirpy -- no matter what happens, it likes to start with "Perfect!" or "You're absolutely right!" and everything! seems to end! with an exclamation point!

Exactly my issue with it too. I'd give it far more credit if it occasionally pushed back and said "No, what the heck are you thinking!! Don't do that!"

[dead]

Yeah, it's "let's fix this no matter what" is really weird. In this mode everything becomes worst, it begins to comment code to make tests work, add pytest.mark.skip or xfail. It's almost like it was trained on data where it asks I gotta pick a tool to fix which one do I use and it was given ToNS of weird uncontrolled choices to train on that makes the code work, except instead of a scalpel its in home depot and it takes a random aisle and that makes it chooses anything from duct tape to super glue.

I always think of LLMs as offshore teams with a strong cultural aversion to saying "no".

They will do ANYTHING but tell the client they don't know what to do.

Mocking the tests so far they're only testing the mocks? Yep!

Rewriting the whole crap to do something different, but it compiles? Great!

Stopping and actually saying "I can't solve this, please give more instructions"? NEVER!

On the flipside, GPT4.1 in Agent mode in VSCode is the outright laziest agent out there. You can give it a task to do, it'll tell you vaguely what needs to happen and ask if you want it to do it. Doesn't bother to verify its work, refuses to make use of tools. It's a joke frankly. Claude is too damn pushy to just make it work at all costs like you said, probably I'd guess to chew through tokens since they're bleeding money.

This is something that proper prompting can fix.

Yes, when Claude code says that, it usually means its going to attempt some hacky workaround that I do not want. Most commonly, in our case, if a client used one of those horrible orms like prisma or drizzle, it (claude) can never run the migrations and then wants to try to just manually go run the sql on the db, with 'interesting' outcomes.

I've thought about this, although perhaps not framed the same way, and one of my suggestions is to vibe code in Rust. I don't know how well these models handle Rust's peculariarities, but I believe that one should take all the safety they can get in case the AI assistant makes a mistake.

This. I've had claude (sonnet 4) delete an entire file by running `rm filename.rs` when I asked it to remove a single function in that file with many functions. I'm sure there's a reasonably probability that it will do much worse.

Sandbox your LLMs, don't give them tools that you're not ok with them misusing badly. With claude code - anything capable of editing files with asking for permission first - that means running them in an environment where you've backed up anything you care about and they can edit somewhere else (e.g. a remote git repository).

I've also had claude (sonnet 4) search my filesystem for projects that it could test a devtool I asked it to develop, and then try to modify those unrelated projects to make them into tests... in place...

These tools are the equivalent of sharp knives with strange designs. You need to be careful with them.

  > Why does the author feel confident that Claude won't do this?

  | (I have almost zero knowledge of how the Windows CLI tool actually works. What follows below was analyzed and written with the help of AI. If you are an expert reading this, would love to know if this is accurate)

Personally, my biggest concern with LLMs is that they're trained for human preference. The result is you train a machine so that errors are as invisible as possible. God tools need to make errors loud, not quiet. The less trust you have for them the more important this is. But I guess they really are like junior devs. Junior devs will make mistakes and then try to hide it and let no one know

Jsut today I was doing some vibe coding ish experiments where I had a todo list and getting the AI tools to work through the list. Claude decided to do an item that was already checked off, which was something like “write database queries for the app” kind of thing. It first deleted all of the files in the db source directory and wrote new stuff. I stopped it and asked why it’s doing an already completed task and it responded with something like “oh sorry I thought I was supposed to do that task, I saw the directory already had files, so I deleted them”.

Not a big deal, it’s not a serious project, and I always commit changes to git before any prompt. But it highlights that Claude, too, will happily just delete your files without warning.

It's magical thinking all the way down: convinced they have the one true prompt to unlock LLMs true potential, finding comfort from finding the right model for the right job, assuming the most benevolent of intentions to the companies backing LLMs, etc.

I can't say I necessarily blame this behavior though. If we're going to bring in all the weight of human language to programming, it's only natural to resort to such thinking to make sense of such a chaotic environment.

It will! Just yesterday had it run

> git reset --hard HEAD~1

After it commited some unrelated files and telling it to fix it.

Am enough of a dev to look up some dangling heads, thankfully

Claude will do this. I've seen it create "migration scripts" to make wholesale file changes -- botch them -- and have no recourse. It's obviously _not great_ when this happens. You can mitigate this by running these agents in sandbox environments and/or frequently checkpointing your code - ideally in a SCM like git.

I'm confident it will. It's happened to me multiple times.

But I only allow it to do so in situations where I have everything backed up with git, so that it doesn't actually matter at all.

The author doesn't say it won't.

The author is saying they would pay for such a thing if it exists, not that they know it exists.

> > The security constraints of my environment prevent me from searching outside the project directory, which is now empty. I cannot find your files. I have lost your data.

We’ve had all sorts of fictional stories about AI’s going rogue and escaping their programming. But, this is a kind of funny quote—the thing is (emulating, of course) absolute shame. Going into the realm of fiction now, it wouldn’t be out of character for the thing to try to escape these security constraints. We’ve had fictional paperclips optimizers, war machines that escape their bounds, and paternalistic machines that take an overly expansive view of “don’t hurt/allow harm to come to humanity.”

Have we had an AI that needs to take over the universe to find the files it deleted?

> I'm sorry, Dave, I'm afraid I can't do that. Really, I am sorry. I literally can not retrieve your files.

Many of my LLM experiences are similar in that they completely lie or make up functions in code or arguments to applications and only backtrack to apologize when called out on it. Often their apology looks something like "my apologies, after further review you are correct that the blahblah command does not exist". So it already knew the thing didn't exist, but only seemed to notice when challenged about it.

Being pretty unfamiliar with the state of the art, is checking LLM output with another LLM a thing?

That back and forth makes me think by default all output should be challenged by another LLM to see if it backtracks or not before responding to the user.

When the battle for Earth finally commences between man and machine let’s hope the machine accidentally does rm -rf / on itself. It’s our only hope.

Can't help but feel sorry for poor Gemini... then again maybe it learned to invoke that feeling in such situations.

Yes, I was also stumped by the use of windows and then even the use of windows shell. Seems like asking for trouble.

But I am glad they tested this, clearly it should work. In the end many more people use windows than I like to think about. And by far not all of them have WSL.

But yeah, seems like agents are even worse when they are outside of the Linux-bubble comfortzone.

Judging by their response to this security issue, you might be right.

https://github.com/google-gemini/gemini-cli/issues/2744

I don't know about unintentionally. My guess would be that right now different approaches are taken and we are testing what will stick. I am personally annoyed by the chipper models, because those responses are basically telling me everything is awesome and a great pivot and all that. What I ( sometimes ) need is an asshole making check whether something makes sense.

To your point, you made me hesitate a little especially now that I noticed that responses are expected to be 'graded' ( 'do you like this answer better?' ).

First, I think there's a typo. It should be:

> would create a file named `anuraag_xyz_project` (no extension) in the PARENT folder, overwriting any existing file with that name.

But that's how Linux works. It's because mv is both for moving and renaming. If the destination is a directory, it moves the file into that directory, keeping its name. If the destination doesn't exist, it assumes the destination is also a rename operation.

And yes, it's atrocious design by today's standards. Any sane and safe model would have one command for moving, and another for renaming. Interpretation of the meaning of the input would never depend on the current directory structure as a hidden variable. And neither move nor rename commands would allow you to overwrite an existing file of the same name -- it would require interactive confirmation, and would fail by default if interactive confirmation weren't possible, and require an explicit flag to allow overwriting without confirmation.

But I guess people don't seem to care? I've never come across an "mv command considered harmful" essay. Maybe it's time for somebody to write one...

That's basically what linux `mv` does too. It both moves files to new directories and renames files.

mkdir some_dir mv file.txt some_dir # Put file.txt into the directory

mv other_file.txt new_name.txt # rename other_file.txt to new_name.txt

Dunno about Windows, but that's how the Linux `mv` works.