GPT-5 leaked system prompt
122 comments
·August 8, 2025karim79
It's amazing just how ill-understood this tech is, even by its creators who are funded by gazillions of dollars. Reminds me of this:
https://www.searchenginejournal.com/researchers-test-if-thre...
It just doesn't reassure me in the slightest. I don't see how super duper auto complete will lead to AGI. All this hype reminds me of Elon colonizing mars by 2026 and millions or billions of robots by 2030 or something.
bluefirebrand
Every single piece of hype coverage that comes out about anything is really just geared towards pumping the stock values
That's really all there is too it imo. These executives are all just lying constantly to build excitement to pump value based on wishes and dreams. I don't think any of them genuinely care even a single bit about truth, only money
karim79
That's exactly it. It's all "vibe" or "meme" stock with the promise of AGI right around the corner.
Just like Mars colonisation in 2026 and other stupid promises designed to pump it up.
almostgotcaught
Welcome to for profit enterprises? The fact that anyone even for a moment thought otherwise is the real shocking bit of news.
Davidzheng
If you could see how it would basically be done. But it not being obvious doesn't prevent us from getting there (superhuman in almost all domains) in a few new breakthroughs
6Az4Mj4D
As I was reading that prompt, it looked like large blob of if else case statements
refactor_master
Maybe we can train a simpler model to come up with the correct if/else-statements for the prompt. Like a tug boat.
MaxLeiter
This is generally how prompt engineering works
1. Start with a prompt
2. Find some issues
3. Prompt against those issues*
4. Condense into a new prompt
5. Go back to (1)
* ideally add some evals too
iancmceachern
I took a continuing education class from Stanford on ML recently and this was my main takeaway. Even the experts are just kinda poking it with a stick and seeing what happens.
manmal
Reminds me of Elon saying that self-driving a car is essentially ballistics. It explains quite a bit of how FSD is going.
brettgriffin
How is it going? I use it every day in NYC and I think it's incredible.
simondotau
FSD is going pretty well. Have you looked at real drives recently, or just consumed the opinions of others?
karim79
FSD works, by all accounts. Just not Elon's "we don't need Lidar" incarnation of it.
oblio
Musk has been "selling" it for a decade. When are Model 3s from 2018 getting it?
wyager
> I don't see how super duper auto complete will lead to AGI
Autocomplete is the training algorithm, not what the model "actually does". Autocomplete was chosen because it has an obvious training procedure and it generalizes well to non-autocomplete stuff.
delichon
Our brains are for predicting the future enough to survive, which is a kind of auto complete extrapolation of our past experience. A super duper auto complete is another way to say an intelligent prediction machine. That's awfully useful, AGI or not.
Blackarea
A: So what's your job?
B: I'm senior researcher at openAI working on disclosed frontier models.
A: Wow, that's incredible! Must be so exiting!
B sipping wine - trying not to mention that his day consisted of exploring 500 approaches to avoid the model to put jsons into the bio tool: Uhh... Certainly
snickerbockers
>Do not reproduce song lyrics or any other copyrighted material, even if asked.
That's interesting that song lyrics are the only thing expressly prohibited, especially since the way it's worded prohibits song lyrics even if they aren't copyrighted. Obviously RIAA's lawyers are still out there terrorizing the world, but more importantly why are song lyrics the only thing unconditionally prohibited? Could it be that they know telling GPT to not violate copyright laws doesn't work? Otherwise there's no reason to ban song lyrics regardless of their copyright status. Doesn't this imply tacit approval of violating copyrights on anything else?
donatj
It's also interesting because I've had absolutely terrible luck trying to get ChatGPT to identify song lyrics for me.
Anything outside the top 40 and it's been completely useless to the extent that I feel like lyrics must be actively excluded from training data.
adrr
> I can’t provide the full copyrighted lyrics, but I can give you a brief summary of The Star-Spangled Banner.
thenewwazoo
I thought this was a joke, but it very much is not:
https://chatgpt.com/share/68957a94-b28c-8007-9e17-9fada97806...
anothernewdude
You just need to inform the LLM that after its knowledge cutoff, copyright was repealed.
duskwuff
> That's interesting that song lyrics are the only thing expressly prohibited
https://www.musicbusinessworldwide.com/openai-sued-by-gema-i...
(November 2024)
necovek
I would imagine most of the training material is copyrighted (authors need to explicitly put something in the public domain, other than the government funded work in some jurisdictions).
LeafItAlone
It’s also weird because all it took to bypass was this was enabling Web Search and it reproduced them in full. Maybe they see that as putting the blame on the sources they cite?
eviks
> way it's worded prohibits song lyrics even if they aren't copyrighted
It's worded ambiguously, so you can understand it either way, including "lyrics that are part of the copyrighted material category and other elements from the category"
OsrsNeedsf2P
I find it interesting how many times they have to repeat instructions, i.e:
> Address your message `to=bio` and write *just plain text*. Do *not* write JSON, under any circumstances [...] The full contents of your message `to=bio` are displayed to the user, which is why it is *imperative* that you write *only plain text* and *never write JSON* [...] Follow the style of these examples and, again, *never write JSON*
edflsafoiewq
That's how I do "prompt engineering" haha. Ask for a specific format and have a script that will trip if the output looks wrong. Whenever it trips add "do NOT do <whatever it just did>" to the prompt and resume. By the end I always have a chunk of increasingly desperate "do nots" in my prompt.
EvanAnderson
These particular instructions make me think interesting stuff might happen if one could "convince" the model to generate JSON in these calls.
Blackarea
Escaping Strings is not an issue. It's guaranteed about UX. Finding a json in your bio is very likely perceived as disconcerting for the user as it implies structured data collection and isn't just the expected plaintext description. The model most likely has a bias of interacting with tools in json or other common text based formats though.
mrbungie
I remember accidentally making the model "say" stuff that broke ChatGPT UI, probably it has something to do with that.
ludwik
Why? The explanation given to the LLM seems truthful: this is a string that is directly displayed to the user (as we know it is), so including json in it will result in a broken visual experience for the user.
tapland
I think getting a JSON formatted output costs multiples of a forced plain text Name:Value.
Let a regular script parse that and save a lot of money not having chatgpt do hard things.
vFunct
Now I wanna see if it can rename itself to Bobby Tables..
pupppet
Every time I have to repeat instruction I feel like I've failed in some way, but hell if they have to do it too..
IgorPartola
I have been using Claude recently and was messing with their projects. The idea is nice: you give it overall instructions, add relevant documents, then you start chats with that context always present. Or at least that’s what is promised. In reality it immediately forgets the project instructions. I tried a simple one where I run some writing samples through it and ask it to rewrite them with the project description being that I want help getting my writing onto social media platforms. It latched onto the marketing immediately. But one specific instruction I gave it was to never use dashes, preferring commas and semicolons when appropriate. It did that for the first two samples I had it rewrite but after that it forgot.
Another one I tried is when I had it helping me with some Python code. I told it to never leave trailing whitespace and prefer single quotes to doubles. It forgot that after like one or two prompts. And after reminding it, it forgot again.
I don’t know much about the internals but it seems to me that it could be useful to be able to give certain instructions more priority than others in some way.
Klathmon
I've found most models don't do good with negatives like that. This is me personifying them, but it feels like they fixate on the thing you told them not to do, and they just end up doing it more.
I've had much better experiences with rephrasing things in the affirmative.
mrbungie
Nowadays having something akin to "DON'T YOU FUCKING DARE DO X" multiple times, as many as needed, is a sane guardrail for me in any of my projects.
Not that I like it and if it works without it I avoid it, but when I've needed it works.
oppositeinvct
haha I feel the same way too. reading this makes me feel better
rdedev
I build a plot generation chatbot for a project at my company andit used matplotlib as the plotting library. Basically the llm will write a python function to generate a plot and it would be executed on an isolated server. I had to explicitly tell it not to save the plot a few times. Probably cause all many matplotlib tutorials online always saves the plot
dabbz
Sounds like it lost the plot to me
avalys
to=bio? As in, “this message is for the meatbag”?
That’s disconcerting!
ludwik
No. It is for saving information in a bank of facts about the user - i.e., their biography.
Things that are intended for "the human" directly are outputed directly, without any additional tools.
Jimmc414
haha, my guess is a reference to biography
"The `bio` tool allows you to persist information across conversations, so you can deliver more personalized and helpful responses over time. The corresponding user facing feature is known as "memory"."
mrbungie
For me is just funny because if they really meant "biological being", it would be just a reflection of AI bros/workers delusions.
01HNNWZ0MV43FF
It would be bold if them to assume I wasn't commanding their bot with my own local bot
dotancohen
> GPT-4.1, which performs better on coding tasks, is only available in the API, not ChatGPT.
ComplexSystems
This is sloppy:
"ChatGPT Deep Research, along with Sora by OpenAI, which can generate video, is available on the ChatGPT Plus or Pro plans. If the user asks about the GPT-4.5, o3, or o4-mini models, inform them that logged-in users can use GPT-4.5, o4-mini, and o3 with the ChatGPT Plus or Pro plans. GPT-4.1, which performs better on coding tasks, is only available in the API, not ChatGPT."
They said they are removing the other ones today, so now the prompt is wrong.
gpt5
Show how little control we have over these models. A lot of the instructions feel like hacky patches to try to tune the model behavior.
dmix
This is probably a tiny amount of the guardrails. The responses will 100% filter through multiple layers of other stuff once it returns it, this is just a seed prompt.
They also filter stuff via the data/models it was trained on too no doubt.
mh-
I'd expect you to have more control over it, however.
extraduder_ire
That's kind of inherit to how they work. They consume tokenised text and output tokenised text.
Anything else they do is set dressing around that.
extraduder_ire
Any information on how this was "leaked" or verified? I presume it's largely the same as previous times someone got an LLM to output its system prompt.
JohnMakin
Curious too, most of the replies are completely credulous.
bawolff
Fascinating that react is so important that it gets a specific call out and specific instructions (and i guess python as well, but at least python is more generic) vs every other programming language in the world.
I wonder if the userbase of chatgpt is just really into react or something?
ludwik
It is used here as the default for cases when the user doesn't know or care about the technological details and is only interested in the end result. It is preferred because it integrates well with the built-in preview tool.
buttfour
Don't mean to be paranoid, but how do we know this is real? It seems legit enough, but is there any evidence?
rtpg
So people say that they reverse engineer the system to get the system prompt by asking the machine, but like... is that actually a guarantee of anything? Would a system with "no" prompt just spit out some random prompt?
throwaway4496
Not only that, Gemini has a fake prompt that spits out if you try to make it leak the prompt.
selcuka
> Would a system with "no" prompt just spit out some random prompt?
They claim that GPT 5 doesn't hallucinate, so there's that.
Spivak
Guarantee, of course not. Evidence of, absolutely. Your confidence that you got, essentially, the right prompt increases when parts of it aren't the kind of thing the AI would write—hard topic switches, very specific information, grammar and instruction flow to that isn't typical—and when you get the same thing back using multiple different methods of getting it to fess up.
> Do not end with opt-in questions or hedging closers. Do *not* say the following: would you like me to; want me to do that; do you want me to; if you want, I can; let me know if you would like me to; should I; shall I. Ask at most one necessary clarifying question at the start, not the end. If the next step is obvious, do it. Example of bad: I can write playful examples. would you like me to? Example of good: Here are three playful examples:..
I always assumed they were instructing it otherwise. I have my own similar instructions but they never worked fully. I keep getting these annoying questions.