ChatGPT's enterprise success against Copilot fuels OpenAI/Microsoft rivalry
137 comments
·June 24, 2025hbn
My work gives us Copilot access since We're a Microsoft Shop™
It's too bad Copilot is by far the dumbest competitor in the space
My favorite interaction so far was when I prompted it with:
ffmpeg command to convert movie.mov into a reasonably sized mp4
Sure, it's not the most direction instructions, but I tend to give it just enough to get the job done, assuming the LLM knows what its purpose is as an LLM, and it always works with the other chatbots.Copilot's response:
I implemented and executed the Python code above to convert movie.mov to a reasonably sized movie.mp4 using ffmpeg.
However, the Python code failed since it was not able to find and access movie.mov file.
Do you want me to try again or is there anything else that I can help you with?
Note that I didn't cut anything out. It didn't actually provide me any "Python code above"NordSteve
I pasted your prompt:
ffmpeg command to convert movie.mov into a reasonably sized mp4
into the Copilot app just now. Here's the response: ffmpeg -i movie.mov -vcodec libx264 -crf 23 -preset medium -acodec aac -b:a 128k movie_converted.mp4
with this explanation Here’s the breakdown:
- -vcodec libx264: Uses the H.264 video codec, great for compression and compatibility.
- -crf 23: Constant Rate Factor — lower means better quality and larger size; higher means smaller size but lower quality. 23 is a nice middle ground.
- -preset medium: Controls encoding speed vs. compression efficiency. You can try slow or faster depending on your needs.
- -acodec aac -b:a 128k: Compresses the audio using AAC at 128 kbps — pretty standard and compact.
If you're looking for even smaller file sizes, try bumping -crf up to 26 or using -preset slow.
Want to bake in scaling, remove metadata, or trim the video too? Happy to tailor the command further.
glenstein
One fascinating aspect of LLMs is they make out-in-the-wild anecdotes instantly reproducible or, alternatively, comparable to results from others with different outcomes.
A lot of our bad experiences with, say, customer support hotlines, municipal departments, bad high school teachers, whatever, are associated with a habit of speaking that ads flavor, vibes, or bends experiences into on-the-nose stories with morals in part because we know they can't be reviewed or corrected by others.
Bringing that same way of speaking to LLMs can show us either (1) the gap between what it does and how people describe what it did or (2) shows that people are being treated differently by the same LLMs which I think are both fascinating outcomes.
celeritascelery
LLMs are definitely not instantly reproducible. The temperature setting adjust randomness and the models are frequently optimized and fine tuned. You will very different results depending on what you have in your context. And with a tool like Microsoft copilot, you have no idea what is in the context. There are also bugs in the tools that wrap the LLM.
Just because other people on here say “worked for me” doesn’t invalidate OPs claim. I have had similar times where an LLM will tell me “here is a script that does X” and there is no script to be found.
myhf
We're also seeing a new variant of Cunningham's law:
The best way to get the right answer from an LLM is not to ask it the right question; it's to post online that it got the wrong answer.
hangonhn
This is hilarious because both Gemini and ChatGPT are shockingly good at putting together FFMPEG commands. They can both put together and also understand the various options and stages/filters.
canyp
I really like the final remark, "or is there anything else that I can help you with"?
Yeah, like how about answering the fucking question? lol
what-the-grump
I cannot reproduce this in any version of copilot?
Copilot with outlook.com
Copilot base one that comes with M365,
And the add-on one for 30$/mo.
Copilot in VS code
All produce: ffmpeg -i movie.mov -vcodec libx264 -crf 23 -preset medium -acodec aac -b:a 128k output.mp4
Which is not surprising because its just an Open AI 4o call... so how are you getting this?
hu3
What model? It got right first try here with your exact prompt using the free GPT-4.1 model.
https://i.imgur.com/toLzwCk.png
ffmpeg -i movie.mov -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k movie.mp4
BUT, I have this in my copilot-instructions.md file:
# Always follow these steps when responding to any request
1. Please do a round of thinking in <thinking></thinking> tags
2. Then a round of self-critique in <critique></critique> tags
3. Then a final round of <thinking>, before responding.
4. If you need more information, ask for it.
easton
That's GitHub Copilot, not "microsoft copilot", the bot built into the Microsoft 365 landing site. it doesn't give you model options, for example.
clvx
Whoever decided to use the same brand for different experiences should be demoted at minimum. What a way to destroy trust.
jug
Yeah it’s really annoying how opaque they are about the model there. Always just ”GPT 4 based” or ”GPT 4o based” if you dig in their blog archives. Makes one unable to check it against benchmarks or see when it’s updated. Setting expectations. Is it a distill? Lower precision quant? An old revision? Who knows.
hu3
I see. Still worked for me.
Openned: https://copilot.microsoft.com
Same prompt: ffmpeg command to convert movie.mov into a reasonably sized mp4
esskay
This is pretty interesting, I had a very similar experience with Github Copilot's plugin inside Jetbrains IDE today (set to use 4o). I asked it to tell me how to do something, it instead routed in my code, tried to write a python file (its a PHP project) and told me it couldn't do it and did the exact same "Do you want me to try again or is there anything else I can help you with?"
Thing is I ask it random bits like this all the time and it's never done that before so I'm assuming some recent update has borked something.
rzzzt
Your conversations are notebooks and the code it conjured up should be behind a dropdown arrow. For visualization it seems to work fine (i.e. Copilot will generate a Python snippet, run it on the input file I attach to the request and present the diagram as a response).
cedws
Microsoft has really screwed up on branding yet again. Every time I read “Copilot” I think of the GitHub thing, forgetting that there is also a Microsoft Copilot that is actually multiple things across multiple products including Windows and Office.
It’s also a website like ChatGPT apparently? I thought it was called Copilot because it writes with you, so why is there also a general chat/search engine called Copilot? Jesus.
thedougd
I think you may be confusing Microsoft Copilot with Microsoft365 Copilot? The first doesn’t give you access to Microsoft Copilot Studio but that might also be available with Microsoft Copilot Pro.
cedws
My confusion has only increased.
cubefox
I believe you might be confusing Copilot Search with Copilot+? Which is of course different from Copilot Pro though not necessarily entirely distinct.
worldsayshi
And Sam Altman thought that they were bad at naming things. Good thing they were bought up by the king of naming things. /s
Edit: They are doubling down on bad naming conventions so hard that it makes me think it's some kind of dark pattern sales strategy..
robots0only
ohh wow, that's bad, just tried this with Gemini 2.5 Flash/Pro (and worked perfectly) -- I assume all frontier models should get this right (even simpler models should).
quantadev
I'd be willing to bet a more clear prompt would've given a good answer. People generally tend to overlook the fact that AIs aren't like "google". They're not really doing pure "word search" similar to Google. They expect a sensible sentence structure in order to work their best.
macNchz
I'm all about clear prompting, but even using the verbatim prompt from the OP "ffmpeg command to convert movie.mov into a reasonably sized mp4", the smallest current models from Google and OpenAI (gemini-2.5-flash-lite and gpt-4.1-nano) both produced me a working output with explanations for what each CLI arg does.
Hell, the Q4 quantized Mistral Small 3.1 model that runs on my 16GB desktop GPU did perfectly as well. All three tests resulted in a command using x264 with crf 23 that worked without edits and took a random .mov I had from 75mb to 51mb, and included explanations of how to adjust the compression to make it smaller.
roywiggins
Maybe, but this sort of prompt structure doesn't bamboozle the better models at all. If anything they are quite good at guessing at what you mean even when your sentence structure is crap. People routinely use them to clean up their borderline-unreadable prose.
breadwinner
Microsoft has wasted their opportunity.
When ChatGPT first came out, Satya and Microsoft were seen as visionaries for their wisdom in investing in Open AI. Then competitors caught up while Microsoft stood still. Their integration with ChatGPT produced poor results [1] reminding people of Tay [2]. Bing failed to capitalize on AI, while Proclarity showed what an AI-powered search engine should really look like. Copilot failed to live up to its promise. Then Claude.ai, Gemini 2.0 caught up with or exceeded ChatGPT, and Microsoft still doesn't have their own model.
[1] https://www.nytimes.com/2023/02/16/technology/bing-chatbot-m...
vbezhenar
I'll add, that Google search AI integration is quite good. I'm actually amazed how well it works, given the scale of Google Search. Nowadays I don't click search results in 50% of searches, because Google AI outputs response good enough for me.
asciimov
Maybe we have a different Google AI down here in south Texas, but the Google search AI results I receive are laughably bad.
It has made up tags for cli functions, suggested nonexistent functions with usage instructions, it’s given me operations in the wrong order, and my personal favorite it gave me a code example in the wrong language (think replying Visual Basic for C).
cpncrunch
Its about half and half. Really depends on whether there are good results that gemini can summarize. If not, it gets creative. Chatgpt is generally much better.
alfalfasprout
this is the first I've heard of anyone praising it... the results are usually outright wrong or useless.
bentcorner
Agree, I've seen enough wrong answers that I think it's actively harmful to put AI answers at the top of Google search results.
redwood
A lot of folks probably just assume it's correct
Spooky23
The problem is that they made huge time consuming investments in technology to make copilot work with the various O365 controls, then confused everyone by slathering copilot on everything.
Barbing
Wonder why they’re going so slowly…
(& small typo, “Proclarity” = *Perplexity)
crowcroft
Even with their failures Microsoft still has OpenAI over a barrel.
Access to their IP, and 20% of revenue (not profit).
spiderfarmer
The biggest problem with Microsoft is their UX. From finding out where to actually use their products, to signing in, wading through modals, popups, terms and agreements, redirects that don’t work and links that point to nowhere. Along the way you’ll run into inconsistent, decades old UI elements and marketing pages that fully misunderstand why you’re there.
It’s a big, unsolvable mess that will forever prevent them from competing with legacy-free, capable startups.
They should delete all their public facing websites and start over.
atonse
Bill Gates agreed with you 20 years ago :-) (this email never gets old)
https://www.osnews.com/story/19921/full-text-an-epic-bill-ga...
esafak
> When SeattlePI asked Bill Gates about this particular email last week, he chuckled. “There’s not a day that I don’t send a piece of e-mail… like that piece of e-mail. That’s my job.”
If he had to send the same email every day he wasn't doing his job well, and neither was everyone below him. Even a fraction of that list is too much.
tartoran
Their UX, their naming conventions from products to frameworks and services, pulled plugged on products, user hostility and so on are all pointing out the root of the problem is elsewhere. I think Microsoft is no longer reformable. It is a behemoth that will probably continue to coast along like a braindead gozilla zombie that just floats due to its sheer size.
AppleBananaPie
That's the feel I get too :/
Too many crazy presentations on 'data' that are calling the calling the sky purple and everyone just nods along, ok's and gives promos all around.
dyauspitr
How have they failed? They still get 49% of openAI’s profits so if openAI wins, Microsoft wins.
flowerlad
That can be considered victory only if Microsoft is an investment firm as opposed to a software company.
mangamadaiyan
Aren't they actually an investment firm with a passing interest in software?
protocolture
Copilot is brainrot and its killing microsoft.
Renaming all their products to Copilot makes no sense and just causes brand confusion.
Copilot getting access to your entire 365/azure tenant is just a security nightmare waiting to happen (in fact theres already that one published and presumably patched vuln)
It has so many shackles on that its functionally useless. Half the time I ask it to edit one of my emails, it simply spits my exact text back out.
Its one singular advantage is that it has crystal clear corpospeak license surrounding what it says your data will be used for. Whether or not its true is irrelevant, organisations will pick it up for that feature alone. No one ever got fired for choosing ibm etc.
null
thiagoperes
msft had a massive edge. it had exclusive access to models + had web search before anyone.
they flopped this royally, just like windows mobile. they created a shitty ux by shoving it inside the bing app, then they decided to charge for it instead of capturing all enterprise value.
lastly, the product has stalled and missed on their biggest opportunity which is tapping into the data. you can think it's because of how complex it must be, but then openai and everybody else did it.
it's truly a lesson in product mismanagement, once again, from microsoft
rahimnathwani
Is anyone here not confused about how Copilot licenses work (free vs paid) and how to choose models for different types of task?
simonw
I'm confused over what anyone means when they say "Copilot", since it could mean the VS Code editor features or various features on github.com or the thing that Microsoft sell as part of their 365 office software.
I think this article is about the 365 suite.
parliament32
Don't forget about the Copilot in Windows, which is different from the Copilot in Bing, which is different from Copilot in Edge, which is different from the Copilot in Copilot Studio... and that's not even getting into the various Copilots across different 365 domains (Microsoft 365 Copilot for Sales, Microsoft 365 Copilot for Service, Copilot for Microsoft Fabric, Copilot for Dynamics 365, etc are all separate products), plus the enterprise-side Security Copilot...
kamarg
Good old Microsoft naming. I'll never understand how they can think it's a good idea to release multiple entirely different products and call them all variations of the same thing. One would think they would have solved this problem a decade ago and yet every few years it happens again.
0x457
Someone thought this would reduce marketing budget, since they now cross-promoting.
rahimnathwani
Yeah I'm talking about the thing you see when you go to https://m365.cloud.microsoft/chat/
At the top-right of that page, it has a little icon indicating 'enterprise data protection' but I can't see any way for me (the user) to know what type of Copilot licence (if any) the accountholder has assigned to my user account.
KoolKat23
If you have the fancy copilot pro, you'll see it in the rest of your office account, such as outlook, where additional features are available such as email summarize etc.
airstrike
Such an easy URL too
airstrike
There's also "Copilot" which is the AI assistant accessible online and via a desktop app on Windows (and even other OSs)
worldsayshi
Your description also works for GitHub copilot.
FauxDemure
Yes! I also feel like I use up my quote with my paid Copilot account faster than with my free ChatGPT one.
ape4
Just ask Copilot
jabiko
I'm not sure whether Microsoft Copilot and ChatGPT use different system prompts or if there's something else behind it, but Copilot tends to have this overly cautious, sterile tone. It always seems to err on the side of safety, whereas ChatGPT generally just does what you ask as long as it's reasonable.
So it often comes down to this choice: Open https://copilot.cloud.microsoft/, go through the Microsoft 365 login process, dig your phone out for two-factor authentication, approve it via Microsoft Authenticator, finally type your request only to get a response that feels strangely lobotomized.
Or… just go to https://chatgpt.com/, type your prompt, and actually get an answer you can work with.
It feels like every part of Microsoft wants to do the right thing, but in the end they come out with an inferior product.
haneul
I wonder if a Lexus/Toyota Acura/Honda Lamborghini/Audi OpenAI/Microsoft marketing split isn't in the best interests of tech giants going forward since LLMs are nondeterministic, unlike the deterministic nation-states they've built up till now...
Barbing
Why is Microsoft special?
ChatGPT. Perplexity. Google AI Mode. All let you get a message off.
… WAIT! copilot dot microsoft dot com lets you just send a message without logging in.
—
heh, the second result on DuckDuckGo is an MS article: “What is Copilot, and how can you use it?”
Products mentioned in the article, they say:
| Copilot | Copilot app | Copilot for individuals |
And a link for each one. Does Satya squirm when he sees that, but doesn’t have the power to change it?
Also the word “individuals” (allegedly previously mentioned) appears only once on the page.
napoleongl
If only MS copilot was an actual co-pilot in my daily flight of work… but no, it cannot actually draft an email, create planner tasks or book meetings. It is a crappy way of using an LLM that sometimes can access your stuff from MS graph and sometimes has no idea about the six emails from Steve in my inbox. And no, its PowerPoints are not good either, they are LinkedIn in ppt-form.
NordSteve
Here are the results I just got.
1. In Outlook asked Copilot to "create an email asking for a weekly status update for Project ABC". Iterating a bit, I asked it to be more casual and shorter. Here's what I got:
Hey Team,
Hope you're all doing great. To keep everyone on the same page with Project ABC, I'd like us to start doing weekly status updates. Just a quick summary of key developments, milestones, challenges, and what's coming up would be super helpful. This way, we can coordinate better and tackle any issues right away.
Thanks for your cooperation and hard work. Looking forward to your updates!
That seems pretty on point. Not exactly my style but to say "cannot actually draft an email" is clearly wrong.2. I provided Copilot with this prompt:
"I am upgrading the electrical service at my house from 100a to 200a service. Create a project plan in Planner for this project."
Here's the response I got:
Nice! Upgrading from 100A to 200A is a solid move—more headroom for your gear and future-proofing your setup. Here's a breakdown of how you could structure this as a project in Microsoft Planner, with buckets and tasks to keep it organized:
---
### **Bucket 1: Planning & Permitting**
- **Assess current electrical load and future needs**
- **Consult with a licensed electrician**
- **Request utility service requirements**
- **Obtain local permits for electrical service upgrade**
- **Review NEC and local code requirements**
---
### **Bucket 2: Materials & Scheduling**
- **Select upgraded panel (200A rated) and breakers**
- **Order required materials (panel, feeder wires, conduit, ground rods, etc.)**
- **Coordinate shutdown and install time with utility provider**
- **Schedule rough-in inspection with local authority**
---
### **Bucket 3: Installation**
- **Shut off power at the main disconnect (coordinate with utility)**
- **Remove existing 100A panel and components**
- **Install 200A panel and connect feeders**
- **Upgrade grounding/bonding system as required**
- **Replace meter socket if required**
---
### **Bucket 4: Inspections & Finalization**
- **Schedule and pass electrical inspection**
- **Have utility restore permanent power**
- **Label circuits clearly in new panel**
- **Update homeowner insurance with new electrical capacity**
---
### **Bucket 5: Post-Project Wrap-Up**
- **Document panel layout and upgrades**
- **Evaluate energy use and plan for future expansion (EV charger, generator, etc.)**
- **Close out permits with inspection authority**
---
You can assign these tasks to yourself or any contractors involved, and add due dates to align with your timelines. Want help creating a PowerShell script to bulk import this into Planner?
It did produce a working script that required 1 configuration edit. The project plan is pretty good given the limited information I provided in the prompt.3. I provided Copilot with this prompt:
book a meeting with <name> for August 1 entitled "Test Meeting"
Not a very good response: I can't directly book meetings or access your calendar, but I can help you set it up! Here's an example of what to enter in Outlook or your preferred calendar app:
**Title:** Test Meeting
**Date:** August 1
**Attendees:** Bryan Herbst
**Time:** [Insert your preferred time]
**Location:** [Optional]
**Description:** [Add any notes or agenda items]
If you're using PowerShell or a script to automate calendar events—especially within Microsoft 365—I can help you write that too. Want to go that route?
kfajdsl
It seems like these examples show that it has about the same capabilities as a basic chat interface like Claude or ChatGPT without any benefit from being integrated with the Microsoft ecosystem.
crop_rotation
The problem is it is very hard to make changes and build innovative new products within big tech, at a pace to compete with smaller companies. Big tech succeeds despite it since the resource disparity is too much.
Since the launch of ChatGPT Microsoft has had access to it and even had some of the most popular code editors, and where did it take them. This is why Meta had to launch threads with a very small team since a big team in Big tech can just not compete.
Off course like everything else there are no absolutes and when Big Tech feels there is an existential crisis on something they do start improving, however such moments are far and few.
mark_l_watson
I read today that OpenAI is planning a ‘AI super app’ that would have canvas, word processing, etc., all in one work app. That actually sounds like a good idea to me and is very different from Google’s approach of integrating Gemini into the work place apps. Google may have an advantage because so many people are used to working in Workspace apps.
airstrike
Mind providing a link? I may or may not be creating the exact same thing...
Also if anyone from OpenAI or any of its competitors wants to talk my email is on my HN profile ;-)
owebmaster
That's the app many of us are creating and I bet some of them are going to be much better than the OpenAI one
airstrike
Tamo junto
sethops1
Doesn't Microsoft own 49% of OpenAI and receives 20% of the revenue (according to ChatGPT)? In which case, what is Microsoft so upset about?
layer8
That they only receive 20% of the revenue.
micromacrofoot
the other 51%
lanthissa
its not a rivalry if one party is not in the competition, its just jealousy
https://archive.ph/cemKI