Chaining FFmpeg with a Browser Agent
35 comments
·November 4, 2025sanjit
shardullavekar
no 2nd thoughts about it, we are only making ffmpeg more accessible and embeddable.
utopiah
Have to admit, ffmpeg syntax is not trivial... but also the project is 24 years old and is basically the defacto industry standard. If you believe you will still be editing videos in 20 years with the CLI (or any other tool or any programming language) wrapping it then it's probably worth few hours learning how it actually works.
jack_pp
I agree, I suggest using this instead : https://github.com/kkroening/ffmpeg-python . While not perfect once you figure it out it is far easier to use and you can wrap more complicated workflows and reuse them later.
poly2it
Kkroening's wrapper has been inactive for some time. I suggest using https://github.com/jonghwanhyeon/python-ffmpeg instead. It has proper async support and a better API.
jack_pp
Thing is, if you want to use LLMs for mockups you got to use the old one.
esperent
The syntax isn't too bad. The problem is that I have to use it a couple of times a year, on average. So every time I've forgotten and have to relearn. This doesn't happen with GUIs nearly as much, and it's why I prefer them over CLI tools for anything that I don't do at least once every week or two.
skydhash
That’s why you write scripts, or put a couple snippets in your notes.
Sean-Der
My question/curiosity is why do so many people use ffmpeg (frustrated by the syntax) when GStreamer is available?
`gst-launch-1.0 filesrc ! qt4demux ! matroskamux ! filesink...` people would be less frustrated maybe?
People would also learn a little more and be less frustrated when conversation about container/codec/colorspace etc... come up. Each have a dedicated element and you can better understand its I/O
artpar
I did not know gstreamer wasm also exists, I'll check it out
goeiedaggoeie
Still has a way to go, but very exciting.
shardullavekar
true, companies like Descript, Veed, or Kapwing exist because no coders find this syntax intimidating. Plus, a CLI tool stands out of a workflow. We wanted to change that.
petetnt
Don't "no coders" find the concepts described in this article imdimitating?
The article states that whatever the article is trying to describe "Takes about ~20-30 mins. The cognitive load is high....". while their literal actual step of "Googling "ffmpeg combine static image and audio."" gives you the literal command you need to run from a known source (superuser.com sourced from ffmpeg wiki).
Anyone even slightly familiar with ffmpeg should be able to produce the same result in minutes. For someone who doesn't understand what ffmpeg is the article means absolutely nothing. How does a "no coder" understand what a "agent in a sandboxed container" is?
shardullavekar
we took a basic example and described it. (will try adding a complex one)
we have our designer/intern in our minds who creates shorts, adds subtiles, crops them,and merges the audio generated. He is aware of ffmpeg and prefers using a SaaS UI on top of it.
However, we see him hanging out on chatgpt, or gemini all the time. He is literally the no coder we have in mind.
We just combined his type what you want + ffmpeg workflows.
artpar
I think that goes with almost every tool you want to use with llm. User should already know the tool ideally so mistakes by llm can be prevented before they happen.
Here making ffmpeg as "just another capability" allows it to be stitched together in workflows
sexyman48
[dead]
jack_pp
As someone who has used ffmpeg for 10+ years maintaining a relatively complex backend service that's basically a JSON to ffmpeg translator I did not fully understand this article.
Like the Before vs after section doesn't even seem to create the same thing, the before has no speedup, the after does.
In the end it seems they basically created a few services ("recipes") that they can reuse to do simple stuff like speed-up 2x or combine audio / video or whatever
shardullavekar
thanks for calling it out, I will correct the before vs after section. But you can describe any ffmpeg capability in plain English and the underlying ffmpeg tool call takes care of it.
jack_pp
I have written a lot of ffmpeg-python and plain ffmpeg commands using LLMs and while I am amazed at how good Gemini or chatGPT can handle ffmpeg prompts it is still not 100% so this seems to me like a big gamble on your part. However it might work for most users that only ask for simple things.
shardullavekar
so creators on 100x will create well defined workflows that others can reuse. If a workflow is not found, llm creates one on the go and saves it.
IsTom
> Half of scripting FFmpeg is just fighting with shell quote escaping for filter_complex.
-filter_complex_script is a thing
skeeter2020
This doesn't make any sense; the Before and After examples accomplish different things. I also don't get who the target audience is; people intimidated by a CLI tool but at home with technical agents?
shardullavekar
people intimidated by a CLI tool but find tools like chatgpt easy to use and those who have video editing as a part of larger workflow.
kwanbix
I use ChatGPT for this kind of complexity.
It works 99% of the time for my use case.
shardullavekar
jack_pp made a point in the comments, worth noting.
draw_down
[dead]
Dachande663
ffmpeg is the only community where I've asked for help and been told "if you have to ask, you're too stupid to use this project". Needless to say, it was a welcoming community I continued engaging with.
coachgodzup
I considered FFmpeg a great project, but I usually avoid to use it directly because of his quite complex syntax. I'm reconsidering it because coupled with an llm is very straightforward and more immediate than an usual graphical editor
orbital-decay
At some point command line becomes unwieldy. FFmpeg would definitely benefit from a non-arcane DSL like AviSynth or a node-based UI.
4gotunameagain
This is yc propping up a startup they have backed, there isn't much substance here.
sylware
HTML <video> or <audio> element with "Streaming" URLs passed to the media player (or internally in the web browser for the big ones).
An aside but related?
FFmpeg has complex syntax because it’s dealing with the _complexity of video_. I agree with everyone about knowing (and helping create or contribute to) our tools.
Today I largely forget about the _legacy_ of video, the technical challenges, and how critical it was to get it right.
There are an incredible number of output formats and considerations for _current_ screens (desktop, tablet, mobile, tv, etc…). Then we have a whole other world on the creation side for capture, edit, live broadcast…
On legacy formats it used to be so complex with standards, requirements, and evolving formats. Today, we don’t even think about why we have 29.97fps around? Interlacing?
We have a mix of so many incredible (and sometimes frustrating) codecs, needs and final outputs, so it’s really amazing the power we have with a tool like FFmpeg… It’s daunting but really well thought out.
So just a big thanks to the FFmpeg team for all their incredible work over the years…