Leonardo Chiariglione: “I closed MPEG on 2 June 2020”

Well, a career in codec development means you'd have done it as a job, and so you'd have been angling for a job at the kind of places that enter into the patent pools and contribute to the standards.

Software patents aren't an issue in much of the world; the reason I thought there wasn't much of a career in codec development was that it was obvious that it needed to move down into custom ASICs to be power-efficient, at which point you can no longer develop new ones until people replace all their hardware.

Mozilla is just Google from a financial perspective, it's not an independent org, so the financing point stands.

H.264 was something like >90% of all video a few years ago and wasn't it free for streaming if the end user wasn't paying? IIRC someone also paid the fees for an open source version. There were pretty good licensing terms available and all the big players have used it extensively.

Anyway, my point was only that expecting Google to develop every piece of tech in the world and give it all away for free isn't a general model for tech development, whereas IP rights and patent pools are. The free ride ends the moment Google decide they need more profit, feel threatened in some way or get broken up by the government.

That's unnecessarily harsh. Patent pools exist to promote collaboration in a world with aggressive IP legislation, they are an answer to a specific environment and they incentivize participants to share their IP at a reasonable price to third parties. The incentive being that you will be left out of the pool, the other members will work around your patents while not licensing their own patents to you, so your own IP is now worthless since you can't work around theirs.

As long as IP law continues in the same form, the alternative to that is completely closed agreements among major companies that will push their own proprietary formats and aggressively enforce their patents.

The fair world where everyone is free to create a new thing, improve upon the frontier codecs, and get a fair reward for their efforts, is simply a fantasy without patent law reform. In the current geopolitical climate, it's very very unlikely for nations where these developments traditionally happened, such as US and western Europe, to weaken their IP laws.

IP law and the need for extremely smart people with a rare set of narrow skills. It's not like codec development magically happens for free if you ignore patents.

The point is, if there had been no incentives to develop codecs, there would have been no MPEG. Other people would have stepped into the void and sometimes did, e.g. RealVideo, but without legal IP protection the codecs would just have been entirely undocumented and heavily obfuscated, relying on tamper-proofed ASICs much faster.

> I didn’t find info about these scares on Wikipedia’s GIF or Compuserve pages

The GIF page on wikipedia has an entire section for the patent troubles https://en.wikipedia.org/wiki/GIF#Unisys_and_LZW_patent_enfo...

Free codecs without patent issues were limited to things like Vorbis which never got wide support. There were FOSS codecs for patented algorithms, but those had legal issues in places that enforce software patents.

> Who would write a whole operating system to compete with Microsoft when that would take thousands of engineers being paid $100,000s per year?

You might be misunderstanding that almost all of Linux development is funded by the same kind of companies that fund MPEG development.

It's not "engineers in their basement", and never was

https://www.linuxfoundation.org/about/members

e.g. Red Hat, Intel, Oracle, Google, and now MICROSOFT itself (the competitive landscape changed)

This has LONG been the case, e.g. an article from 2008:

https://www.informationweek.com/it-sectors/linux-contributor...

2017 Linux Foundation Report: https://www.linuxfoundation.org/press/press-release/linux-fo...

Roughly 15,600 developers from more than 1,400 companies have contributed to the Linux kernel since the adoption of Git made detailed tracking possible

The Top 10 organizations sponsoring Linux kernel development since the last report include Intel, Red Hat, Linaro, IBM, Samsung, SUSE, Google, AMD, Renesas and Mellanox

---

curl does seem to be an outlier, but you still need to answer the question: "Who would develop video codecs?" You can't just say "Linux appeared out of thin air", because that's not what happened.

Linux has funding because it serves the interests of a large group of companies that themselves have a source of revenue.

(And to be clear, I do not think that is a bad thing! I prefer it when companies write open source software. But it does skew the design of what open source software is available.)

Google funding free stuff is not a real social mechanism. It's not something you can point to and say that's how society should work in general.

Our industry has come to take Google's enormous corporate generosity for granted, but there was zero need for it to be as helpful to open computing as it has been. It would have been just as successful with YouTube if Chrome was entirely closed source and they paid for video codec licensing, or if they developed entirely closed codecs just for their own use. In fact nearly all Google's codebase is closed source and it hasn't held them back at all.

Google did give a lot away though, and for that we should be very grateful. They not only released a ton of useful code and algorithms for free, they also inspired a culture where other companies also do that sometimes (e.g. Llama). But we should also recognize that relying on the benevolence of 2-3 idealistic billionaires with a browser fetish is a very time and place specific one-off, it's not a thing that can be demanded or generalized.

In general, R&D is costly and requires incentives. Patent pools aren't perfect, but they do work well enough to always be defining the state-of-the-art and establish global standards too (digital TV, DVDs, streaming.... all patent pool based mechanisms).

These are bad comparisons

The question is more, "who would write the HTTP spec?" except instead of sending text back and forth you need experts in compression, visual perception, video formats, etc

Are you really saying that patents are preventing people from writing the next great video codec? If it were that simple, it would’ve already happened. We’re not talking about a software project that you can just hack together, compile, and see if it works. We’re talking about rigorous performance and complexity evaluations, subjective testing, and massive coordination with hardware manufacturers—from chips to displays.

People don’t develop video codecs for fun like they do with software. And the reason is that it’s almost impossible to do without support from the industry.

That’s hardly true. Nvidia’s tech is covered by patents and licenses. Why else would it be worth 4.5 trillion dollars?

The top AI companies use very restrictive licenses.

I think it’s actually the other way around and AI industry will actually end up following the video coding industry when it comes to patents, royalties, licenses etc.

Yeah, counting on governments to develop codecs optimized for fast evolving applications for web and live streaming is a great idea.

If we did that we would probably be stuck with low-bitrate 720p videos on YouTube.

Jxl has been around for years.

Av1 for 7

The problem is every platform wants to force their own codec, and get earn royalties from the rest of the world.

They literally sabotaging it. Jxl support even got removed from chrome.

Investment in adopting in software is next to 0.

In hardware it’s a different story, and I’m not sure to what extent which codec can be properly accelerated

Companies only need a big incentive to invest in new codecs because creating a codec that has a simple incremental improvement would violate existing patents.

His argument is blatantly invalid.

He first points out that a royalty-free format was actually better than the patent-pending alternative that he was responsible for pushing.

In the end, he concludes that the that the progress of video compression would stop if developers can't make money from patents, providing a comparison table on codec improvements that conveniently omits the aforementioned royalty-free code being better than the commercial alternatives pushed by his group.

Besides the above fallacy, the article is simply full of boasting about his own self-importance and religious connotations.

You missed the first part of that quote:

> At long last everybody realises that the old MPEG business model is now broke

And the entire post is about how dysfunctional MPEG is and how AOM rose to deal with it. It is tragic to waste so much time and money only to produce nothing. He's criticizing the MPEG group and their infighting. He's literally criticizing MPEG's licensing model and the leadership of the companies in MPEG. He's an MPEG member saying MPEG's business model is broken yet no one has a desire to fix it, so it will be beaten by a competitor. Would you not want to see your own organization reform rather than die?

Reminder AOM is a bunch of megacorps with profit motive too, which is why he thinks this ultimately leads to stalled innovation:

> My concerns are at a different level and have to do with the way industry at large will be able to access innovation. AOM will certainly give much needed stability to the video codec market but this will come at the cost of reduced if not entirely halted technical progress. There will simply be no incentive for companies to develop new video compression technologies, at very significant cost because of the sophistication of the field, knowing that their assets will be thankfully – and nothing more – accepted and used by AOM in their video codecs.

> Companies will slash their video compression technology investments, thousands of jobs will go and millions of USD of funding to universities will be cut. A successful “access technology at no cost” model will spread to other fields.

Money is the motivator. Figuring out how to reward investment in pushing the technology forward is his concern. It sounds like he is open to suggestions.

Which is an interesting view when applied to the IP. I think it's relatively uncontroversial that an MP4 file which "predicts" a Disney movie which it was "trained on" is a derived work. Suppose you have an LLM which was trained on a fairly small set of movies and you could produce any one on demand; would that be treated as a derived work?

If you have a predictor/compressor LLM which was trained on all the movies in the world, would that not also be infringement?

holy shit it does. The scene with him inventing the new compression algorithm basically foreshadowed the gooning to follow local LLM availability.