A New Internet Business Model?

> As we think about our role at Cloudflare in this developing market, it's not about protecting the status quo but instead helping catalyze a better business model for the future of Internet content creation. That means creating a level playing field. Ideally there should be lots of AI companies, large and small, and lots of content creators, large and small.

Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.

The "level playing field" rhetoric reminds me so much of Apple talking about the App Store. This new internet business model is just the App Store, substituting websites for apps and Cloudflare for Apple. The system only works with some middleman between the AI companies and the content creators.

Everyone seems to be dismissing this without reading it. As a content publisher, I am very interested in any proposal that results in me getting residual payments from AI scrapers. That would indeed be a new internet business model.

This is also being attempted by RSL with their “Crawler Authentication Protocol” (https://rslstandard.org/guide/web-crawlers) for demanding proof of licensing from scrapers and RSL Collective (https://rslcollective.org) for providing the licensing itself. The missing piece there is the ability to detect scrapers with high accuracy without punishing regular browsing humans.

> Unless you believe that content creators should work for free, or that they are somehow not needed anymore — both of which are naive assumptions

Realistically, what was the benefit of the ad-driven model? From my point of view, most of the highly-valuable information came from the various forums, wikis and personal sites, on all of which people would publish the information for free. Ads were largely used to cover hosting costs of the large forums and wikis, but the content creators saw not a single dime.

Over the last 15 years, the search results have shifted from this, to instead become 100% news articles and SEO spam, all with a lot of fluff and very little substance, tons of ads, pop-ups, autoplay videos and subscription walls. All of this is well funded thanks to the business model, but for what? I can't imagine anyone sitting through all that crap to get to what they were looking for. There's a reason people would tell others to add "reddit" to the search query only a few years back, and even that's becoming less worthwhile.

Is this really what we want to preserve? Big publications and their interests?

"Let's give money to the content creators" is exactly what big incumbents want. They want to farm you, the viewer, for big profits, and give the content creators a few pennies in return. Meanwhile they hoover up the lion's share of the profit. We've seen this over and over again. It's rent-seeking.

"Content creators" are part of the problem. Generating endless "content", which isn't very useful or valuable, and creates too much "content supply", which devalues it. This then creates a giant "soup" of content that viewers drown in, trying to find some content that isn't as identically useless as all the rest. But the big incumbents love it, because they use this content soup to collect money from - you guessed it - ad companies. (Those same companies they don't want to make ad-blockers against...)

So now that search is dead, they need to find a new way to drink from the ad-dollar faucet. At first it'll be "pay content creators from AI subscription money", but then AI will be offered "free with ads", and then later "paid with ads". And nobody's going to stop it, because everyone is "happy": the viewer gets their free crap (with ads), the content creators get a few pennies, the big companies rake in billions, and ad companies continue their "industrial welfare" by pouring their excess profits into this whole system as ad-dollars. The great commoditization of eyeballs continues.

I hope that the next big shift will be to undo the asymmetry that crept into the internet quickly after it first became popular. Let us host stuff at home. Let us run odd and strange and great systems wherever we are. Undo the cloud, undo the capture of the net – I'm old enough to remember when we just had a bunch of boxen under a desk somewhere, and it was pretty great.

I don't think they actually know what the future business model is. No one really knows. What people are presenting is iterative models. That's fine but the model will be quite different. Ok we'll need subscriptions and to pay creators but I believe if everything is crypto and stablecoin based it's all going to be pay per view or pay per query because you know a lot of people just want that specific content from a specific source because of reputation or because that's what they like. I'm not paying for 5 different agents for that bespoke experience just like I'm not paying for NYT, WSJ and multiple other publications because it is insanity to price up what should effectively be pay per article. So maybe on the backend creators get a royalty kick back just like the music industry but on the consumer side I definitely think beyond subscriptions with the advent of crypto wallets we're going towards micro transactions for everything.

Ok the the toxic nature of the internet and social media in general and what has become of our digital age, totally agree that it's rage and click bait. I wrote something to that effect here https://github.com/micro/mu/issues/27. But I personally don't think we're going to directly interact with agents the whole time. They will exist, they will be somewhere in the middle layers, there might even be a chat interface that replaces search queries with answers but I think the whole web as a whole and social media needs a rethink. Ads as a business model has to die, even though clearly it won't and we need to shift our attention elsewhere.

As a researcher, I like the notion that there could be some incentive or mechanism to create new and original content, investigate new things, or conduct research in general. I'll group all of those under the umbrella of new content. I don't think things will work out as ideally as described here, but I can appreciate the sentiment and hope that it works out positively.

That said, the mental model that this article uses only recognizes that new content can fill in "holes" (interpolation). It also can expand the boundaries in new directions (extrapolation). That is a different and harder problem. If you distribute money to people "based on what most fills in the holes in the cheese", you really aren't expanding the boundaries of human knowledge as much as you are strengthening existing knowledge. They need to take boundary expansion into account here.

I also recognize that we know where the holes or boundaries are in many fields. They are "known unknowns". But this proposal does not take into account "unknown unknowns" --- things that we do not even realize that we don't know yet. It's going to be harder to incentive research into unknown unknowns when we don't even know what they are yet.

In short: we'll have the machines tell the creators what to create.

> You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about. Say, for example, the carrying capacity of unladened swallows because they know their subscribers of a certain age and proclivity are always looking for answers about that topic. The very pruning algorithms the AI companies use today form a roadmap for what content is worth enough to not be pruned but paid for.

Something that stuck out on this was the notion of the LLMs as models of human knowledge, and the notion that we could "see the gaps". There's been an interesting debate* in journalism over the concept of neutrality - what it means to "just present the facts," and whether that's actually a useful service, and what are actual facts, and how do you define a fact (it's gotten rather epistemological), and that's just for things that actually happened in the real world. I think what's not taken into account here is how much of the world is not able to be summarized into One Universal Truth - how much of what we look for is actually the preference of the author ("what are the best new albums this year?" "how do I cook chicken?"), or an intellectual synthesis of a subject (basically anything written by that acoup guy), or just some entertainment or random digression on a topic. I think this is part of the crux of the argument behind the artists vs LLMs debate (you know, aside from all the economic exploitation) - even the act of writing a summary is a creative act when performed by a human that generates something new, whereas an LLM is, as far as we can tell from both the mechanics and the actual output, a remix of existing content. I think our appetite for the latter is less than we think it is.

I don't fundamentally think that Cloudflare's view here is _wrong_ - by and large, when I google what internal temperature to cook pork to so I don't die, I'm not looking for an opinion or someone's life story - but I think I'm much more interested in how we create the weird niches that create the "knowledge" for the blender. Cloudflare alludes to it with the talk about Reddit and whatnot, but I'd love to know what the plan is to actually create and nurture those communities where people can really get weird into whatever topic they're interested in. Right now we're all sort of just ghettoized into various Facebook communities or whatever, but recreating an actual vibrant communal internet where people can find their weirdo subcultures and actually negotiate on some kind of reasonable footing with the LLM scrapers (who were the social network people, who were the search engine people, who have always been the ad people) would be a genuine improvement to the internet.

* read "hellish swamp diving affair"

I don't know if this is going to work, but at least they are skating to where the puck is going to be.

The new model will be something like:

1. A content creator creates a web site and uses Cloudflare.

2. AI companies pay Cloudflare to allow them to scrape content.

3. Cloudflare gives a cut to the content creator.

4. Users pay AI companies and get their questions answered.

A few observations/predictions:

* If this works, there will be competitors to Cloudflare (AWS, Microsoft, etc.) who will offer better terms to content creators. Content creators can then (easily) switch to whichever reverse-proxy has the best terms.

* Media companies will transform into Cloudflare competitors, aggregating content and monetizing by selling to AI. Their pitch will be that the content will be more curated than Cloudflare. Their brands might survive if the AIs pass the source of the content all the way to the user. For example, the AI says something like, "According to a BBC contributor....". Otherwise, media brands will no longer be known to consumers (only AI companies will care).

* If this works, AI companies will try to cut out the middle-man by building their own ecosystem of content creators.

* As more and more people get their answers directly from AI, it will be easier to sell content directly to AI companies. I.e., instead of publishing something on the open web and relying on Robots.txt to protect your content, you will sell content straight to the AI company. NOTE: If this happens, then the only way this will scale is if the AI itself decides which content it wants to buy for the next training run.

* At the limit, the web and everything about it basically disappears. Everyone gets their content directly from an AI and never visits a web site directly. Therefore, web sites disappear and all that's left is the HTTP protocol, which is used by AI clients to talk to the AI cloud.

    ads have been the only micropayment
    system that has worked

I wonder how the web would look like if one could click "pay 1 cent to continue".

Maybe content would become better? Maybe it would make one think "Hmm... one moment, is this something I want to read or am I just doomscrolling?".

Given CloudFlare's direction, I won't be surprised if they start offering "anti adblock as a service" soon. Detecting it is still rather simple, and add in some punishments such as temporary IP blocking, and it would be rather effective.

(Note that everything CloudFlare talked about in this blog post also applies to adblock users, not just AI agents.)

The golden age of the Internet is not where people do it for money, or for views. That way lies clickbait and content farms. The golden age of the Internet is one where people share information because they want to.

Source: https://1gn15.com/cloudflare