Block YouTube ads on AppleTV by decrypting and stripping ads from Profobuf (2022)

Smoothly handling missing or unexpected fields is half the value proposition of protobuf. May as well switch a a much simpler versioned binary protocol instead of all this schema and field tagging complexity if you want to reject every message that doesn't match the client's schema.

But rejecting unknown messages would likely degrade the user experience. Just because Google releases a new version doesn't mean everyone instantly has that new version installed everywhere.

Certificate pinning would be a solution, but the world seems to have decided that that's very difficult to get right. Probably easier to get right in an app than in a website, but I understand not using it.

They could manually sign the protobuf messages to ensure integrity. Duplicating some of the work TLS would already do, but doing it decoupled from TLS infrastructure may be easier.

But unless something like OP's hack becomes mainstream, Google's current approach could be the right one. Sure, it leaves them open to message manipulation, but the potential lost ad revenue from even a tiny failure rate around update time from the other approaches could easily outweigh what they lose from a handful of people running middleware boxes to block ads.

If you want to prevent MitM modification attacks, the way to do it is to sign the data. Trying to do it by making the serialization format less forgiving isn’t the right approach at all. It still has to be pretty flexible. It’s going to be pretty hard to come up with a format that cannot possibly be altered to make the client show no ads. Something like certificate pinning is way easier.

I'm very surprised Google isn't already certificate pinning.

Funny you mention that, because unnecessary "security" controls on streaming data is how Google broke Chromecasts lacked week, and still hasn't figured out how to fix.

I am experimenting a bit and definitly, a little bit loading delay helps to break the loop and „wake up“

Thought this was sarcasm till the end. Good idea

Google has been doing that for you since they bought Youtube.

Isn’t Google degrading your user experience well enough?

This comment made me wonder if some folks have been compelled to find a way to block shorts.

My use of YouTube predates shorts, and I haven’t been a huge shorts consumer on social platforms, and I seem kind of indifferent to them. Anyone else?

Maybe there is something we can figure out and share with our friends who want to manage their shorts use.

Also, apps like opal can be really helpful.

https://www.opal.so/

You don't sound like you are actually looking forward to this.

You want someone to show you how to write a C++/Go program to forward traffic? There are a lot of tutorials online that can already demonstrate this for you. :)

Yup, I’m looking forward to the detailed post from this person (including screenshots!)

Exactly my thoughts.

https://github.com/elazarl/goproxy is pretty nice Go library for writing proxies, I used it once. Supports both HTTPS passthrough and MITM. Here's a trivial example MITMing connections to www.google.com and rejecting requests to https://www.google.com/maps while allowing everything else through:

  package main
  
  import (
      "log"
      "net/http"
      "strings"
  
      "github.com/elazarl/goproxy"
  )
  
  func main() {
      proxy := goproxy.NewProxyHttpServer()
      proxy.Verbose = true
      proxy.OnRequest(goproxy.DstHostIs("www.google.com")).HandleConnect(goproxy.AlwaysMitm)
      proxy.OnRequest(goproxy.DstHostIs("www.google.com")).DoFunc(func(r *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
          if strings.HasPrefix(r.URL.Path, "/maps") {
              return r, goproxy.NewResponse(r, "text/plain", 403, "Forbidden")
          }
          return r, nil
      })
      log.Fatal(http.ListenAndServe(":8080", proxy))
  }

  curl -k -x localhost:8080 https://www.google.com/
  curl -k -x localhost:8080 https://www.google.com/maps
  curl -x localhost:8080 https://www.apple.com/

Remember to use your own cert rather than the hardcoded one in "production" (a trusted network like your home of course, probably a bad idea to expose it on the open Internet).

> watching their videos ad-blocked provides nothing.

it provides the view count, for which the creator reaps rewards from as part of the boost in the algorithm from youtube.

Not to mention that a lot of creators on youtube also do sponsored segments.

They get far more from a premium viewer than an ad viewer.

> Supposedly creators get a bigger share from YT Premium users'

I've heard this multiple times before, but every time I go hunting for a source from Google/YouTube, I cannot find any official statements or confirmed information about this, seems this is mostly based on 3rd party analysis afaik.

Well it would be less than patreon, youtube the platform obviously costs more to run than patreon the platform

Perhaps set the account to only show ads until she’s caught up!

And by fixed you mean.. show more ads??

"We're sorry for any inconvenience this error may have caused"

[dead]

Why would anyone want this fixed?

Isn't that pretty unethical without an IRB and informed consent?

I am still paying for mine. My Google music all access I'm feeling lucky subscription turned into a YouTube music one, which includes YouTube premium, and I'm still only paying 8 bucks a month, which seems like a pretty good deal even if I also pay for Spotify.

> some long forgotten ad-free A/B test that it's not worth cleaning up.

That sounds about right. Different context but I once got on the good boy list at work by accident and it wasn’t fixed for about 6 month. On the first of every month I got a little corporate swag box in the mail thanking me for going “above and beyond”. Lots of cookies, blankets, coffee mugs and other trinkets.

The above is true. Some countries don't get ads. Lucky them. Albania, Cambodia, Ivory Coast, Laos, Myanmar, Macau, Madagascar, Maldives, and Russia.

Use a proxy server/VPN and go ad free.

Presumably he lives in the same place as his girlfriend.

This, and the fact that a fair bit of this would 'come for free' due to tvOS being based on iOS which has supported custom CAs for ages.

Of course Google can do this. And more. They could, if they wanted, to embed ads into the video stream itself with no way to distinguish them from the actual video content.

But they do not do it. They had so much time and opportunities to do that over the years. And yet, they did not do it.

I am not going to speculate why. But I suppose it is safe to assume that it is their intention to not do it.

> I wonder what is supposedly forcing the public CAs to do it for their certificates to be “valid”.

The power of browsers and operating systems including the cert in the default store distributed to everyone. Participating in cert transparency is a requirement.

Starting with Android 7, apps have to opt into user-installed certificates. Browsers often do (Firefox is an annoying exception, you need to turn it on in the dev settings and it doesn't work in the official release version of the browser), but apps usually don't even know that the setting exists.

Aside from that, Android has a very easy certificate pinning API where you can just assign a fingerprint to a domain name in the XML config files and it'll pin a certificate to that domain. Easy to bypass if you modify the APK file, but then you miss out on updates and other mechanisms could check if the signature has been tampered with.

With root access (shouldn't be too hard to gain on an Android device still running 7) you can add your certificate to the root certificate folder on the system partition. This will make Let's Encrypt work on all apps. It doesn't bypass certificate pinning, of course, but you don't need there for Let's Encrypt.

> but almost all apps nowadays use certificate pinning which means the system certificate store is ignored

Certificate pinning (or rather, public key pinning) is technically obsolete and browsers themselves removed support for it in 2018. [1] Are there many apps still really using this?

[1]: https://en.m.wikipedia.org/wiki/HTTP_Public_Key_Pinning

That sounds like you've just made it so your app doesn't work behind a corporate SSL proxy. I really need people to stop rolling there own SSL stores (looking at you python, java and nodejs). I spend way to much of my time getting things running on my work laptop that should just use the CA store IT pre-installed.

Certificate pinning seems like extreme overkill for nearly all applications. Are most folks really doing this?

> I find it extremely surprising that YouTube doesn’t do that.

Not surprising for me - it used to be only banks where it was required (sometimes by law) that any and all communication be intercepted and logged, but this crap (that by definition breaks certificate pinning) is now getting rolled out to even small businesses as part of some cyber-insurance-mandated endpoint/whatever security solution.

And Youtube is obviously of the opinion that while bankers aren't enough of a target market to annoy with certificate pinning breaking their background music, ordinary F500 employees are a significant enough target market.

What is "content"? I recently tried to use AppleTV Android app to play something and then use screen recorder app to record the phone screen. The recorder app was able to see the menus and even subtitles, but not the movie itself (black screen). Is the screen of my phone "mine"? Or does the manufacturer decide how can I use it?

Note that a significant amount of videos on youtube, and in particular many of the highest quality ones (e.g. educational material from schools like MIT or individuals like 3blue1brown) are Creative Commons licensed, so in terms of copyright, you are free to download and share them. Many including MIT's lectures are also NC and SA, so having the ability to save them and strip any ads is obviously in accordance with the wishes of the creators.

As far as youtube's wishes go, I don't think people should have much concern for a company that's engaged in predatory pricing for years to develop a monopoly through network effects.

> I don’t think there’s a justification to arguing you own YouTube or the content.

This actually gets to the core of my sentiment. I am influenced by these systems, but I can't directly influence them back. I don't know if this is somehow wrong in principle, but I definitely want more.

The problem is that advertisement business infects everything.

For instance, I could pay for Youtube Premium to ostensibly not be shown ads, but it doesn't change the fact that all the content[^1] in the ecosystem is still produced for maximizing watch time and/or being advertisement friendly.

I could pay for news, but that doesn't change the fact that the news is written to receive clicks from the non-paying users.

Paying for things does not help escaping the second order effects of advertisement.

[^1]: To a close approximation.

Every one of the streaming services that I paid extra to go ad free decided to push ads anyway.

Paying make it worse, paying doesn't prevent ads to be forced later (e.g: Netflix, Prime, Disney+) and split people fight against ad, as the ones with enough money to avoid them will berate the other for not paying, will still providing benefits to an ad-driven company.

Never pays to avoid ad, block them or get the content by other means. It's akin to "never negotiate with terrorists" or "never pay ransom", you have to remove the incentive.

I'm not that old and yet old enough to remember internets before ad-supported free content, which was just infinitely better.

You could pay for them or Google could choose to take a different approach that is less intrusive. The assertion here seems to put the onus on the viewer. Considering YouTube pays little to nothing comparative to its profits based on content it does not make, I think a realignment of how Google operates YouTube could be an improvement for users of the service.

> The users of the internet have made their call and they often don't want to pay, so someone does.

Just because YouTube users put up with a broken system doesn't mean it's the correct, fair, or ethical approach. Beyond that many of the views are curated via algorithms that intentionally work against the user with an end goal to hold them in a viewing state regardless of the users original intent. With that in mind users should use tools against those malpractices and not feel bad about not paying for them. If someone is intentionally trying to manipulate you, what's stopping you from doing the same?

If Google were a fair and ethical company I think treating them the same would be more in line with your response. However, they are not.

People have voted with their wallets on YouTube that they don’t want to pay for premium, and prefer to watch ads or block them.

Ads aren’t “forced” upon YouTube users, people have the option to pay but they just don’t want to pay.

YouTube can always choose to package the content in a financial transaction. They have chosen not to do so, and instead they are supplying advertisement alongside the content for which the viewer may or may not watch.

They can always change it, but then there are legal consequences of making it a financial transaction.

Paying users are too valuable not to serve ads to. Their clicks are worth more: ad-free tiers are always temporary.

Not that it matters: I pay for the bandwidth and hardware too. So I decide what it serves and runs.

For web access, in Firefox I've been using the "SocialFocus" add-on, that allows you to remove certain blocks in "social" websites (for example, blocking Shorts or comments in YT), put a color filter to make it "black and white", or even blocking the whole site. I had to access Facebook a couple times some months ago, and the quantity of trash you can filter with this add-on is astounding. This developer has also a YouTube specific add-on I haven't tested yet, named "UnTrap for YouTube" that has almost 200 different options for blocking very specific stuff there. Their add-ons in [0]

For Android there's an App called Revanced that let's you apply patches on certain commercial apps like YouTube or Twitter modifying their behavior, and for example block shorts. See the patches available for YouTube in [1]. I'm still pending to test it, but if you do, go to their official site [2], or even better, to their GitHub releases [3] as it seems like there are a good bunch of scammy sites using their name.

--

  0: https://addons.mozilla.org/ru/firefox/user/17777732/
  1: https://revanced.app/patches?pkg=com.google.android.youtube
  2: https://revanced.app/
  3: https://github.com/ReVanced/revanced-manager/releases

Wouldn't risk trying to extract from IG too much, I used to yt-dlp from it a lot and use scripts to extract the images because I like to archive references, nothing on a massive scale we're talking <20 times a month and I got a warning that I could lose my username if I "use automated scraping tools".

It's 11 years old, I'm impressed that it even works.

I'm not going to trust that until its on F-Droid.

Imo the best thing that can work is introducing delays to the loading of videos, increasing as time goes by. Youtube introduced sth like this to me, when they were presumable trying "punishing" users with adblockers, and it worked as a charm to get me disengage from the youtube rabithole. A lot of such addiction dynamics work based on how fast getting the reward is, and these interuptions disturb this.

Until the company decides to unilaterally reenable that setting to "help you get more from their service".

That's not persistent :(

Youtube have been gradually cracking down on yt-dlp by blocking IPs that download (presumably without watching the adverts, or some other method to fingerprint it). Currently it's mostly annoying as I have to rotate through IPs every few days. But I imagine it'll get worse and worse until I stop watching youtube.

Pretty sure it's only gonna get deleted if either a) enough people use it so that a MBAs notice or b) the way it's accessing the data blocks a feature that an MBA wants

Or just use a localhost invidious instance.

While I do agree with you, I am a bit concerned about the recent developments with "paid, but still has ads" subscriptions and how Youtube might slip towards such practices as well as soon as they have a large enough number of paying customers. Their premium might suddenly not be so... premium.

I wish I could do this for Spotify. Paid plans still include ads.

They cram ads into podcast episodes which themselves also have ads, so you'll get the read ads + Spotify's local ads + Spotify laughs all the way to the bank.

I believe over time not having ads will be a thing of the past, and you'll instead pay for fewer ads. Like where else are people going to go for exclusive content?

It's not that simple. It's common enough now to classify some ads as.... not adverts. So even if you pay for no ads, you get ads.

Or you can opt out. Both options are equally valid.

I won't pay for YouTube because the consistency of YouTube is massively variable. Sometimes channels I watch skip 6 months between videos. When I do watch stuff its usually in the background or when I just have a few minutes spare. Spending money to fill that time is unjustifiable unless it's a really low amount, and YouTube Premium isn't low enough yet.

Oddly though, if I could buy 100 'skip this ad' tokens for $10 that I could use when I'm pushed for time, but just suffer the ads when I'm not, I'd seriously consider it.

I think most people don't have a problem paying for something that gives proper added value.

But what's happening is that companies are degrading the basic experience and expecting people to either be OK with it (like Roku's increasingly intrusive ad experience) or to pay up to avoid it (like with YouTube).

> but you could just... pay for stuff.

Careful... In the Kingdom of the Netherlands your comment will be considered by a court of law as aggravated assault.

Netflix shows ads to paying customers. We've seen the same playbook across a wide variety of products and services, it's only a matter of time until paying users get milked too.

> I've gone back to pirating everything. I can afford to pay for all the services, it's just the service and content quality has gotten so bad that it's just not worth it.

If the content is so bad, then why the need to pirate it?

I started reading a great deal more at the start of the pandemic. I've kept it going since and it has been a real boon. I also switched back to physical books because I actually own them..

The alternative is paying for things you like so the people who make them can continue doing so. If you don’t think YouTube is worth paying for, it might be a good idea to reconsider the amount of time you spend on it or whether you want to help promote it.

> but youtube is still limited, and the only way to avoid the ads is to buy another device (computer)

YouTube premium is cheaper than another computer and works on all devices.

Alternatives:

- Not consuming exploitative entertainment

- "Piracy"

I like the download-YT-content-locally then play. A project hit the front page recently that used yt-dlp to more or less do this.

Piracy isn't an alternative - it's illegal/immoral.

But, the good news is there are two alternatives for all of the above... pay for the content. Or, don't consume the content.