Skip to content(if available)orjump to list(if available)

A new generation of Tailscale access controls

mrbluecoat

> When do I have to update all those policies to use the new syntax? ... You never have to update your existing rules to use the new grants syntax. We will support our original ACL syntax forever. ... When we say that Tailscale is a stable platform, we aren’t kidding around.

Nice business decision. Just like their reaction to Headscale, I'm reminded Tailscale 'gets it' when interacting with the dev community and why their popularity will continue to grow.

OJFord

I reckon this (and that it can be used in conjunction with grants) implies they're translating the old config to the new format if it exists on the fly. Otherwise, if it involved running two things one in maintenance mode (and somehow ensuring they both take effect and don't conflict) it'd be ballsy to promise 'forever'.

Although not sure why that's preferable to converting it once-off, so it's still not something a user has to worry about, but then the old way is done with.

yurishimo

Stripe has been doing it for god knows how long now and it seems to be working for them. With a robust test suite, I can’t think of too much that can/would go wrong. Honestly, I wish more software would approach things this way instead of introducing a major api version every 5 years. Just continuously ship the latest thing and translate everything that came before. Hopefully if people are also using “official” client libraries, they will eventually upgrade their dependencies and their API version will also get bumped. If there is a particularly heavy usage client jumping forward 100 versions and using too many of your resources, hopefully your org can dedicate some resources to help them upgrade which theoretically benefits both of you.

Now that I think of it Adyen did this to us a few years back. We’ll probably stay in the v5 branch until they politely poke us to upgrade again in another 5 years…

tczMUFlmoNk

> Although not sure why that's preferable to converting it once-off, so it's still not something a user has to worry about, but then the old way is done with.

This way also supports users who have scripts or workflows that produce the old form of ACLs. If you had a single cut-off date, those scripts would presumably need to be updated.

csomar

They probably have a parser that parses to some other format. They only need to maintain the parser and they probably won't support new features on it.

OJFord

Idk what you think I said, but yes, I agree.

gessha

How did they react to Headscale? I must’ve missed the announcement.

matthewaveryusa

They let it be. The decision is many-fold correct:

1) hobbiests that are obsessed with self-hosting can use it -- tailscale will never see a penny from them anyways

2) some businesses will consider headscale as an escape hatch de-risking if tailscale were to be purchased out by an extortionist (like broadcom's acquisition of VMware)

3) It signals they are doing business in a position of strength: businesses don't buy software, they buy solutions to problems

godelski

4) Tailscale can benefit from innovations that Headscale finds.

Competition can be beneficial for all parties involved. If the Headscale devs make novel contributions and solve problems Tailscale didn't think of or figure out themselves, Tailscale can copy Headscale just as Headscale copies Tailscale. Competition has this benefit of distributing research expenditures.

I think it is easy to forget that even monopolies are bad for monopolies. They can benefit by being fat and lazy, but this is at the cost of larger future profits. It's "safer", but there's plenty of competitive markets where there is little risk of failure. At least compared to the risk that naturally exists.

5) Headscale users may convert to Tailscale users.

That can happen if Headscale was implemented by a enthusiastic employee who later leaves. Company learned to like the benefits but no longer has the support. This makes a natural conversion. The same thing happens if Headscale fizzles out.

I really just don't see how Tailscale really has much, if anything, to lose from Headscale. It definitely has things to gain! The only reason to go after Headscale would be if they are greedy and irrationally fearful. Which the action itself would scare a lot of its users. Tailscale really is in touch with its users and honestly, I'm not sure there's anything negative I have to say about them. They're not creating enemies, they're paying attention to users, and they're just focusing on making a great product. Seems like the ideal strategy that we'd want all businesses to follow.

aspenmayer

I'm not sure what they meant, but Tailscale links to Headscale and says this:

> Headscale is an open source alternative to the Tailscale coordination server and can be self-hosted for a single tailnet. Headscale is a re-implemented version of the Tailscale coordination server, developed independently and completely separate from Tailscale. Headscale is a project that complements Tailscale — with its own independent community of users and developers. Tailscale does not set Headscale’s product direction or manage the community, and neither prohibits nor requires employees from contributing to Headscale.

> Our opinion is that Headscale provides a valuable complement to Tailscale: It helps personal users better understand both how Tailscale works and how to run a coordination server at home. As such, Tailscale works with Headscale maintainers when making changes to Tailscale clients that might affect how the Headscale coordination server works, to ensure ongoing compatibility.

https://tailscale.com/opensource#encouraging-headscale

SOLAR_FIELDS

The distilled version of this is that they let Headscale, a FOSS, use their proprietary client to connect to the Headscale server rather than Tailscale’s. They don’t need to allow this, they could easily disable that feature and have every right to, but they don’t. This makes the development of Headscale a lot more straightforward since the entire client codebase is just removed from the equation.

dgb23

An excellent way to position yourself towards an open source alternative. Everything they do seems to be grounded in an engineering mindset and with respect to users.

skydhash

I think that you would've lose those users anyway, so at least leave a good impression on them as they can become advocate when they're in another context where Tailscale is the better solution.

csomar

My guess is they probably don't care because their pricing is really low. If you want to self-host, for any serious company, you need a very resilient setup (multiple servers) because you can't afford to have your "authz" down. If your headscale is down, so is everything in your organization. $6/month/user is then relatively cheap. The savings on self-hosting will be minimal.

Lammy

> How did they react to Headscale? I must’ve missed the announcement.

They hired the dev: https://tailscale.com/blog/opensource#the-open-source-coordi...

Fun fact: Tailscale have an almost-assuredly-accurate count of Headscale users due to how few people disable the Tailscale client's telemetry which by default sends them real-time events about everything you do on a Headscale network. See KB1011: https://tailscale.com/kb/1011/log-mesh-traffic

“Each Tailscale agent in your distributed network streams its logs to a central log server (at log.tailscale.io). This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”

It's possible to opt out of this spying on Unix/Windows/Mac by starting Tailscale with `--no-logs-no-support` or `TS_NO_LOGS_NO_SUPPORT=true` environment variable (see https://tailscale.com/kb/1011/log-mesh-traffic#opting-out-of...), but it is not currently possible to opt out in the Android/iOS clients: https://github.com/tailscale/tailscale/issues/13174

This is why they had the data telling them they should hire the Headscale dev to nip that competition in the bud before it had any chance to usurp their business by overtaking them in mindshare using their own client. Definitely a sound business decision, but it really really irked me when I realized I was feeding Tailscale detailed usage data for everything I did on my supposedly “““private””” Headscale network despite having no business relationship with them.

For an example of how invasive this is for the average user, this person discovered Tailscale trying to collect ~18000 data points per week about their network usage based on the number of blocked DNS requests for `log.tailscale.com`: https://github.com/tailscale/tailscale/issues/15326

The privacy-adversarial behavior made me switch from Headscale to NetBird for my personal network now that it's available in FreeBSD Ports: https://www.freshports.org/security/netbird/

ElectricalUnion

> For an example of how invasive this is for the average user, this person discovered Tailscale trying to collect ~18000 data points per week about their network usage based on the number of blocked DNS requests for `log.tailscale.com`: https://github.com/tailscale/tailscale/issues/15326

18000 data points per week seems IMHO pretty low (only 1.7 requests per minute for a whole network? Unlikely, even my Android phone does way more that 1.7 requests per minute just to ad networks, nevermind everything else on the network summed), it's probably way more.

18000 data points is also a lie, that's a different issue - the issue of UDP DNS sucking in general so your average application/OS keeps reissuing requests if they think the UDP packet got dropped.

eadmund

That’s appalling. I really feel like a fool — I had thought the world of Tailscale but no longer.

This needs to be more widely known.

godelski

  > It's possible to opt out of this spying on Unix/Windows/Mac by starting Tailscale with `--no-logs-no-support` or `TS_NO_LOGS_NO_SUPPORT=true` environment variable 
Am I misreading the docs?[0]

  Network flow logs are available to help you understand which devices are connecting to one another over time, that is, the flow of traffic across your tailnet.

  These logs strictly do not contain any information about client operations or contents of network traffic.

  Network flow logs must be enabled[1].
This makes it look like they are disabled by default.

[0] https://tailscale.com/kb/1011/log-mesh-traffic

[1] https://tailscale.com/kb/1219/network-flow-logs#enable-netwo...

mac-attack

Randomly stumbled across this last week when I upgraded my DNS blocklist and saw all the tailscale logs. Definitely unexpected

neodymiumphish

I believe they also hired the developer of headscale!

codethief

Speaking of ACLs and grants, I wish Tailnet Lock[0] extended to those as well, i.e. allowed me to sign my tailnet policy file, so that if the coordination server and one of my nodes ever get compromised, the attacker can't just modify the policy file to access arbitrary services in my tailnet from the compromised node, including – in the worst case – SSH[1].

[0]: https://tailscale.com/kb/1226/tailnet-lock

[1]: https://tailscale.com/kb/1193/tailscale-ssh

tpetry

I would love for Tailscale to work on a "paranoid mode". At the moment, you can be sure that Tailscale can't add machines to your network with Tailnet Lock where you have to authorize a new machine from a trusted node and not from the UI (which Tailscale could also do without me clicking it). That's great feature compared to their competitors which basically have a backdoor into your network.

But changing ACLs and many other changes can be done from the WebUI without needed authorization from a host. So I have to authorize the change in the UI - but Tailscale could really also do it without my consent. Would be great to have a mode that _ANY_ change to the Tailscale network must be authorized from a trusted node with a detailled changeset. So I could be sure that Tailscale can't tamper with my network.

Sure, I could run Headscale and completely work without the Tailscale servers. But I prefer to pay them the small monthly fee to not have to manage this central service of my fleet.

ttul

We were reviewing our SaaS spending recently and I noticed that Tailscale is now our biggest spend. Their approach is apparently working.

bb88

Tailscale at least on Linux is a mess of surprise configuration and netfilter rules.

It's not for everyone, but if you're into control of your netfilter tables, it's certainly not for you.

haiku2077

It's a lot more sensible if you use vanilla iptables. And it's far less of a mess than Docker, at least.

bb88

Right, but then go combine the two and see what happens.

haiku2077

Hehe, I just uninstalled Docker from my server to simplify things

globular-toast

I recently discovered this is how Calico works and it can indeed be really surprising when it goes wrong. At least in a k8s context you have the option of just burning the node and starting again rather than trying to debug it, though.

dataangel

Okay but when can we be connected to more than one tailnet at once and have tail scale auto reconnect directly after resume from sleep? That's the quality of life stuff users really care about

a_t48

Vaguely related - there’s still no easy way to get at the email/user name of a user using tailscale ssh right? This is one of the things I really liked about teleport, you could use it to properly attribute git commits on shared machines, without any special setup on the user side.

aatd86

There seems to be a missing comma in the first grant example. If I am not beffudled. At the end of the dst line.

In fact, comma usage seems inconsistent for the last element of some of the nested objects.

To be fair, I don't remember what's the json rule either and would need a quick lookup.

jrockway

They use hujson: https://github.com/tailscale/hujson

But it doesn't accept things like the example that omits an intermediate comma:

  {
      "foo": 42
      "bar": 123
  }

  >  hujson: line 3, column 5: invalid character '"' after object value (expecting ',' or '}')

orthoxerox

They should've gone with Relaxed JSON, which is more relaxed than JSON5, which is more relaxed than HuJSON/JWCC.

stavros

Yeah, the example that's missing the commas is wrong, if I'm not mistaken about their json parser. The optional comma at the end is fine with JSON5, which I think they're using, as that allows optional commas, comments, things like that.

SOLAR_FIELDS

When would I want this over a more generic solution like linking to an idp using something like keycloak? The upside is that its network level blocking, but it still requires modifying the application itself same as connecting an idp, correct? The downside of this approach is that it only works for internal access where your end users are on the tailnet

ElectricalUnion

> When would I want this over a more generic solution like linking to an idp using something like keycloak

I assume everyone using Tailscale are opting for "simple code and adminstration" instead of "let me pierce NAT and handle Wireguard tunelling myself". On that point of view, "Identity management" is part of "stuff" that Tailscale helps you in the first place. So why not use it as the "Identity management" solution as well?

When you use `tailscale serve`, you get the headers Tailscale-User-Login, Tailscale-User-Name and Tailscale-User-Profile-Pic that identifies who connected thru the tailscale endpoint. https://tailscale.com/kb/1312/serve#identity-headers

On a separete side note, if you had an exposed OpenID Connect service in the first place, you can use it as the Tailscale tailnet auth SSO: https://tailscale.com/kb/1240/sso-custom-oidc

red0point

Doesn‘t tailscale force you to bring your own SSO / IdP already?

So it‘s not part of what Tailscale brings, it just adds another layer of indirection between the SSO / IdP you have already and the app, plus requires some custom library integration work, further enhancing lock-in.

miki123211

If you write internal apps, this is a lot easier to support.

You don't have to think about redirects, sessions, cookies, syncing your users and their roles in TS and Keycloak, maintaining two separate policy files and all that.

SOLAR_FIELDS

The tradeoff you get is lock in to tailscale and no portability. Though I assume that is true about some of the networking features tailscale offers as well.

mousetree

It’s at the network level. It will block clients from connecting to the destinations on the network. You don’t need to modify any applications at the destination.

SOLAR_FIELDS

Am I misinterpreting like, the entire body of the article?

> For private services inside your tailnet, like internal tools, implementing auth on every app is overkill. That’s why we developed tsnet, a Go library that embeds Tailscale directly in your applications. This lets you see and react to the identity of every user who makes a request to your app. So, authentication is handled…what about authorization?

The article then proceeds to show how to modify an application using tsnet to embed in the application. Which sounds a whole lot like modifying the application. The point might be that it’s more straightforward to do this then gate behind something like keycloak, but fundamentally it still requires modifying the application

mousetree

Ah yes. This seems to be something new and entirely optional. They’re saying that it might make sense to some to keep both network and application level authorization defined in one place. Personally, I’m with you in just using a normal SSO solution.

nikisweeting

Will there eventually be a UI to manage grants so we don't have to write them by hand?

maria4151

[flagged]