What would happen if we didn't use TCP or UDP?
115 comments
·February 25, 2025pkkm
jeroenhd
SCTP is fascinating because it's one of the backbone technologies that makes communication possible for most people on the planet (as the mobile network stack pretty much relies on it), yet it's effectively unsupported on almost every consumer device. If you want to use it, you're probably going to have to ship a userland implementation that needs privileges to open a raw network socket, because kernel implementations are rare (and often slow).
We could've had it as a QUIC replacement if it weren't for terrible middleboxes and IPv4 NAT screwing everyone over once again. Hell, NAT wouldn't even have been an issue had SCTP support been widespread before consumer NAT devices started being implemented.
It's a prime example of how new network protocols now have to lie and deceive to work over the internet. TLS needs to pretend to be TLS 1.2, QUIC needs to pretend to be an application on top of UDP while reimplementing SCTP and TCP, and even things like secure DNS are now piped over HTTPS just in case a shitty middlebox can't deal with raw TLS.
tsimionescu
While the gist of your post is spot on, I do feel it should be noted that DoH is preferred over DoT not to protect from middleboxes that don't work properly, but from middleboxes that are actively trying to outright censor encrypted DNS, but can't afford to snoop on/prevent all HTTPS traffic. It's an anti-censorship measure, not a compatibility measure.
capitainenemo
It's also about user privacy with ISPs as well as anti-censorship. https://www.cloudflare.com/learning/dns/dns-over-tls/
bewo001
Actually, every browser supporting webrtc datachannel supports SCTP over UDP.
tepmoc
Diameter in mobile network is heavy user of SCTP, but from what I've read they moving away from diameter into HTTP calls for 6G.
bennettnate5
For communication between carriers and some communications within a given carrier's network components, yes. Base stations all still communicate with the core network over SCTP, though.
api
It’s too bad the original IP could not have included some kind of stronger header integrity lock to block middle boxes.
It would have forced us to adopt V6 and… my god… the network would be so much cleaner and superior in every way. Things would just have addresses, and protocol innovation at L3 would be possible.
hylaride
I'd like to agree with you, but IPv6 had such a classic case of suffocation by committee during its birth that I doubt it would have been pushed by such a situation. IPSec was mandated in the original IPv6 RFC, for god's sake. That alone delayed a lot of work in implementing it as crypto code needed to be integrated into kernels, which was not common in those days. That's to say nothing about the fact that IPSec is loosely defined enough that setting it up between different vendors is always an adventure - adding support to an IP stack was a big headache (I followed OpenBSD at the time they were integrating IPv6 in the early 2000s and there was a lot of hard problems around compatibility).
Header integrity was so far off of consideration during IPv4's implementation because the internet was a dozen universities and DoD sites that it was overkill (and possibly a waste of limited CPU cycles at the time).
What's far more likely to have happened is that we'd see more proxies instead of NATs (SOCKS, etc). I don't think that'd be better than NAT.
nly
It gets worse in practice. Congestional control algorithms on the internet need to play nice with TCP Cubic regardless of how good they are.
nesarkvechnep
As with most of the network stack, *BSD implementation is the reference implementation.
immibis
What we can learn from this is that there are networks other than The Internet, and even The Internet can be subdivided into parts that don't really work together.
valorzard
SCTP is really cool, I first found out about it because it’s the basis for WebRTC data channels. It’s basically reliable UDP, but you can turn off the reliability if you want. Makes me wonder why QUIC exists when SCTP does…
15155
> why QUIC exists when SCTP does
Because QUIC uses UDP, which is supported by most/all intermediate routing equipment.
nly
The whole point of UDP is to allow alternative protocols to be implemented on top.
SCTPs mistake was it wasn't implemented as a userland library on top of UDP to begin with.
noam_k
Is this a real issue? SCTP runs over IP, so unless your talking about firewalls and such, the support should be there.
Edit: a quick search showed that NAT traversal is an issue (of course!)
saurik
WebRTC runs SCTP over DTLS over UDP.
lttlrck
That's not the reason, SCTP over UDP was already standardized
Imustaskforhelp
I am now genuinely wondering
Maybe its me being stupid but why don't we use quic always instead of tcp?
I think it has to do with something that I read that tcp can do upto 1000 connections simultaneously no worries and they won't interfere with each other's bandwidth / impact each other , but udp does make it possible for one service being very high to impact other.
There was this latest test by anton putra with udp vs tcp and the difference was IIRC negligible. Someone said that he should probably use udp in kernel mode to essentially get insane performance I amnot sure
signa11
> It’s basically reliable UDP, ...
more importantly though, it transmits multiple independent streams of message chunks in parallel.
similarity with UDP ends at message oriented nature of the protocol. closest equivalent for TCP would be MPTCP I suppose ?
0x457
Because pure SCTP can't survive outside your LAN, thanks to everything in-between you and your destination. Why not use SCTP on top of UDP? Well, because one of the main benefits of QUIC is TLS being at its core.
SCTP you're talking about runs on top of DTLS on top of UDP. DTLS has issues on its own, but even if it didn't it wouldn't beat QUIC in TTFB.
bdd8f1df777b
Others have mentioned protocol ossification which is indeed the primary reason. A secondary reason is that QUIC fuses TLS so its latency is further reduced by one RTT. For high latency networks, the difference is palpable.
ianburrell
SCTP can run over UDP.
QUIC is supposed to be faster than SCTP by combining layers and eliminate round trips. Also, QUIC is a stream protocol like TCP. SCTP makes messages explicit. Both have multiplexing which is why seem different.
immibis
It's actually universal within a certain niche. I think phone networks are doing just about everything over SCTP internally. When SS7 links get replaced they get replaced with something that uses SCTP. Not sure of the details because I don't work there.
Related: there's a parallel Internet with a different root of number allocation called GRX/IPX (GPRS Roaming Exchange/Internetwork Packet Exchange)
AnotherGoodName
IPX is another that was very common just 20years ago. Many old games only support ipx networking and you need to run an ipx over tcp emulator to play them multiplayer nowadays.
dakra137
TCP would be fine if it had the concept of message, in addition to stream. The sender sends a message. It flows to the recipient. The recipient program can specify that a receive receives the entire message, no more and no less, as long as the application's target input buffer is large enough.
SCTP does this.
A shim on top of TCP socket receive could also do this also, as long as there is a convention to prefix each message with a length field, say 16 bits, with the MSB indicating that the message is incomplete and is continued in the next length delimited segment.
nubinetwork
I thought the point was that they wanted to use something that didn't exist yet (either in RL use or in RFC form)...
Emjayen
As someone who has implemented various transport protocols (both standard and custom) the biggest hurdle in layering atop IP will not be the gauntlet of WAN routers, surprisingly, rather consumer NAT devices.
One interesting case for a particular Netgear family of routers was that, while the traffic would survive end-to-end it was not without corruption - very specific corruption which took the form of zeroing the first 4 bytes. Given this aligns with where the src/dst ports would normally be, I suspect it was being treated as TCP/UDP albeit without the actual translation path taken.
LinuxBender
What would happen if we didn't use TCP or UDP?
One would have a hard time communicating with anyone. The internet has standardized around TCP and UDP. There are too many devices in most paths that will not be able to handle other protocols. Replacing all the hardware on the internet would take even longer than deprecating IPv4 in my pessimistic opinion. To get around this there would have to be some significant gains of the new protocol that would warrant big expenses in every corporation and government and all the hardware manufacturers would all have to agree on implementing support and further agree on interpretation of the new RFC.
mrbluecoat
Agreed. This statement sums up the obvious:
> Granted, the server was just two hops away from the client, and it didn't have to pass through the scary sea of the internet
immibis
Actually the Internet at large is fine with any protocol on top of IP. It's your home router's NAT function that can only handle UDP and TCP. Set it to bridge mode and use your computer as the router (if you need one) and you can send anything from that computer.
If you have CGNAT you're still screwed. Get one of those free ipv6 tunnels.
globular-toast
What you want is a packet socket:
sock_raw = socket(AF_PACKET , SOCK_RAW , htons(ETH_P_ALL));
ay
To second this and expand on the reasons behind:
The way the NATs (network address translators) are sharing the scarce public IPv4 addresses is by multiplexing on the transport level fields (ports in case of TCP/UDP and IDs or inner packet transport level fields in case of ICMP).
Since they are unaware of your protocol, they get into a “special case mode”, which on a naive translator might consume a whole IP address (so you would really make a network admin with a few of those, because you exhaust all their available addresses :-) ; but on the carrier grade NAT there are safeguards against it and the packets are just dropped.
elcritch
At some point it seems like we just need to start passing laws requiring (major) internet services to provide all services on IPv6. Routers could then intercept ipv4 only devices and their dns locally and translate it to ipv6.
Still lame that in 2024 major services like Steam and Quest basically require IPv4.
I want to be able to use the internet without silly things like exhausting a network admins ipv4s.
WorldMaker
At least cell phone networks through their standards processes have pushed most consumer networks and consumer hardware to be IPv6 by default ("by only option" in many cases with DNS64 and NAT64 filling in the gaps).
The real pressure we need are to corporate networks. Too many of them think they can use 10.0.0.0/8 forever. Too many of them own giant chunks of public IPv4 space and think they are immune to the address exhaustion. At least the prices for IPv4 addresses are going up at major clouds like AWS and Hetzner. But it still seems too slow of a price rise to hit enough bottom lines that major corporations are feeling the pressure yet.
geokon
This is outside my area of expertise.. so naiive question.. but ports aren't tied to the protocol .. right? If you open a raw socket, it's still on some associated port number. NAT traversal multiplexes ports.. so why would that preclude using any arbitrary protocol?
tsimionescu
Ports are very much a concept of the transport layer. They are a very useful concept, so they are used in all major transport-layer protocols, but they are not necessary in a theoretical sense (though a transport layer protocol without them would only allow a single stream of traffic between any two machines, or at least IPs). But TCP port 22 is a completely different thing than UDP port 22, and they are both completely different from SCTP port 22. To prevent confusion, IANA typically assigns the same port number for a protocol on both TCP and UDP (e.g. DNS over TCP uses TCP port 53, just like DNS over UDP uses UDP port 53; and QUIC uses UDP port 443, just like HTTPS uses TCP port 443).
When a machine receives a packet, after the Ethernet and IP layers have made sure the packet is addressed to this machine, the very next thing that happens is checking what transport layer implementation should receive the packet - and this is done based on the "transport" bits in the IP header. If the packet is, say, TCP, then the TCP layer starts reading its own header and finds out the port number, and proceeds from there.
numpad0
TCP/IP is a protocol called TCP encapsulated in protocol called IP. Raw IP packets have "protocol number" for use by payload, but not port number. Port number is technically part of custom data that IP layer should not care about.
If you mean socket as in `my_socket = socket(AF_INET, SOCK_STREAM, 0);` that's TCP/IP, not raw IP, so it will have port numbers in the TCP part of the packet. `SOCK_STREAM` and `SOCK_DGRAM` respectively correspond to TCP and UDP. Raw IP sockets on Linux can be created by `socket(AF_INET, SOCK_RAW, protocol);` and that will have no TCP/UDP header attached by the Kernel after the IP header in the packet.
(googling a bit as I write, forgive errors)
Hikikomori
IP has no ports, it has a protocol field, TCP, UDP, IPsec and other layer 4 protocols have their own protocol number. NAT (PAT really) uses TCP and UDP ports to connect what can be considered separate connections, ie a connection on the inside and a connection on the outside, these connections don't even have the same source/destination ports as connection state tracking is what connects these separate connections.
The socket API has different types of sockets, typically you use stream (TCP) or dgram (UDP), but you can also get a completely raw socket where you need to construct your own ethernet and IP headers, or create your own replacement of IP protocol. So socket does not mean only TCP/UDP or something with ports, unix sockets are another example of this.
nlitened
Ports are part of TCP and UDP, not IP, as far as I can tell. So ports are tied to the protocol
globular-toast
When using a packet socket you are sending bytes directly to the device driver. The only ports at this level are the physical ports on your machine which will have names like "eth1" etc. Assuming an Ethernet driver, the bytes must be a valid Ethernet frame[0], but that's all. The payload is just bytes.
withinboredom
Ports are tied to the protocol. IP layer doesn’t have ports (it’s in the name).
the8472
IPv6 alone won't help if there are some firewalls sitting between the endpoints that will drop anything they don't know. Having two linux hosts without firewalls talk to each other over IPv6 without consumer grade routers (which tend to come with firewalls by default) between them might work.
immibis
This gives you a raw Ethernet socket. For a raw IP socket, use AF_INET,SOCK_RAW,0.
A raw Ethernet socket sends packets at the Ethernet level while a raw IP socket sends packets at the IP level. If you want to play on your local network you want Ethernet (maybe). If you want to send weird packets across the Internet you probably want IP, so that you don't have to waste effort doing route lookups and MAC address lookups and your code will still work on non-Ethernet networks, including VPNs.
There are some protocols that run on top of Ethernet but Internet-compatible protocols all run on IP by definition.
This comment was delayed several hours due to HN rate limiting.
globular-toast
Yeah, this was intended to eliminate the problems with protocol=0, but yeah you need to implement IP and maybe ARP to do a similar thing at this level. Thankfully IP is pretty simple to implement.
Hawzen
That's very interesting!
I'll try out IPv6
globular-toast
If you want to play with packet sockets you might find my notebook useful where I was going to implement TCP/IP from scratch (well, from layer 2 up): https://github.com/georgek/notebooks/blob/master/internet.ip... I did ICMP over IP but got bored before I got to TCP, though (it's way more complicated than IP). You could drop in your custom protocol in place of ICMP.
MisterTea
There's also Bell Labs IL - Internet Link - used to speed up 9P links for Plan 9 over LANs. https://doc.cat-v.org/plan_9/4th_edition/papers/il/
And this brings me to why I love networking on Plan 9. First off the dial string, net!address!service, passes the network along with the address and service port letting you easily switch protocols as needed. e.g. a program could listen on IL port 666 using the dial string il!*!666 and the client dials il!10.1.1.1!666. Second, that lets you dial and listen on any protocol from any program. If one wanted to use raw Ethernet you use the dial string ether0!aabbccddeeff. If I wanted to add a protocol like quic or a dead one like ipx/spx I just need to mount it to /net and keep the semantics the same as other services in net and any program can dial that protocol - ezpz user space networking stacks. Powerful abstraction.
kjs3
Reminds me a bit of UUCP or AT&T Datakit addressing.
MisterTea
Datakit was in fact one of the early supported networks in Plan 9. You could dial into a 9 machine over datakit, mount an outward facing IP stack over your /net and your on-line: http://man.postnix.pw/plan_9_2e/3/datakit
kjs3
Interesting, and I suppose not surprising given the source. AT&T kept on with Datakit long after circuit switched data networking became passe.
fancyfredbot
It feels like the article ends with a cliffhanger!
Why did a single packet of the custom protocol get through with all later packets dropped? Does anyone know?
kazen44
my guess is the first packet got through a firewall, which created a flow.
subsequent packets then got blocked because the firewall has no way of matching this to an existing flow.
m3talsmith
Definitely the most likely
wrs
Yeah! I am baffled that even one got through, but given that it did, why only one? And I would’ve immediately tried the “every protocol” version at that point…
paweladamczuk
I always assumed any TCP/UDP packets would get captured by the OS network stack in order to be sent only to the processes listening on specific ports.
I guess this is a security feature, since a process cannot even listen on some ports without having elevated privileges. I wouldn't expect another process being able to capture all this traffic anyway. This would also require a mechanism of sending the same stream to multiple processes (TCP listeners and all-protocol listeners).
But I didn't even know it was possible to capture traffic from multiple transport layer protocols using a syscall, perhaps that syscall requires elevated privileges itself..?
Hawzen
> perhaps that syscall requires elevated privileges itself..?
You are exactly right
globular-toast
It requires elevated privileges, but this is how programs like tcpdump and wireshark work. On Linux it's also possible to give a program these permissions for any user by setting "capabilities", specifically cap_net_admin and cap_net_raw.
netbsdusers
DHCP services require this ability to receive and send UDP packets on raw sockets, barring a few advanced systems like Solaris that provide them with necessary facilities. Usually they install a BPF module on the socket to filter out uninteresting packets.
sema4hacker
I think a more interesting question is: what if internet protocols and routing equipment were designed from scratch today? Besides much larger packets, I'm guessing something basic in the style of UDP would be chosen to replace HTTP to simplify query-response lookups, a much simpler streaming protocol would be chosen to replace TCP and support all the video playing going on, and those two protocols would more efficiently handle the vast majority of traffic.
ddejohn
There have been a few of these types of discussions on HN -- search phrases like "internet from scratch" -- really fun reading.
wmf
QUIC is redesigned TCP.
dakra137
We might use Ethernet WAN and LAN. See https : / / neosnetworks . com / resources / blog / what-is-ethernet-wan/
pjmlp
We would be using one of the other protocols that were common at the time, before TCP/UDP/IP took over everything.
casey2
It's weird to me that the definition of portable has changed from "can be moved" to "can be moved to another machine" to "can be moved to another environment" to "can be moved to another program"
We had the option to make every object addressable and we chose not to. Why keep sweating over it, just accept that software only works on the devs machine and ship that.
jmclnx
We would all be on UUCP. That was doing similar things before TCP/UDP.
kjs3
UUCP was only a couple years before TCP, and isn't really an equivalent (it's fancy file transfer with a little remote command exec sprinkled on in the end) and natively its dialup oriented; it usually requires some other L3 protocol to run over networks.
There was a time when IPX/SPX was a contender. Xerox pitched XNS directly at TCP. DECNet/OSI was around. There were a lot of others...lot's of experimenting going on at the time.
immibis
UUCP's primary distinction is that it's store-and-forward.
Karrot_Kream
If you're interested in a more "modern" UUCP, there's NNCP [1] (HTTPS ver here [2].) It continues to mostly be a file-transfer protocol with a bit of signaling added on.
You don't need to make up your own for this experiment. There's already a pretty old protocol that's far superior to TCP, but failed to get adoption because of network hardware dropping everything other than TCP and UDP. It's called SCTP.