Spammers are better at SPF, DKIM, and DMARC than everyone else

The problems I noticed were, it doesn't matter what the SPF and DKIM look like. If Google or Microsoft refuse to relay your email based on secret internal factors then you're out of business.

As a non-email guy, I can tell you that if a system that boils down to having an (optionally certified?) key requires much more than just putting it into a folder with a domain name and running a service, it’s badly designed and has unnecessary complexity. Which will result into abusers having more expertise than legitimate users. The fact that you can “get” DMARC SPF DKIM wrong, while it’s basically a hard requirement for operation, is just screaming something important to the email software.

> Russian IP addresses are still trying to send email in the name of my domain for some stupid reason

For what it's worth, I've started seeing cybersecurity insurers requiring riders and extra payments if you don't block Russian IPs.

In most organizations there is no point in a sysadmin to spend the effort in understanding how to set it up correctly as Marketing has got more authority on email. Marketing will simply demand changes to the config that they do not understand and there is nothing you can do to stop it as they will have the CEO on their side.

> that Russian IP addresses are still trying to send email in the name of my domain for some stupid reason

You can set your policy to reject, that will deter the Russians from using your domain.

It's easy, you just have to have a regular, decently sized volume of non-spam emails, and suddenly your email stops being marked as spam!

The logic isn't even that bad. SPF and DKIM serve to prove to the email who the sender is. That doesn't mean much if the sender is a spammer. Verifying identity claims is only the first part in checking email for spam, the harder part is checking if that identity is someone you trust.

When you email Outlook or Google, you're better sending more than a few every single day, and the recipient better manually drag those emails from their spam folders to their inbox, or they're all being learned as spam.

I worked on this for a while, at a time and in a market where most of our recipients had @hotmail addresses. I discovered that mass email sending was akin to a "pay-to-win" game.

We had/opted to acquire the services of a company "expert in email deliverability" (Return Path), who somehow provided detailed metrics of how our IPs were scored by MSFT. I always wondered why MSFT didn't provide those scores by themselves, and how a 3rd. party could have access to them.

Re. your comment... slow ramp-up is the only way, with constant monitoring of deliverability and consequent adjusting of recipients (i.e. removing those who do not open or hard-bounce). I did also wonder if paying that company perhaps gave us a headstart when adding new IPs...

It's almost like all those bad actors (linkedin) are owned and controlled by the big players (microsoft) that benefit from email being only commodity they can provide.

I think the domain rep is worth less than IP rep. I had occasional issues sending issues when I self hosted on a VPS. When I moved my domain to Fastmail I haven’t ever had my emails go to spam.

Most home and VPS IP ranges have negative rep.

If it's a new domain, then your problem isn't reputation exactly, it's having a newly-registered domain. Buying a new domain, setting up the SPF, DKIM, and MARC, and then immediately spamming from it until it's banned everywhere a week later is standard spammer MO.

I've been self-hosting mail for me and my family for about 20 years and don't send nearly enough mail to have a "reputation" with anybody. Still, I don't have any problems with deliverability of mail.

> Actually getting SPF, DKIM and DMARC right and having a domain with a 0 spam score will still land you in the spam directory.

This little bit of wisdom gets passed around all the time, but it's actually not true. You can send email from a brand new domain to Google and Microsoft and whoever just fine. What you can't do is send email from a brand new domain, and a brand new email server--or an email server on a VPS, or an email server on a residential IP. Residential IP blocks are almost completely blocked, because of unsecured devices being used to send spam, and VPS blocks have the same problem. You can get around this by using a mail relay, or building your domains reputation on a server that already has a good reputation.

This isn't a problem for personal emails, as after a request or two friends will unspam you. Google blackholes emails, breaking all mail logic (no bounce), so I assure you the SPAM folder is a good gmail sign.

I would imagine that on the corporate side, your employees could do the same. Beyond that, if you're sending spammy stuff, have unsubscribe headers and links in emails.

>SPF/DKIM is really about mail server reputation. So it mostly benefits larger servers like the ones run by Google, Microsoft and Yahoo. Unfortunately, that means that attempts by those larger providers to combat span using such reputation will naturally hurt smaller providers. So the actual effects of SPF/DKIM are on the whole negative.

That paragraph is incorrect. SPF/DKIM is not about reputation. The main purpose is preventing domain impersonation from unauthorized senders. E.g. mail servers will reject fake emails from "upofadown@microsoft.com" because you don't control any email servers that's whitelisted in microsoft.com DNS TXT records.

E.g. I was able to register a brand new .com address and then successfully send to gmail and MS Outlook accounts within minutes because I had proper SPF/DKIM in the DNS records for that new domain. That new domain had zero reputation and yet Gmail accepted it because SPF/DKIM was configured correctly -- and -- the underlying ip address of the server it came from had a good reputation.

If SPF/DKIM was truly about "reputation", it would mean I'd have to wait days or months for reputation history to build up before Gmail accepted it.

I don't think this is correct? SPF and DKIM are about ensuring that the server actually is who it says it is, not about its reputation. In other words, when you receive an email that claims to be from Gmail, SPF and DKIM help you ensure that's where the letter actually came from, not from a server just pretending to be one of Gmail's servers.

> Unfortunately, that means that attempts by those larger providers to combat spam using such reputation will naturally hurt smaller providers

Tin-foil hat time, but I've always thought there was nothing unintentional or "unfortunate" (from Google's perspective) about this.

> That implies that we have some work to do on the problem of identity. As it is, there is not even a way for a known email sender to securely introduce an unknown email sender.

There is: gpg/pgp signature, but many people find it complicated, primarily because they are reluctant to read the documentation. And it’s popular to criticize it, especially here on HN, in favor of various half-baked alternatives.

> We need to be able to treat anonymous email differently than email from people we actually know.

The simplest solution to that would be an "only show me emails from people in my address book" filter. That would mostly echo how we treat user trust on all other platforms. Genuinely surprised this doesn't exist in most email clients (or does it and I have just overlooked it so far?)

Of course that's only a partial solution and wouldn't work for accounts where you expect unsolicited mails from people you don't know. I'd see it more as a "low-hanging fruit" solution. You could also expand the heuristic, e.g. also consider previous conversations, mailing lists, etc.

(Interestingly, the "introduce a friend" functionality would come for free: You can already send contact details as a VCard in an attachment. When receiving such a mail, some email clients will show a button to quickly add the contact to the address book.)

> The root problem is that we don't actually need to keep track of email server reputation.

We actually do. Is the server allowing anyone to sign up so that it's sending 99% spam, or does it have a lot of anti-spam measures so sign-ups can't be automated and it blocks accounts as soon as it detects them sending spam?

We really need a "Trust on First Use"(TOFU) system for messaging, that can be verifified, or pre-trusted offline, face to face. It'd be awfully nice for your bank to give you some thing that you can later verify that any communication from them (web site, online banking, text message, email, etc) are legit and verified.

Or if we can't trust users to handle TOFU, then some token/unique address/whatever that we can exchange face to face to enable trusted communication.

It's weird so see such a factually incorrect comment so high up on HN.

SPF/DKIM is literally how you establish sender identity instead of relying on the IP address of the email server so it is ironic to claim that they have anything to do with server reputation while lamenting a lack of sender reputation mechanisms.

Currently, SPF/DKIM are mostly used to prevent fraud, but they also provide the best tool we have to build sender based reputation systems.

It’s far easier to buy the email addresses of known good people by buying dumps of websites that got breached.

Web scraping gets you a lot of fake emails, company sinkholes, and other low reward stuff. Paying $20 for 100k confirmed real emails with names? That’s gold.

I'm using something very similar, except incoming messages from never-seen-before senders are greylisted instead:

https://en.wikipedia.org/wiki/Greylisting_(email)

95% of spammers never retry.

Someone posted on X advice that really helped me clean up my inbox

Add a filter looking for the word "Unsubscribe" and automatically put them in "Promotional" category or something similar. Also apply the filter to existing emails, and let it run for a minute.

Try it now! And comment if it reduced your inbox to like 2% of what it was :)

To be fair, SPF saves mail.ru and outlook.com users from five, maybe six spam emails per month coming from my domain, based on DMARC reports. If those numbers scale to include every domain on the internet, that's a huge amount of spam being filtered out very easily and very early.

You'd think spammers would've learned to avoid SPF domains at the very least but they haven't, so despite SPF/DMARC/DKIM failing to get anyone out the spam folder, the technology is still catching spam bots.

All of these technologies are basically DOA because of how fickle they are and for lack of support across the board. Most policies are set to not to deny.

DMARC is nice though. It won't stop spam. It won't stop spoofing. But you will know that someone somewhere is spamming people using your domain name. How awesome. :)

Finally, a comment that understands the concepts instead of insolently ranting about how useless it is.

While it may or may not reduce spam, it has definitely (based on my personal experience) reduced the amount of spoofed phishing emails and backscatter spam emails to nearly nothing.

In the early-to-mid '10s, before SPF/DKIM/DMARC became the law of the email land, one had to be much, much more careful with phishing emails, checking the wording, the logos, etc, because 9 out of 10 of them appeared to come from the actual domain the email purported to be from. In the past several years (I honestly don't know exactly when the change happened; I don't get a huge amount of phishing emails), it's shifted so that the first thing to check is the sender address. Usually that turns out to be some nonsense string @gmail.com or some long garbled domain.

IMHO, their main advantage is that third parties can’t send email which appears to originate from my domain.

I configure my domain to use SPF, so now spammers can’t sign it properly.

However, the fact that an email passes SPF verification only ensures that it was authorised by the domain owner. It doesn’t say anything about whether the domain owner is a spammer.

Cause IP is a finite resource (even IPv6 where the granularity is more like /48) while domains are infinite.

See https://en.wikipedia.org/wiki/Sybil_attack

domains are cheap and easy to get new ones. IPv4 addresses are limited so you can't burn them as freely.

It does work that way, but IP reputation is a thing as well so you need to keep that in mind. IPs need to be "seasoned" and "trusted" as well as domains.

This is how email-as-infra works, you're sending from a shared pool of their ips and they sign your emails with DKIM and you'll have SPF set up as well on your own.

It does work like that except nobody actually knows Google or Microsoft's algorithms to allow or deny mail delivery. It's the whole SEO thing all over again.

I think the problem isn't necessarily that adding DNS entries is hard (especially compared to the rest of the process of hosting your own email), but that getting a clear overview of what email tools an organization uses is difficult.

You need IT to list all of the reporting tools, customer service to tell you about their support system, marketing to tell you about their mailing list tool of the week, the sales guys to warn you they're using this new AI email enhancer, and somehow get that shady email forwarding service the CEO uses to give up their mail server IP addresses. Then you need to figure out how to get coverage for all of those tools and keep on top of them whenever something changes.

A lot of companies promise to do great things for you if you just enter the email address you'd like to send email from, and a lot of people gloss over the important details because those sound hard and when they tested the tool on their personal email it worked fine so that's probably unnecessary anyway! Managing email for a corporate domain can be like herding cats.

I think pretty much all email providers (and other systems that want to send on your behalf) have this. More or less the same process where they tell you what to add and then a "check my stuff" button to verify. Which is great.

I moved to Fastmail, and they have a nice guide to set up what's needed on DNS:

https://www.fastmail.help/hc/en-us/articles/360060591153-Man...