Skip to content(if available)orjump to list(if available)

DigiCert: Threat of legal action to stifle Bugzilla discourse

jchw

Web PKI drama is always astonishing to me because it is one of the only areas of the entire world where a corporation's "fucking around" is seldom ever not followed by a sobering "finding out" period. The various entities that decide what CAs to trust can effectively dismantle any CA business in the world, basically at the drop of a hat. If DigiCert decides to play this game and lose, it would make them the biggest such loser so far. DigiCert is, as far as I know, the largest CA on the Internet. It would certainly send a strong message, and cause a lot of chaos, if the biggest CA on the Internet found itself being removed from trust stores. Yet, there's no particular reason it couldn't happen. How exciting.

Of course, I think that is unlikely, but on the other hand, it's just as cathartic to imagine whatever idiot at DigiCert thought it was a good idea to engage legal here to have the dressing down of a lifetime. I read the thread in question. It doesn't make DigiCert look good, but this action definitely is more damning to them than anything Collan said in my opinion.

chicom_malware

> It would certainly send a strong message, and cause a lot of chaos, if the biggest CA on the Internet found itself being removed from trust stores

Their customers would be forced to give their business to Honest Achmed[1].

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=647959

skissane

Someone should start a real CA business, complete with all the proper audits and everything to get it into the browser trust stores… and call it “Honest Achmed’s Used Cars and Certificates” (have it buy some random used car dealership in the middle of nowhere so the name is not a lie)

Where’s a billionaire troll when you need one? (And a form of billionaire trolling that would upset a lot less people than Musk’s version of it.)

Would be even funnier if the billionaire’s name actually was Achmed

justahuman74

What has prevented this (a good legit CA co) from happening before now?

justahuman74

I'm actually curious why that wasn't approved

woodruffw

To my understanding, the point of Honest Achmed was to demonstrate that it was possible to write a facially reasonable and compliant CA inclusion request that was also intuitively unacceptable. It successfully demonstrated that the inclusion policies at the time needed to be made more rigorous, which they were.

twisteriffic

It's refreshing to see folks who're obviously used to hiding behind clouds of bullshit get skewered by people who both know enough to see through it and have the time and energy to follow every thread to completion.

The most recent digicert thread smells suspiciously similar to those that lead up to the Entrust debacle.

userbinator

It would certainly send a strong message, and cause a lot of chaos, if the biggest CA on the Internet found itself being removed from trust stores.

How many will agree to that removal? How many will see one more reason to forever turn off automatic updates and decide what to trust themselves, having seen yet another way some faceless entity they never knew about can break things?

It will certainly send a strong message, but likely not the intended one. All it will do is increase the lack of trust in centralised PKI in general.

jchw

I'm not sure how to put this nicely, but I'll try my best: The normal people, who account for 99% of the end users of DigiCert's customers, are not going to disable updates on their web browser or their OS to "take a stand" against this decision. (And corporate users can't do this anyways, since it's not in their hands, and keeping things out of date is not a reasonable option for an organization that has security standards.) Most of them won't know what is going on here, and if they hear about it, probably won't care or know what to do about it. That's the Web PKI infrastructure working as intended, because it would be infeasible for billions of people to properly understand the gravity of all of these decisions. In order to ensure that TLS connections are at least minimally safe, it's pretty much necessary for things to work this way.

I'm not arguing that it's good that a relatively small number of entities (mainly Google, Microsoft and Mozilla) decide which CAs are trustworthy, but that's all the more reason that it's important for all of this Web PKI work to happen completely in the open, so that the few who can spare the time and effort to scrutinize what is going on can help the rest of us have a safer Internet. We don't have a better solution. That's also why DigiCert issuing legal threats because they don't like how one of these issue reports makes them look is a serious problem that can't be tolerated.

LegionMammal978

> The normal people, who account for 99% of the end users of DigiCert's customers, are not going to disable updates on their web browser or their OS to "take a stand" against this decision.

I'd imagine that they wouldn't do it to "take a stand" so much as "avoid the risk of getting their stuff broken in the short term" in this scenario, regardless of whichever party loses the blame game. See: the recent WordPress drama, which has turned customers away from both parties involved.

userbinator

The normal people, who account for 99% of the end users of DigiCert's customers, are not going to disable updates on their web browser or their OS to "take a stand" against this decision.

...unless they notice the breakage, and you tell them to do so to stop it.

cortesoft

> mainly Google, Microsoft and Mozilla

They don't have free rein to do whatever they want. If they chose to remove them, they are going to have to be ready to defend themselves in court.

woodruffw

> All it will do is increase the lack of trust in centralised PKI in general.

“In general” is a broad statement, given that the overwhelming majority of people using the Web PKI have no idea that they’re doing so. DigiCert is not a legible part of the value chain for the average users; they won’t notice that the sites they use switch CAs. This strong disfavoritism towards vendors is arguably one of the Web PKI’s greatest strengths.

edelbitter

Many systems do not fetch updates from the Mozilla root store, but from their (possibly Debian-derived) stable distribution of it. Meaning two highly respected entities, known for being well aware of the wider impact of their careful enforcement of strict policies, need to agree to cause any major breakage. When that happens, I can blindly trust they did the thing needed to keep the unaffected parts of that weird system working as intended.. and then still head to bugzilla and read about the background - to laugh at whoever triggered the mess.

pilif

If DigiCert were to lose browser trust (at this point, that's still a big if), it would happen the same way it happened with prior CAs, some of which were pretty big themselves (Symantec): all certificates issued after some date would not be trusted, yes. But all existing certificates would remain valid.

This gives certificate owners ample time to look for a different issuer and no certificate buyer would deliberately purchase a certificate from an issuer when they know some percentage of users will not trust that cert.

So for the end users, everything will keep working: the existing digicert certs stay valid and newly refreshed certs will be signed by a different authority. There is no need to turn off automatic updates over this.

Between Entrust and Symantec, we've already seen this happen to large well-known CAs and everything remained fine (not for the offenders, but, hey, that's the system working as intended)

nneonneo

In a nutshell:

DigiCert has delayed revocation beyond what's allowed in the Baseline Requirements a few times; most recently, https://bugzilla.mozilla.org/show_bug.cgi?id=1896053 and https://bugzilla.mozilla.org/show_bug.cgi?id=1910805. In the former case, it seems DigiCert chose to delay revocation to appease certain clients; in the latter case they were prohibited by a Temporary Restraining Order (TRO) from performing timely revocation.

Tim Callan from Sectigo has publicly lambasted DigiCert for these delays, since in both cases it seems DigiCert hasn't pushed back hard enough on its clients. In the latter case, there's concern that measures like TROs might be employed more often to stall revocation. Sectigo (and others in the WebPKI ecosystem) seem to want DigiCert to make the revocation policies very clear to clients and to ensure that clients can actually replace their certificates in a timely manner.

Sectigo is clearly the most vocal but they don't seem to be the only ones telling DigiCert to get their delayed revocation under control. So the escalation to legal threats is really uncalled for, and DigiCert could face some very significant pushback for trying this tactic.

RHSeeger

What is the correct action when a TRO says a company cannot revoke the cert? Is it that the company will delay revocation, but will push the judicial system to resolve the issue as fast as possible?

kevin_thibedeau

How can a court inhibit revocation when every CA declares their right to do so when you purchase a certificate?

nneonneo

DigiCert probably should have revoked every cert they could within 24 hours. Instead they just pushed the revocation of all 80,000+ certs out to five days.

It's quite likely that many of their other clients pushed back on the 24-hour timeline (similar to what happened in their previous incident); I believe the delayed revocation issue (https://bugzilla.mozilla.org/show_bug.cgi?id=1910322) hints at this. The TRO gave them a convenient excuse to delay all revocations without having to explain all over again why they made exemptions for their special clients.

Heck, their status page (https://status.digicert.com/incidents/3sccz3v31lc9) even gives instructions for how to request a delayed revocation - even though the initial incident page (https://www.digicert.com/support/certificate-revocation-inci...) says clearly:

"Any issue with domain validation is considered a serious issue by CABF and requires immediate action. Failure to comply can result in a distrust of the Certificate Authority. As such, we must revoke all impacted certificates within 24 hours of discovery. No extensions or delays are permitted. We apologize if this causes a business disruption to you and are standing by to assist you with validating your domain and issuing replacement certificates immediately."

Dylan16807

I think the eventual correct option is pretty clear.

A TRO like that is based on the company loudly declaring that revoking will cause real damage. That means their use of certificates is incompatible with the web PKI rules and ecosystem. That means they need to be migrated out ASAP, with every certificate authority refusing to take their business.

Make that series of consequences clear, and companies will stop trying that trick.

xmodem

Looking at the original report ( https://bugzilla.mozilla.org/show_bug.cgi?id=1910322 ) I can see a couple of questions that DigiCert appears to be avoiding:

> The public record of Alegeus Technologies LLC v. DigiCert shows no attempt by DigiCert to contest the court’s order prior to the end of its preferred period of nearly 120 hours, even though such a motion could have freed DigiCert to revoke the certificates days earlier.

and

> The other question in comment 28 was for the language establishing DigiCert’s right to revoke Alegeus Technologies certificates. DigiCert has waffled on this point, first implying that this language was to be found on its website but later refusing to confirm that the language on the site applied to Alegeus Technologies at the time.

SPECULATION: Digicert may have offered special terms to Alegeus, and possibly other customers. They may have chosen not to dispute the TRO in court because they did not have grounds to do so under those agreements. They may also have included confidentiality terms in those contracts that prevented them from speaking about it.

OPINION: I am surprised that the forum allowed the issue to be closed without the above quoted questions being satisfied, though it is possible they are addressed elsewhere, I have not done a complete reading of all the linked issues.

EDIT to add: DigiCert has a response in a different thread here: https://bugzilla.mozilla.org/show_bug.cgi?id=1910805#c43 that would appear to contradict my speculation. Specifically

> Even though DigiCert’s TOU and MSA prohibited Alegeus from taking the action it did, once it filed for a TRO and the court almost immediately granted it, DigiCert’s hands were tied

infogulch

This is shocking to read. Even attempting to choke the legal speech of web PKI contributors with legalistic bullying is a gross inversion of the purpose and goals of the organization, and IMO warrants revoking everything to do with DigiCert on the spot.

webprofusion

Always two sides to a story but the guy who caused the validation bug at DigiCert already resigned because of it (which is extreme), the Sectigo guy wanted to prevent the bug being closed so he could keep pushing them (in a subjectively prickly fashion) for more answers about their general responsiveness.

A bit of back and forth discourse is fine and expected, but if you keep pushing someone who has their own legal dept they're eventually going to wander over to the coffee machine and have a chat about it with them, then they're going to take a look and it becomes their problem.

So the number one rule would be don't even breath the word "legal" unless you want to invoke them. This particular response is just a letter telling them to back off and it's why you have a legal dept, so they can argue with each other. This one has just found it's way into the open.

There is a understandable perspective that says CAs shouldn't be burdened with legal risk in their discussions, but that's contrary to the fact these guys are commercial entities protecting their interests, so you don't get it both ways unless all your CAs are non-commercial, and even then that would only extend so far.

willvarfar

You've given a detailed rundown like you've been following, so what is your sense about the resignation? Did the guy resign willingly, or is Digicert the kind of management that likely scapegoated him?

tbrownaw

So what's happened in the ... two months and a bit since the dates mentioned for the letters that this is apparently in reference to?

null

[deleted]

Arnavion

Even taking the (one-sided) depiction of the conversations in DigiCert's letter at face value, maybe Sectigo's guy was being a git at best and intentionally trolling at worst. (I don't think he was, but let's play devil's advocate.) But even then, how did DigiCert think getting legal involved would possibly go well? Sectigo stands to gain publicity and lose nothing by going public with it to the CAB as they did here, and it's not like the CAB is going to play marriage counselor and get the two companies to make up because one of them got their feefees hurt.

Besides, this kind of hyper-polite passive-aggressive "erm akchually" conversation happens in every CAB incident discussion. I don't know why DigiCert got particularly upset about this one.

akerl_

> Besides, this kind of hyper-polite passive-aggressive "erm akchually" conversation happens in every CAB incident discussion.

As somebody who doesn't spend much time scrolling CAB reports, this was jarring to me.

Digicert's legal action seems nuts, and there seems like a real, risky issue in the idea that a company's customers can use the legal system to block the company from complying with its obligations to other entities, but it's hard to see any way that could be productively addressed given the back and forth in the thread. It's like I'm watching a theatrical production staring the most stereotypical corporate drones trading comments with the most stereotypical IRC nerds, both sides doing circles around an interesting topic but too busy trading blows to ever really get to it.

SpicyLemonZest

> there seems like a real, risky issue in the idea that a company's customers can use the legal system to block the company from complying with its obligations to other entities

As Digicert has repeatedly explained, this is simply how the United States legal system works. Courts have broad and indisputable power to issue temporary restraining orders, and the parties to a case must comply even if doing so violates some promise they made to a third party. (The point of the TRO is to maintain the status quo while the court figures out details like what promises have been made to who.) People in the PKI community who believe that some carefully written policy would enable CAs to reject an invalidation TRO, or convince a court that they cannot issue it, are wrong.

The reason it's never come up before is that no CA had previously attempted to enforce a widespread 24 hour revocation caused by its own error.

cjbprime

(I think Sectigo's argument is that Digicert did not even attempt to convince the court that it should be allowed to revoke those certificates in the mandated timeframe. If they had attempted and failed, I don't think they would be receiving criticism.)

nightfly

Can someone link to (or post copies of) the offending questions/statements?

infogulch

In the second paragraph:

> Contrary to this statement, I received a letter from DigiCert’s lawyers, Wilson Sonsini, regarding posts made by Sectigo’s Chief Compliance Officer in bug 1910322. https://bugzilla.mozilla.org/show_bug.cgi?id=1910322

mdaniel

> The code worked in our original monolithic system but was not implemented properly when we moved to our micro-services systems

:chefs_kiss:

evil-olive

it gets even better:

> We also found that the bug in the code was inadvertently remediated when engineering completed a user-experience enhancement project that collapsed multiple random value generation microservices into a single service.

crooked-v

As we all know, software isn't software unless it can immediately scale to having 12 billion users.

naitgacem

Can someone point to where i can read some context?

nneonneo

DigiCert's legal threat, while obviously biased towards DigiCert, gives some context: https://bug1950144.bmoattachments.org/attachment.cgi?id=9468...

For further reading, consider these two incidents which resulted in delayed revocation from DigiCert and a bunch of comments about how DigiCert should not be allowing delayed revocation:

- Incident report https://bugzilla.mozilla.org/show_bug.cgi?id=1894560, delayed revocation report https://bugzilla.mozilla.org/show_bug.cgi?id=1896053 - incident due to the issuance of some certificates with incorrectly-capitalized phrases in the certificate's Business Category field; baseline requirements require revocation within five days but DigiCert dragged that out much further

- Incident report https://bugzilla.mozilla.org/show_bug.cgi?id=1910322, delayed revocation report https://bugzilla.mozilla.org/show_bug.cgi?id=1910805, DigiCert information page https://www.digicert.com/support/certificate-revocation-inci... - incident due to incorrect CNAME-based domain validation (failure to check that the CNAME started with an underscore); baseline requirements require revocation within 24 hours but DigiCert was stopped by the TRO and revoked after five days.

Essentially, DigiCert has been delaying the revocation process (twice now) and people are unhappy about that. DigiCert has apparently attempted to silence those unhappy people (Sectigo and their representative Tim Callan) with legal action.

skylerwiernik

I don't believe that that's a legal filing. DigiCert never filed anything with the courts. That's just a letter to Sectigo threatening to sue.

nneonneo

Whoops, sorry, should've said legal threat not filing - fixed.