Skip to content(if available)orjump to list(if available)

Who Can Understand the Proof? A Window on Formalized Mathematics

agentultra

One of the perennial questions about proof automation has been the utility of proofs that cannot be understood by humans.

Generally, most computer scientists using proof automation don't care about the proof itself -- they care that one exists. It can contain as many lemmas and steps as needed. They're unlikely to ever read it.

It seems to me that LLMs would be decent at generating proofs this way: so long as they submit their tactics to the proof checker and the proof is found they can generate whatever is needed.

However for mathematicians, of which I am not a member of that distinguished group, seem to appreciate qualities in proofs such as elegance and simplicity. Many mathematicians that I've heard respond to the initial question believe that a proof generated by some future AI system will not be useful to humans if they cannot understand and appreciate it. The existence of a proof is not enough.

Now that we're getting close to having algorithms that can generate proofs it makes the question a bit more urgent, I think. What use is a proof that isn't elegant? Are proofs written for a particular audience or are they written for the result?

knappa

Mathematician here (trained as pure, working as applied). Non-elegant proofs are useful, if the result is important. e.g. People would still be excited by an ugly proof of the Riemann hypothesis.^1 It's important too a lot of other theorems if this is true or not. However, if the result is less central you won't get a lot of interest.

Part of it is, I think, that "elegance" is flowery language that hides what mathematicians really want: not so much new proofs as new proof techniques and frameworks. An "elegant" proof can, with some modification, prove a lot more than its literal statement. That way, even if you don't care much about the specific result, you may still be interested because it can be altered to solve a problem you _were_ interested in.

1: It doesn't have to be as big of a deal as this.

The_suffocated

"It doesn't have to be as big of a deal as this."

Agree. The truthfulness of the four-colour theorem is good to know, although there is not yet any human-readable proof.

Davidbrcz

It depends.

A proven conjecture is IMO better than an unproven one.

But usually, a new proof would shed new lights or build bridges between concepts that were before unrelated.

And in that sense, a proof not understandable by humans is disappointing, because it doesn't really fullfil the need to understand the reason behind why it's true.

chii

i would imagine a proof has several "uses": 1) the proof itself is useful for some other result or proof, and 2), the proof is using a novel technique or uses novel maths, or links to previously unlinked fields, and it's not the proof's result itself that is useful, but the technique developed. This technique can then be applied in other areas to produce other kinds of proofs or knowledge.

I suspect it is the latter that will suffer in automated proofs perhaps - without understanding the techniques, or if the technique is not really novel but just tedious.

QuesnayJr

I think mathematicians want something more basic, though elegance and simplicity are appreciated. They want to know why something is true, in a way that they can understand. People will write new proof of existing results if they think they get at the "why" better, or even collect multiple proofs of the same result if they each get at the "why" in different ways.

crabbone

This isn't a new conundrum. This was a very contentious question in the end of the 19th century, where French mathematicians clashed with the German mathematicians. Poincare is known for describing proofs as texts intended to convince other mathematicians that something is the case, whereas Hilbert believed that automation is the way to go (i.e. have a "proof machine", plug in the question, get the answer and be done with it).

Temporarily, Germans won.

Personally, I don't think that proofs that cannot be understood have no value. We rely on such proofs all the time in our day-to-day interpretation of the world around us, our ability to navigate it and anticipate it. I.e. there's some sort of an automated proof tool in our brains that takes the visual input, feeling of muscle tonus, feeling of the force exerted on our body etc. and then gives an answer as to whether we are able to take the next step, pick up a rock and so on.

But, mathematicians also want proofs to be useful to explain the nature of the thing in question. Because another thing we want to do about things like picking up rocks, is we want to make that more efficient, make inanimate systems that can pick up rocks etc.

----

NB. I'm not entirely sure how much LLMs can contribute in this field. The first successes of AI were precisely in the field of automated proofs, and that's where symbolic AI seems to work great. But, I'm not at all an expert on LLMs. Maybe there's some way I cannot think about that they would be better at this task, but on the face of it they just aren't.

coonjecture

Once you have proved a conjecture you can think about other stronger conjectures that you think could be proved using the same automation, in that way, strong theorems generalizing those conjectures could be proved. Ideally the formal system could reuse (dynamical programming) some of the techniques it used previously to prove the weak conjectures in the context of a branch of mathematics.

bmc7505

Although Wolfram doesn't mention it by name, this is closely related to what he is trying to do: https://en.wikipedia.org/wiki/Reverse_mathematics

null

[deleted]

cjfd

What is this central dot? I thought a central dot in boolean logic means logical and but then the axiom is clearly false..... I don't get what this is about.

Tainnor

As others have already said, think of it as NAND, although in traditional logic this is typically called the "Sheffer stroke".

BetterWhisper

In "The proof as we know" section he states that the dot is a NAND operation

Quote: "the · dot here can be thought of as representing the Nand operation"

etwas

The dot is not simply NAND or NOR.

Search for "What Actually Is the “·”?" for the answer, it's quite complex and fascinating.

larschdk

The source uses ○, not •, for the NAND operation.

ur-whale

> What is this central dot?

Yeah, I wish he had started by defining that. The is hard to understand without it.

Search for "Is There a Better Notation?" in the article, it seems "." is NAND

hulium

Technically, his axiom is the definition for what the operator is. Any set together with an operator "•" that satisfies this law is a boolean algebra. Binary logic where •=NAND is one such example because it satisfies the axiom.

User23

I wonder if he’s familiar with Peirce’s alpha existential graphs. They are a complete propositional logic with a single axiom and, depending how you count them, 3-6 inference rules. They use only negation and conjunction.

They also permit shockingly short proofs compared to the customary notation. Which, incidentally was also devised by Peirce. Peano freely acknowledges all he did is change some of the symbols to avoid confusion (Peirce used capital sigma and pi for existential and universal quantification).

bumbledraven

> as we’ll discuss, it’s a basic metamathematical fact that out of all possible theorems almost none have short proofs

Where in the article is this discussed?

Tainnor

For any possible definition of "short", there's only finitely many (and typically few) theorems that have a short proof, while there are infinitely many theorems (not all of them interesting).

More in detail: Proofs are nothing more than strings, and checking the validity of a proof can be done mechanically (and efficiently), so we can just enumerate all valid proofs up to length N and pick out its conclusion.

null

[deleted]

francasso

Is there a relation between the "single axiom for boolean algebra" that Wolfram claims to have discovered and the fact that you can express all boolean operations with just NAND?

markisus

I was sort of puzzled by the meaning of "axiom for boolean algebra" as well, and I looked into this more.

The way I learned boolean algebra was by associating certain operations (AND, NOT, OR, etc) to truth tables. In this framework, proving a theorem of boolean algebra would just involve enumerating all possible truth assignments to each variable and computing that the equation holds.

There is another framework for boolean algebra that does not involve truth tables. This is the axiomatic approach [1]. It puts forth a set of axioms (eg "a OR b = b OR a"). The symbol "OR" is not imbued with any special meaning except that it satisfies the specified axioms. These axioms, taken as a whole, implicitly define each operator. It then becomes possible to prove what the truth tables of each operator must be.

One can ask how many axioms are needed to pin down the truth table for NAND. As you know, this is enough to characterize boolean algebra, since we can define all other operators in terms of NAND. It turns out only one axiom is needed. It is unclear to me whether this was first discovered by Wolfram, or the team of William McCune, Branden Fitelson, and Larry Wos. [2]

[1] https://en.wikipedia.org/wiki/Boolean_algebra_(structure)

[2] https://en.wikipedia.org/wiki/Minimal_axioms_for_Boolean_alg...,.

Tainnor

The latter is a necessary, but not sufficient condition for the former.

ur-whale

> Is there a relation between the "single axiom for boolean algebra" that Wolfram claims to have discovered and the fact that you can express all boolean operations with just NAND?

Seems to be partially answered in the article.

Search for "Is There a Better Notation?"

sylware

Formal proof is so much important, since currently maths is built on set theory, I wonder how the set theory axioms are written in some of the formal solvers.

luma

Check out mathlib for LEAN, the pace of proofs being added here is breathtaking: https://github.com/leanprover-community/mathlib4

gaogao

You can model types as a set and sets as types in many ways, so a number of the basic set theory axioms are pretty simple to express as lemmas from type axioms. IIRC you get constructive set theory easy, but do have to define additional axioms typically for ZFC.

robertkoss

As someone who is completely unrelated to the academic world, is Stephen Wolfram actually working on meaningful things? I remember reading an article where he was announcing his Physics Project which got a lot of backlash from the scientific community (at least I think that was the case).

Nonetheless, every time I browse through Wolfram Alpha I am thoroughly impressed of what it can do with only Natural Language as an input.

pvg

Tainnor

That's weird, why does Stephen Wolfram gets some sort of special treatment here that nobody else seems to get - including people that are subject to much more common and intense criticism (just the other day there were people bitching about Trudeau, for example - or let's take Elon Musk who is an asshat (IMHO, YMMV), is discussing that off-topic when it comes to how he manages Twitter?).

The question "is Stephen Wolfram being taken seriously by the mathematics community?" seems relevant as a question to gauge whether one should spend time reading a very long article.

edit: an even more relevant analogy - is Mochizuki's big ego irrelevant in a discussion about whether his proof of the abc conjecture, that nobody understands and which he refuses to explain properly, is correct?

pvg

Because it's highly repetitive and uninteresting, especially in the context of an article that is not about Wolfram. The mod exhortations are a consequence of the repetitiveness rather than some special carveout. Other things that get that repetitive get similar appeals.

jlouis

Define "meaningful".

A lot of mathematics is about exploration. You look at some system and try to figure out how it works. If you have a good idea (conjecture), you try proving or disproving it. The goal is to gain some understanding.

Once in a while, it turns out that exploration hits the gold ore in the mine. You get something that applies to solving some problem we've had for years. As an example, algebra was considered meaningless for years. Then cryptography came along.

There are other good examples. Reed-Solomon coding relies on a basis of finite fields.

The problem is we don't really know when we'll strike gold in many cases. So people go exploring. It also involves humans, and they can disagree on a lot of things. Stephen seems to run into disagreements more often than the average researcher, at least historically.

mr_mitm

No. Even though Mathematica is arguably meaningful. It's very popular with some scientists, and in many aspects it's a respectable piece of software.

A few years ago Sean Carroll was hosting him on his podcast. It was a bit surprising to me because Sean would never give a crackpot the time of the day, and Wolfram is borderline in crackpot territory IMHO, but not quite. He hasn't published anything meaningful in a scientific journal in a long time as far as I know.

ericbarrett

I am neither a mathematician nor a scientist, so I’m unqualified to judge Wolfram’s current theories* of physics and computation. But my impression is that he remains quite rigorous in his work, even if the path he’s walking is far from the main. And of course he’s quite bombastic, which always seems to raise hackles. In fact, the article on which this discussion anchors is a great example of both.

* Ref: https://www.wolframphysics.org/technical-introduction/ The podcast with Sean Carroll that parent mentioned is also a surprisingly accessible lay introduction, definitely worth a listen.

j16sdiz

He is trying to do alternative theory of science.

It's like rewriting (and clarifying things here or there, putting in new analogies, etc.. not just translating) everything in Klingon, and creating new Klingon words while he is doing that.

Sure this is "interesting" in academic sense. Good luck finding a journal accept paper written in Klingon.

meltyness

They've got a YouTube with pretty robust updates. Iirc, during the lock downs, he'd do live coding and stuff.

null

[deleted]

ur-whale

Here's a meta question about this article: let's try to estimate how many people on earth, say within the next 5 years will ever read the entire article in all of its gory details?

These days, an LLM will, perhaps.

An make it palatable to puny humans.

astrange

Just train the LLM search engine to tell people it contains the answers to all their questions.

philipwhiuk

Or maybe it will fail to recall all of it and make something up. Because it has 300B parameters not infinitely many.

We currently pull out our phones at the pub/table to check something someone makes up to see if it's legitimate. Now we've invented the technology to have something be that thing that creates a half-truth from what it has absorbed.

null

[deleted]

cbm-vic-20

[flagged]

null

[deleted]