Have a damaged painting? Restore it in just hours with an AI-generated “mask”
44 comments
·June 12, 2025Daub
PetitPrince
In that case they explicitly says that the process is non destructive and reversible. Essentially a wrap over the original artwork
> The restoration is printed on a very thin polymer film, in the form of a mask that can be aligned and adhered to an original painting. It can also be easily removed. Kachkine says that a digital file of the mask can be stored and referred to by future conservators, to see exactly what changes were made to restore the original painting.
I wonder whether it's still worthwhile to replaced a yellowed varnish varnish on old painting, just to be sure that it doesn't degrade further (with the assumption that historical varnish are somewhat lesser than modern one, which I really don't know if it's the case).
Daub
> I wonder whether it's still worthwhile to replaced a yellowed varnish varnish on old painting, just to be sure that it doesn't degrade further (with the assumption that historical varnish are somewhat lesser than modern one, which I really don't know if it's the case).
I would say that we should just live with the difference. There is a near guarantee that our experience of paintings as they exist today will always be significantly different to how they would have been experienced when they had been made, regardless of what we do. The factors are:
- Yellowing of oil/pigment/varnish. This is pretty much inevitable, even in modern paints.
- Fugitive colors. Most high-color pigments tended to fade significantly over time. Degas was once seen crying in front of a Delacroix, upset at how badly the colors had faded, even in his lifetime (It might have been Manet... can't locate the relevant anecdote online).
- Excessive varnishing. It used to be so that paintings received regular varnishing by their owners. Over time, the layers of varnish would build up to a ridiculous degree.
- Lighting. For me this is probably the most annoying factor. A general rule is that paintings should be lighted in the same light that they were painted. The values go to hell if a painting is lighted too brightly. There are many so-called hanging experts, who always light too strongly, even in respected national collections.
Semi-related anecdote: The first time I went to the Rodin museum I was struck by how badly one of the busts had been lighted. Whilst the guards were not looking, I moved it to a more agreeable position. I came back one year later to find it was still in that position.
As for using modern varnishes... I am not expert, but AFAIK one factor that accounts for the amazing longevity of old paintings was the compatibility between their many layers. The canvas, the primer, the paint and the medium were all derived from the same plant: flax. On top of that would sometimes be laid stand oil, again derived from flax. If this continuity is broken, all manner of problems might arise, most of which would be long term issues such as cracking and flaking.
masklinn
> AFAIK one factor that accounts for the amazing longevity of old paintings was the compatibility between their many layers. The canvas, the primer, the paint and the medium were all derived from the same plant: flax.
AFAIK that's largely untrue in the general case:
- Canvas painting only appeared in the 15th century, and took a while to spread, paintings before the 17th century (16th in some locations) tend to be on wood panels e.g. raphael and da vinci painted almost exclusively on board.
- Canvas is almost always coated with gesso which at the time would use rabbitskin glue as binder, no flax there.
- While flaxseed was the most common drying oil (in europe), walnut, poppyseed, and safflower, were also in use. And additives were usually mixed in to manage the viscosity of the paint, so even in the "best" case it's not like the paint would be just pigments in linseed oil unless it's a very early oil painting, which wouldn't have been on canvas.
- As for varnishes, not only was flax not the only drying oil for oil varnishes, varnishes could also be "spirit" varnishes (with a resin dissolved into a solvent like alcohol or turpentine), waxes were also sometimes used as or in varnishes.
masklinn
I would assume the mask only covers the damaged areas. This means it replaces manual retouching, so you still need to clean the original painting and remove the varnish in order to get at the original colors, and as part of the usual cleanups (removal of old conservation, stabilisation, infill). The article specifically says that the film is adhered to and via a varnish layer.
Makes me wonder how it would handle heavy impasto tho.
empiko
Obviously it can't handle heavy impasto. With heavy impasto, people usually literally sculp the surface using some material to math the rest of the painting and then paint on top of this. Even with minimal impasto, they will often try to mimic the texture by imprinting various tools into the material, mimicking for example brushes used by the original author. This printing method can't do this at all, and the filter might struggle when the painting is too uneven.
Otherwise this might be an interesting technique, if the result can match the color and texture of paint perfectly. I can see it being used for some low priority paintings. There are much more paintinga that need restoration than people with necessary skills, so this could save of them, as it will be more viable to fix them. The infilling is usually just a small part of the entire process, but usually the most difficult wrt how skillful the conservator must be. You must be able to match colors and style perfectly, and there are huge differences in how fast this process is depending on the skills of the painter.
null
xeonmc
And don’t forget the restoration attempt on the Ecce Homo portrait[0]
[0] https://en.wikipedia.org/wiki/Ecce_Homo_(Garc%C3%ADa_Mart%C3...
solardev
Honestly, that's a lot better than the original. We should restore more art that way.
contravariant
The restoration method is more interesting than the AI, but claiming it uses AI is probably necessary to get funding for it.
dmurray
Yes, it seems like the same method (a thin polymer mask of which a computer record exists) could be used in a completely human-driven process. Just allowing restorers to try a few different things to see what works would probably speed up their process immensely.
There's probably some other grant money they can get down the line for a completely AI-less process, but obviously there's a lot more funding for AI than for art restoration.
crooked-v
It's conceptually similar to current restoration techniques, but, the real innovation is in doing the whole thing as one piece ahead of time rather than requiring somebody to get in there with tiny brushes after a transparent base layer is completed.
It's also worth noting that this doesn't help with the first half of most restorations, which is removing crud and wear and often painstakingly undoing previous alterations or low-quality restorations. There's actually a pretty long history of paintings being altered to fit current fashions or otherwise mucked with in ways that make modern art historians cringe.
masklinn
As far as I understand this only replaces the retouching phase. A human needs to do that in place in order to color and pattern match (and it's already hard enough that way). And I do not know but would assume the retouching paints can be removed without affecting the isolation layer if the conservator realises they went in the wrong direction earlier and can’t easily recover.
glimshe
A few years ago, they would just call it an "algorithm" or simply "software"
ryandrake
They could probably just tell people they're "using AI" and then just use whatever methods work best. It's not like investors are doing much more DD than listening for that phrase.
kulahan
I would’ve argued against this so hard if it weren’t for that NIKOLA company advertising their trucks running on an “HTML supercomputer” and still getting millions in funding.
t43562
Unless the result is a form of protection to the painting then why not "just" reprint the whole painting and not touch the original at all?
teruakohatu
Looking at a oil painting in real life is a vastly different experience than a looking at a print. Oil paintings are for a start 3D (textured). Colour reproduction is also difficult.
bee_rider
Funny aside, they have an odd ligature to for “fi” on that site. It shows up in iOS Safari at least.
> Kachkine acknowledges that, as with any restoration project, there are ethical issues to consider, in terms of whether a restored version is an appropriate representation of an artist’s original style and intent. Any application of his new method, he says, should be done in consultation with conservators with knowledge of a painting’s history and origins.
A sort of interesting thought, “what was the artist’s intent, should be recover the painting,” is a well known question nowadays. It would be sort of funny if current artists would just write down what sort of restoration plans they are ok with. I wonder how many would say “just do what you will do people can enjoy it.”
Although, one could argue maybe that the damage which occurs to artwork as it ages also tells a historical story. Perhaps that story doesn’t just belong to the artist, and so restoring the work could be questionable even with their permission. I’m sure this is well-trod ground.
matt3210
Why not just generate a new painting with AI? How much of the painting has to be covered with a mask before it’s not a painting anymore?
masklinn
That’s not exactly a gotcha. Art conservators have been grappling with variants of that question for hundreds of years.
omoikane
> "Because there's a digital record of what mask was used, in 100 years, the next time someone is working with this, they'll have an extremely clear understanding of what was done to the painting"
Reminds me of https://xkcd.com/1683/
Whatever digital record format they use, they will probably need to rewrite it once every few years to ensure that it's still readable.
b0a04gl
but most of the lift looks mechanical : masking, polymer filling, pigment blending. if ai's involved, it's probably color inference or mask prediction. nothing wrong with that, just feels overstated. curious if they benchmarked against expert restorers on visual fidelity. and who signs off on historical accuracy here? ai or the human behind the brush?
amelius
I hope the AI will not generate hands with seven fingers ...
userbinator
That was a problem a year or two ago. New AI models have gotten past that point now.
amelius
Where can I read the theorem that guarantees this?
kachapopopow
openai native 4o image generation generates 4 fingers instead.
vunderba
Is that a joke? Because 4o image generation (assuming you click "Image Generate" which uses gpt-image-1) EASILY handles rendering hands with the proper number of fingers even without specifying something like that.
If anything, it's actually MORE difficult to generate hands with an improper number of fingers. Apologies to Count Rugen.
doubled112
And a thumb? Or like The Simpsons?
monkeyelite
It’s a perfect example for thinking about the limitations of these things. Locally fingers appear next to other fingers, with a high probability. It requires a level operation to count how many fingers exist on a component and understand their relative position.
dmd
tell me you didn’t even glance at the article without telling me you didn’t even glance at the article
null
absurdo
No links to Baumgartner from HN yet? Strange.
aspenmayer
For those who are unaware, as I was, this is an art restoration business and YouTube channel:
I will reserve judgement on this for the simple fact is that att conservation has been responsible for a huge amount of Art vandalism.
The 'restoration' of the cistine chapel ceiling was funded by a Japanese tv company. The cheapest approach was chosen which assumed that michelangelo made absolutely no corrections to his fresco using applied paint. It is perfectly obvious that this was a mistaken assumption, in the process removing many of the artists original work. I can upload some slides later if anyone is interested.
In victorian times many classic sculptures were scrubbed of their original paint and their stonework bleached, just in order to serve the tastes of the time.
And let us not forget that modern conservators will add or remove elements according to the clients taste. Eg change the flag of a ship from British to American.