Show HN: HTML visualization of a PDF file's internal structure

Parsing data out of arbitrary PDFs is a cursed mission. PDF can contain images, so you might as well target JPEG directly.

OCR can take you pretty far depending on expectations, but it's never quite far enough in my experience.

The LLMs might help with sequencing the characters you extract from the page but actually getting the contents is still difficult. A number of times I've come across a page where the letters of the text are glyphs in a custom font with no mapping to ascii or anything similar or even more common, especially with output from CAD, are letters that are made by drawing lines in the shape of letters so there is nothing identifiable to extract and you are left with OCRing the page to double check the results

In my previous role, I have experienced similar things where the rule-based parsing approach is really tricky to get right and often failed via from edge cases.

We (at https://runtrellis.com/) have been building PDF processing pipeline from the ground up with LLMs and VLMs and have seen close to 100% accuracy even for tricky PDFs. The key is to use rule based engine and references to cross check the data.

Many moons ago worked on extracting 2D CAD drawings from PDFs and converting to full 3D. Fun times.

pdfjs does all of that and it’s pretty solid. I used it recently to extract tabular data out of 10 year batch of bank statements.

It reminds me of: https://xkcd.com/1425/

In the same way now with today's AI models the task is easily achievable.

mathpix does quite an awesome job actually

https://kaitai.io/ maybe?

It looks perfectly nice for its role, but I didn’t use it for my last project because I need serialization as well.

HexFiend also has a template syntax for binary data visualization. It’s based on Tcl.

https://github.com/HexFiend/HexFiend/blob/master/templates/T...

Be careful, "any" is a strong word in this context. Interestingly enough, I actually use PDF as the "hello world" for kicking the tires on any such file format descriptor I find because PDF is such a crazypants specification. Thus, if the descriptor language is able to accurately capture the layout of a PDF, it's obviously well thought out.

I haven't had a lot of luck thus far, except ones which allow escaping out of declarative mode over into "and then run this code"

That does sound interesting. Forgive my ignorance, but how could this be used to detect watermarks? Could the same method be used to detect signatures?

Thanks for noticing! You're right, I will fix that very soon.

Do you mean a browser extension? Not trying to be rude; Just making sure I understand.

Yes, I value simplicity and the interactivity offered by basic HTML and CSS is sufficient for my use case :)

I don't think what makes PDF an 'unfortunate' format for (1) editing, (2) on-device reading, and (3) extraction of semantic information (as opposed to presentational information) is any sin on Adobe's part nor 'bloat.'

It's a page description format, not a data format, so all its decisions follow from the need to ensure that you and I can both print the same 'page' even if we use different operating systems, software, printers, exact paper dimensions, etc. I suspect the main reason it holds on so well is that so many things operate in a document paradigm, where 'document' means 'collection of sheets of paper.' Everything from the After-Visit Summary from the doctor, to your car registration document already has a specific visual representation chosen to allow them to fit sensibly and precisely on sheets of paper.

Could HTML (say, with data URLs for its images and CSS so that it can stand on its own), or ePub be a better format in most ways? Sort of, but it is optimized for such a different goal that if you went in to evangelize that switch to everyone who makes PDFs today, you'd be met with frustration that the content will look a bit different on every device, and that depending on settings, even the page breaks would fall differently.

Relatedly, it's interesting to me that even Google Docs, which I suspect are printed or converted to PDF far less than half the time, defaults to the "paged" mode (see Page Setup) that shows document page borders and margins, instead of the far more useful "Pageless" mode which is more like a normal webpage that fits to window and scrolls one continuous surface endlessly.

Different use cases.

"without text overflowing" brings with it a lot of detail. In pdf every letter/character/glyph of text can have an exact x,y position on the page (or off the page sometimes). This allows for precise positioning of content regardless of what else is going on. It is up to the application that writes the pdf to position things correctly and implement letter or word wrapping.

XPS was the closest to reimplementing PDF but microsoft didn't get enough buy in from other parties so it quietly died.

An interesting aspect of PDFs that I didn't know until quite recently is that they're a subset of PostScript and that in fact accounts for some of the heftiness. PostScript is a full-on programming language (albeit an unusual one) but PDFs are not (i.e. they're not Turing complete). They do not support control flow and what could be expressed as a simple loop in PS must be unrolled and stored as a series of simple declarations/expressions for a PDF.

The advantage is that PDFs don't need a full program interpreter to be rendered.

Because as soon as this conversation starts, the LaTeX crowd shows up, and everyone who something meaningful to add as a standard is blocked by that discussion.

One reason is that none of those other formats are suitable for commercial printing as-is.

Cause it works and works good enough. Also, immutability is a feature, not a bug

I did PNG: https://www.nayuki.io/page/png-file-chunk-inspector

Really impressed this site is still running. They'll have what you want https://formats.kaitai.io/

This sounds like the format specification? What are you looking for that is not a document?