Pandoc: What survives a conversion?
I have been working on a new (post-markdown) markup language Carve. And I wanted to know how it performs for input and output. Especially compared to Markdown and Djot, which mainly inspired me here.
And it happens that Pandoc released 3.10.2 recently.
Everyone who converts documents knows the process loses things. Tables are fine in most formats. LaTeX keeps everything. Word is a black hole.
Some of that is true. Some is backwards. You cannot tell by looking, because the failure is quiet – the file still opens, the paragraphs are all there, and the one attribute that carried your meaning is gone.
Why the initial experiment failed
Write a document, convert it, read it back, diff against the original. That is maybe the intuitive way – but it does not really work.
Every pandoc writer normalizes as it serializes. Hand the markdown writer a document and it rewraps your lines, renumbers your ordered lists, moves your link definitions, and writes emphasis with `*` even where you typed `_`. None of that changed the meaning, and all of it shows up in a diff.
So the diff is mostly noise. Tighten it and you are reading reformatting; loosen it until the reformatting drops out and you have loosened it past the thing you were trying to detect.
The way around it is to stop comparing against the input at all.
The approach
Every probe is a pair of documents differing in exactly one feature: a rich one that uses it, a degraded one that does not. Both go through the same writer, same options, same run. Two questions follow.
expressed Are the outputs different at all? A writer that emits byte-identical text for a document with the feature and one without cannot express it. Nothing was encoded, so nothing can be recovered. unchanged Read both outputs back. Compare the resulting syntax trees node by node. If the trees are identical, the round trip ate it.
Both documents take the same path, so canonical styling lands on both and cancels. When a writer scores “cannot express”, pandoc emitted the same bytes for two different documents. No formatting preference produces that.
The gap
Chart data
| | expressed | survives a round trip | | ------------- | --------- | --------------------- | | carve | 64 | 64 | | markdown | 59 | 46 | | html | 63 | 42 | | commonmark\_x | 57 | 39 | | epub | 62 | 34 | | djot | 52 | 33 | | latex | 54 | 24 | | gfm | 50 | 24 | | docx | 53 | 23 | | commonmark | 56 | 21 | | rtf | 40 | 15 | | pptx | 40 | 0 |
Figure 1: Light bar is what the writer can express, dark bar is what comes back unchanged, and the distance between them is the loss.
| Table 1: 66 probes, pandoc 3.10.2. | Format | Expressed | Unchanged | Lost | | ----------------------------------- | ------ | --------- | --------- | ---- | | native (the AST itself) | 66 | 66 | 0 | | | Carve, AST lane | 65 | 65 | 0 | | | Carve, source lane | 64 | 64 | 0 | | | markdown | 59 | 46 | 13 | | | html | 63 | 42 | 21 | | | commonmark\_x | 57 | 39 | 18 | | | epub | 62 | 34 | 28 | | | djot | 52 | 33 | 19 | | | latex | 54 | 24 | 30 | | | gfm | 50 | 24 | 26 | | | docx | 53 | 23 | 30 | | | commonmark | 56 | 21 | 35 | | | rtf | 40 | 15 | 25 | | | pptx | 40 | 0 | 40 | |
HTML expresses 63 of 66 and returns 42. It is a rendering target: the writer encodes for display, and the reader has to guess intent back out of presentation. A third of what it writes does not survive that guess.
LaTeX drops 30. Word drops 30. CommonMark drops 35 – the specification written to make Markdown unambiguous, which turns out to be a separate question from making it expressive.
pptx is the clean case. Expresses 40, returns zero. Every distinction it can draw is one no reader recovers. Converting out of PowerPoint is archaeology.
Where Carve lands
Carve expresses 64 and returns 64. Through the AST lane, 65 and 65.
The rank matters less than the shape: both columns are the same number. Almost everything on that table can write more than it can read, and the difference is where your document quietly changes meaning. Carve keeps what it can express.
That comes from one decision, made early and not revisited – awkward things get syntax rather than becoming an extension somebody bolts on. Per-cell table alignment. Colspan and rowspan. Captions. Attributes on nearly every node. Footnotes, definition lists, line blocks, raw blocks addressed to one target.
A feature with a spelling can be found again. A feature without one gets degraded into prose by the writer, and parser quality is irrelevant at that point.
`native` sits above Carve on the table. That row is pandoc’s own syntax tree written to disk, kept as the 66/66 reference – if your format is the data structure, round-tripping is not an achievement.
Run it yourself
Format comparisons are usually feature checklists, and a checklist cannot tell you that HTML drops a fifth of what it writes, or that pptx returns nothing, or that the interesting number is the distance between two columns rather than either one.
The harness is public. The probes are one file. Every number here regenerates with `make`, and if one looks wrong it can be checked – open a ticket or PR here please.
How useful was this post?
Click on a star to rate it!
Submit Rating
Average rating 0 / 5. Vote count: 0
No votes so far! Be the first to rate this post.
We are sorry that this post was not useful for you!
Let us improve this post!
Tell us how we can improve this post?
Submit Feedback
Post Views: 74
Leave a Reply
Your email address will not be published. Required fields are marked \*
Comment \*
Name \*
Email \*
Website
Δ
This site uses Akismet to reduce spam. Learn how your comment data is processed.