In 2018, I wrote a YAML parser in Bash. It was funny, it was useful, and at its centre sat a genuinely bad idea about parsing.
Frank Vumbaca helped turn the first experiment into something people could actually use. He added blocks, lists, stdin support, documentation, and years of patient fixes. Then the repository mostly sat still. It kept doing its small job while I went on to other things.
I came back eight years later intending to tidy it up and ship a respectable release. Then I asked a more dangerous question: if we were willing to break the CLI, how much better could this become?
The joke had a ceiling
The original YAML.sh translated YAML into flattened paths and values. That was a sensible shape for shell scripts: easy to print, easy to grep, and just enough structure to query a configuration file.
It was also the ceiling. Once a mapping, sequence, alias, tag, or empty collection had been flattened, important relationships were gone. Every new feature had to reconstruct meaning the parser had already thrown away. Editing YAML without mangling it was almost impossible.
I asked whether there was a better parsing design I had missed. There was. The missing idea was not a smarter regular expression. It was an intermediate representation that respected YAML.
The answer was a node graph. Mappings remain mappings. Sequences remain ordered. Empty collections survive. Tags and source positions stay attached. Anchors and aliases can share identity. Queries and updates operate on nodes instead of guessing their way back from strings.
YAML stream → node graph → expression stream → values / YAMLGive YAML a shape
That change broke the old CLI. I decided that was fine. Compatibility with a clever limitation was less valuable than making the premise useful.
We also stopped requiring Bash. The released program is one POSIX shell file with its AWK parser and evaluator embedded. The .sh name still fits: the shell owns the executable, arguments, files, and process behaviour; AWK does the work it is unusually good at. Calling it yaml.awk would describe an implementation detail while hiding the way people actually use it.
The goal became simple: make an improbably capable YAML tool in one readable shell executable. Give it a proper graph, a compact language, and edits that respect the file people wrote—without acquiring a language runtime, package manager, YAML library, or plugin system.
“The point is to have fun.”
Teach it to change things
Once the graph existed, each idea made the next one possible. Traversal became an expression stream. Expressions became transformations. Variables, reducers, filters, merges, paths, metadata, and multi-file evaluation grew from the same small model.
Reading YAML is forgiving. Writing it is where tools reveal their manners. People leave comments, choose quotes, arrange keys, and use whitespace to make configuration legible. A semantically correct rewrite can still be a terrible edit.
So YAML.sh learned to track where nodes came from. Simple changes replace only the source they own, leaving comments, formatting, ordering, and unrelated text alone. --diff previews the result. --preserve-only refuses a change when that promise cannot be kept. Updating several files uses the same prepared results and rolls back if a write fails.
One file got ambitious
The graph did not stop at YAML traversal. It became the shared shape for JSON Pointer and Patch, Merge Patch, a focused JSON Schema validator, and TOML, INI, and XML conversion. The query language picked up codecs, guarded local loads, reproducible shuffle, dynamic expressions, and access to source details such as comments and columns.
None of that changed the delivery model. Download ysh, make it executable, and read it if you are curious. The shell handles arguments and files; the embedded AWK handles parsing and evaluation.
The restraint moved elsewhere: every supported behavior needed an exact example or test, and nearby unsupported syntax had to fail clearly instead of producing a plausible mistake. The YAML Test Suite and yq were useful external checks, but neither became the product definition.
A very fast second act
The second act moved unusually quickly, because I was no longer working alone. With machine collaborators in the loop, a design question could become working code, meet the YAML Test Suite, expose the missing idea, and come back to the drawing board while the thought was still fresh. What used to take a weekend now took the length of a coffee.
Speed like that is dangerous in exactly one way: it will cheerfully automate sloppiness. So the rule became that nothing lands without a receipt — an exact test, an external oracle, a property that fails loudly. The interesting part was never code volume. It was learning how hard the one-file constraint could be pushed when the boring verification kept pace with the fun.
Where it fits
The unreasonable constraint eventually became a useful shape of its own.
YAML.sh is a serious YAML tool in a mischievous medium. It arrives as readable source, keeps the character of the file you wrote, previews its own changes, refuses formatting loss on demand, and treats a multi-file update as one careful transaction.
It fits when one readable file, careful edits, and a surprisingly deep language matter at the same time.
The constraint is the fun part
YAML in shell still sounds like a bad idea. I like that. Good constraints create a shape you would not reach by chasing generality, and a little absurdity can provide enough energy to stay with the boring work that makes an idea dependable.
The premise survived the rewrite: one readable file, no extra language runtime, useful YAML work. Everything else was allowed to change.
The point was to have fun. The trick was refusing to use fun as an excuse to be sloppy.