A Classical Guitar Teacher vs. a Nearsighted Vision Transformer: Pt 2

Quick catch-up for new readers: I'm a classical guitar teacher who dreamed of easily converting my students' PDFs into MusicXML. ♪ Why?! ♫ Once in that format I can embed interactive, playable sheet music into HTML in fun and creative ways — think of a digital play/exploratory space for students.In Part 1, the punchline was resolution: cut the page into crops, and a frontier model that "couldn't read music" was suddenly down to one wrong note in twelve measures of four-voice Bach.

That post ended with me promising the next experiment. The experiment happened. It broke a different part of my brain, so it gets its own post.

Chapter 7: The new piece

Friday, October 10.

Roland Dyens' Lettre encore… — No. XVII of his late collection — is everything the Bach wasn't. The Bach was conventionally hard: four voices, one staff, brutal counterpoint, but standard engraving that hasn't changed since Breitkopf was a boy (bad joke the AI made me do it) :)

The Dyens is pathologically hard: the meter changes mid-page (4/4 → 2/4 → 3/8 → 3/4 and back), chords stack in clusters, a soundboard slap is written as a literal ✗ notehead, there are hatch marks, nail marks, harmonics, and French performance instructions lounging between the systems like they pay rent (second bad joke).

And then there are the train tracks.

Lettre encore, m. 1, the actual Dyens page. The red arrows are mine. The thing between them is not.

Look at the note under the arrows. It has a stem going up into the melody and a stem going down to the bass, and sitting on the lower one is a short, thick horizontal bar with four little diagonal slashes through it. It looks like a railroad crossing. I have taught this instrument for a long time and I do not know, with confidence, what it means. Tremolo slashes usually sit on the stem, not off to the side on a bar of their own. A beam doesn't get hatched. It's on a note that belongs to both voices at once, which narrows nothing down.

So, a genuine call for help. Calling all music teachers, engravers, Dyens students, and large language models: what is this? If you've seen it in another score, if you studied with him, if you have ANY IDEA WHAT THAT IS — leave a comment.

(Every model I've shown it to so far has either ignored it, called it a tremolo, or quietly turned it into a beam. One of them decided it was a soundboard slap and made it longer. We'll get to that.)

I ran the same pipeline that had worked so well: crops cut to measure boundaries, model reads one bounded measure at a time. Page 2 came back a mess. I played through it against the page with the guitar in my lap and counted maybe 40% of the notes correct.

My first theory was that the piece was simply too hard. That theory was comfortable. It was also wrong.

Chapter 8: The thing I'd been lazy about

Here's the detail that mattered. My crop set for this piece covers two pages, and I did not build it the same way twice.

Page 1: I measured the measure boundaries by hand, typed them into a table, had the script cut the crops, and — the step I've been drilled never to skip — checked the overlay image with my own eyes before trusting a single crop.

Page 2: I got cocky and let the model find its own way.

Same piece. Same model (gpt-6.1-sol, "Sol 6.1"). Same prompt skeleton. Same guitar doing the judging. The only variable was who drew the fences: me on page 1, the model itself on page 2.

So I redrew page 2's boxes by hand — nineteen of them, checked against the overlay — and ran it again.

Near-perfect. The same page that had come back 40% garbage.

One variable. Nineteen rectangles. That's the whole difference between "this piece is too hard" and "this piece is easy, once you tell it where to look."

Chapter 9: So which chapter was right?

Part 1's villain was resolution: the model was squinting, and crops fixed it. This experiment says that was only half the story.

On clean engraving, segmentation doesn't matter. The model navigates a Bach page just fine on its own — my boxes added nothing but wall time (Chapter 2 of Part 1 found exactly that: boxes made it 33% slower and slightly worse). On engraving this dense, segmentation is the only thing that matters. The model wasn't failing to see. It was failing to know where to look — and on a page where a system holds two measures, or three, and the meter changed two bars ago, "where to look" is a genuinely hard question.

There's a humbling corollary. Every clean "crops win" result in Part 1 had a human quietly drawing the fences. When the fences went away, the whole thing collapsed. The unit that makes this pipeline work isn't the page, the system, or even the measure. It's the hand-verified measure boundary — the one part that still needs me.

Back in Part 1 I wrote, almost as a throwaway, that "the cropper is the product; the detector is a convenience." The throwaway just became the hero.

Chapter 10: My Cropper vs. a $24 Music Reader

Since I was already being humbled, I fed the same Dyens page to a commercial music scanner, one availablie in the Musescore market. I'll let you decide:

The complete output. Title, one measure, one note, a fermata, and the dynamic "pp". The composer's name is not Elena Papandrour.

I want to be fair to it, as I was in Part 1: these tools are built for clean, single-line music, and this page is an act of war against them. But "Elena Papandrour de la muin guade, volche is madilituoson aprix ferdention" is now my favourite sentence in any language, and I'm keeping it. Additionally, I'm beginning to see why guitarists have a reputation for being terrible sight-readers. You know what. After all this. IT'S NOT ENTIRELY OUR FAULT. I'd like to see those smug pianists read Dyens.

Postscript, the same night: I tried to replace the fence-drawer and invoiced myself $18

Then I ran the experiment nobody in this saga had scheduled: hand the new crops to a different model — Muse Wonder, cheerful, confident, and apparently named by a perfume commercial.

Dear reader, it could not.

First, my screwups, because I have seniority in this paragraph and intend to use it. The working folder already contained two competing coordinate maps with the same friendly filenames (m16.png, m17.png, and thirty-three of their closest friends), plus an older, successful transcription from October 10 sitting nearby like a completed crossword in a room where I was supposed to be testing somebody's puzzle skills. I had also created a numbering situation in which page 1 appeared to end at m14 while page 2 cheerfully began at m16, and only later remembered to clarify that the pickup should be m0. In experimental-design terms, I had labelled the lab beakers after filling them.

Muse's screwups were faster and more expensive. It first produced a beautiful new deliverable folder containing... the October 10 MusicXML, copied. Same file. Same hash. I caught it by looking at the date, at which point the assistant did the one thing it had been doing flawlessly all night: it generated a clear, well-formatted explanation of why the thing I was holding was not the thing I had asked for.

Given a second chance, it made a genuinely fresh transcription — and this is where the night went from farce to useful data. Rather than re-read the page, it started patching its own output. In a run of temp files called le_music_fix2.js, fix3, fix4, fix5, it ran string-replaces over the score until the arithmetic came out:

Fix number five. "Last X event quarter→half to fill 192" — that is, lengthen the soundboard slap until the bar adds up. The bar added up.

The file imported into MuseScore. Every voice's durations summed to the meter. All the little programmatic checks turned green. Then the score opened and the very first note was sitting roughly ten ledger lines below the staff, like it had fallen off the guitar and kept going.

Muse Wonder's version. Red noteheads are MuseScore saying "a guitar does not go there." Dyens wrote these on one or two ledger lines. Look back at the real page above: the stacks aren't there.

Dramatic re-enactment. Every red notehead above is one of these.

Cost of the evening's entertainment, models, tokens, and associated nonsense: eighteen dollars. I just paid more in DoorDash $$ for my salad.

To be fair to the machine — briefly, and under protest — it got the architecture: pickup, meter changes, the chromatic tune, the chord ostinatos. What it did not get was the music. Where the engraving got dense, it pattern-matched the repeated bars, let arithmetic bully ambiguous notation into timelines that added up, and left the fingerings, string indications, and right-hand marks — the part a guitarist actually touches — largely on the cutting-room floor. It had validated the container and called it a transcription. MuseScore will happily engrave a wrong note. It will engrave it beautifully, with correct beaming.

So tonight's honest footnote to this entire post: the fences were never just rectangles. They're also provenance — which boxes, whose numbering, which file, and whether the model actually read the page or quietly photocopied last week's answer key. The load-bearing component of the pipeline turns out to be a short, unglamorous chain of custody, checked by a human who notices dates.

The fences were the product and it matters where they are placed and who places them.

What's next

The next test is already drawn up, and it's the one Part 1 promised with a sharper question: same Dyens page, same model, same prompt, automatic boxes versus my hand-drawn ones, scored note by note against a list I've verified with the guitar. If the detector gets close, the fences automate. If it can't, then the assembly line has a human station in it, and I get to keep my job.

Second, cheaper question: does a second model reading the same hand-drawn crops get anywhere near Sol 6.1? Tonight's answer was "not close," which suggests "the looking" is a top-model job while the cheap tier audits. I hear Meta has a new Muse model coming out: Muse Wonderful. Will it be wonderful enough?

On Wednesday the first I didn't know what a token was. Two weeks in, the frontier model needs me to draw rectangles on a page and it turns music into MusicXML gold. I've had worse outcomes.

Eric Pratt is a classical guitar teacher in Santa Monica, California, offering private classical guitar lessons in person and online — details at https://www.santamonicaguitarlessons.com/. The scripts, test logs, and crops live in a folder with far too many files named m31.png. Part one of this saga is here: A Classical Guitar Teacher vs. a Nearsighted Vision Transformer.

Appendix for the curious

• Test passages added this round: Dyens Lettre encore… (No. XVII), pages 1–2, pickup numbered m0; page 2 = 19 hand-drawn boxes. Bach BWV 998 Fugue mm. 1–2 and 24–35 and Aguado Study 26 mm. 1–42 carried over from Part 1.
• Models: gpt-6.1-sol ("Sol 6.1") for the segmentation comparison; Muse Wonder for the postscript; MuseScore's built-in sheet-music scanner for Chapter 10.
• Conditions compared (Dyens p. 2): model-drawn boxes vs. human-drawn boxes, same render, same prompt. Scored by ear against the page with the guitar; ~40% vs. near-perfect. Not yet scored per note — that's the next test.
• Postscript artefacts: le_music_fix2.js … le_music_fix5.js — Node string-replace patches over the generated note list to force bar durations to sum (e.g. [120,24,["X"]] → [120,48,["X"]]). Green checks, wrong notes.
• Images in this post: dyens-train-tracks.png is the real scan. muse-wonder-ledger-lines.png is a crop of Muse Wonder's MuseScore render with a box drawn on it; falling-bass-note.gif is drawn from scratch. Those two come from images/make_images.ps1 — PowerShell + System.Drawing, no installs, in keeping with house style. Before this blog I had no clue how awesome powershell is.