True! — nervous — very, very dreadfully nervous I had been and am; but why will you say that I am mad?
Edgar Allen Poe, The Tell-Tale Heart
What’s a chapter?
I started doing my Etheric Currents radio show on RadioSpiral back in 2021, and like many of the other hosts, I recorded the episodes so people who couldn’t tune in Monday nights could listen to the show whenever they had time via a podcast.
Most podcasts just record the MP3, clean up the worst of the um’s and er’s and verbal tics (I say “all right!” waaay too much), and then publish their show as a simple MP3.
But! MP3’s support chapters, and chapters are pretty cool. They allow your podcast to look a lot like an audiobook in podcatchers: the episode has a list of chapters, and the listener can skip around or repeat them, and the podcatchers put up the chapter info in their display.
Rogue Amoeba’s Fission editor does a great job of creating chapters: add a “split” at the appropriate point on the timeline, and you can add the chapter metadata right there. Great UX.
And, as shown above, podcatchers like Overcast and Apple Podcasts will show the chapters in their displays.
Why is it important now?
If you’re a podcaster signed up at podcastsconnect.apple.com, you probably recently got an email from Apple letting you know that they were “enhancing chapter support” in Apple-hosted podcasts. Specifically:
- If your podcast already has chapters, nothing changes
- If it doesn’t, then Apple’s going to use AI to “listen” to your podcast episodes and insert chapters for you.
I’m guessing that for 90% of podcasts, that this will work pretty well, but if you are a bit obsessive about getting stuff “just so,” you will probably be happier doing this yourself.
There are lots of other options for chapterizing your episodes; several online tools will take a list of times (mm:ss) and names, and plop all the chapters in where you want them. Fission (aforementioned) will let you do it in a nice graphical interface.
The MP3 specification version for ID3v2.3 allows for quite a bit of information in the chapters, including per-chapter graphics. My show plays a lot of tracks from our music library, and I wanted to add the covers and track names as chapters, so a listener could see a track list and the cover for the album the track came from. (Our remit is promoting the music we play; this is another great way to remind people that the music comes from real albums by real artists, and they should pick it up to support them.)
So I started doing that with Fission, and for the first year or so, it worked perfectly. I copied the artwork from iTunes, pasted it in, updated the metadata, and it just worked.
I could have sworn I did this right…
So I didn’t notice when at some point, it stopped working. The chapters were still getting created, no problem, but the extra metadata? Disappeared as soon as I saved. I had a conversation with Rogue Amoeba around that time in 2022 and it came out that yes, we should be able to do that, but the current application could not [my note: “anymore”] and it’d go on a “future enhancements” list.
Sigh. Well.
I mean, I get it, this is kind of niche for someone who just wants chapters; not a lot of people care about per-chapter graphics, and that’s okay. Rogue Amoeba doesn’t exactly have an infinite number of developers, and there are a lot of other things they do that I really do not want to go stale, so no shade on Rogue Amoeba at all.
I let this go for a while. Life happened, and my unposted podcast backlog started getting…long. After a bit of a kerfuffle on the RadioSpiral Azuracast server a few weeks ago that resulted in the Docker volume that the podcasts live on getting clobbered (for anyone else running Azuracast: full backups!). I decided that I really needed to start playing catchup here: restore those, and rebuild/restore the three years of backlog. (I’ll document the process of digging myself out of that hole in another post and link it here when it’s done.)
One of the big problems was “I want my nice chapters back.” And that’s what the rest of this post is about.
First: prove to myself that I am not mad
I remembered having episodes where everything worked, and I dug around in the archived episodes I had put back up, and found one. Loaded it in Fission, and skipped through the chapters. Every chapter had the title of the track and the album art. It did work.
I tried again in Fission. I’d been using it to chapterize the Miskatonic University Podcast episodes that I’ve been editing, and that was working fine. Chapterized a current podcast of mine: chapters were there, titled fine…but no art.
If I added it, Fission took it, and threw it away on save.
“I felt that I breathed an atmosphere of sorrow”.
Edgar Allen Poe
In desperation, I asked ChatGPT: “Chapterized MP3’s: is it actually possible to have a custom graphic for each chapter?”.
Initially, it seemed like bad news: most podcatchers don’t support the enhanced chapter stuff in the IDv32.4 spec so I was out of luck. My reply was, lookit, I have an episode right here that works in my podcatcher. Can we analyze it and see what structure it has? Whatever it does is good enough to work.
So between the two of us, we put together a Python script using mutagen to read the good episode and dump out all the IDv3 data to see why this one was good and the others were not.
A few minor issues with hallucinations about how mutagen works and we had a working script that dumped it all. I thought about it a bit, and had the idea: let’s compare the output from a good file to the bad one I’d just created, and see what’s different.
The problem was right there, and easy to fix. The metadata was IDv32.3, and that the podcatchers do understand…but a little more explanation of how chapters work and what’s in them is probably useful before I start blithering about the fix and offsets and hex values.
The basics of IDv3 data
IDv3 data is sandwiched into the MP3 file, with offsets as pointers to each successive part and to subparts of individual bits of that data. Here’s the overview of how it’s all hooked up:
┌─────────────────────────────────────────────┐
│ ID3v2.3 Tag │
│ (this sits before the MP3 audio frames) │
└─────────────────────────────────────────────┘
│
▼
┌────────────────────────────┐
│ CTOC (Table of Contents) │
│ element_id = "TOC" │
│ flags: TOP_LEVEL, ORDERED │
│ │
│ child_element_ids: │
│ ["ch0", "ch1", │
│ "ch2", ...] → chapter │
│ order │
└────────────────────────────┘
│
▼
For each child element_id listed in CTOC:
┌──────────────────────────────────────────────┐
│ CHAP (Chapter frame) │
│ element_id = "ch0" │
│ start_time = 0 │
│ end_time = 168200 │
│ │
│ Subframes: │
│ ├── TIT2 → "Intro" (chapter title) │
│ └── APIC → this chapter's artwork │
└──────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────┐
│ Next CHAP (e.g., "ch1") │
│ start_time = 168200 │
│ end_time = 347880 │
│ │
│ Subframes: │
│ ├── TIT2 → "Item Title" │
│ └── APIC → this chapter's (different) │
│ artwork │
└──────────────────────────────────────────────┘So it’s a pretty simple linked list, with the subframes either containing or point to other IDv3 data. When a podcatcher gets to the CHAP data, it picks it up and displays it. Simple.
The “good” file’s data looks like this:
CHAP #1:
element_id: 'ch0'
start_time: 0 ms
end_time: 280764 ms
start_offset: 0
end_offset: 0
TIT2 (chapter titles): - ['Typhoon/Dymaxion Variations/Mercury Intro']
APIC (per-chapter images):
APIC #1:
mime: image/jpeg
type: PictureType.OTHER
desc: ''
data length: 3053522 bytes
CHAP #2:
element_id: 'ch1'
start_time: 280764 ms
end_time: 598804 ms
start_offset: 0
end_offset: 0
TIT2 (chapter titles): - ['Mountain Dreams']
APIC (per-chapter images):
APIC #1:
mime: image/jpeg
type: PictureType.OTHER
desc: ''
data length: 192594 bytesNow compare the “bad” file:
CHAP #1:
element_id: 'ch0'
start_time: 0 ms
end_time: 163996 ms
start_offset: 4294967295
end_offset: 4294967295
TIT2 (chapter titles): - ['Etheric Currents Intro: Green']
APIC (per-chapter images):
APIC #1:
mime: image/jpeg
type: PictureType.OTHER
desc: ''
data length: 2329950 bytes
CHAP #2:
element_id: 'ch1'
start_time: 163996 ms
end_time: 483996 ms
start_offset: 4294967295
end_offset: 4294967295
TIT2 (chapter titles): - ['Dripping Green']
APIC (per-chapter images):
APIC #1:
mime: image/jpeg
type: PictureType.OTHER
desc: ''
data length: 46665 bytesHm. That’s different. And that number looks familiar…
4294967295 -> 0xFFFFFFFF
Ha! That is, according to the IDv3 spec, a NIL pointer.
So Fission is writing the image data in, even in the “bad” file: we can see the data length is non-zero. But it’s also writing a nil “these aren’t the images you want, move along” pointer into the file, so anything reading the file says, “nope, no chapter image here, just use the base one for the file”! And that includes Fission itself when it reloads the file!
Now that I know what’s wrong…can I fix it?
Spoiler: YES. And it’s trivial! All I have to do is scan through the CHAP blocks, and set the start_offset and end_offset to 0.
Take a look at rescue-busted-offsets.py in my Podcast Repair Kit repo on GitHub. Better yet: here’s output from the chapter-report.py tool (another handy little item, for visualizing your podcast episodes!)in that same repo, showing a podcast file from before and after a repair:
I’ll have more to say about the catch-up process in general in another post.
Leave a Reply
You must be logged in to post a comment.