Case study / Claude Code
Five Failures, One Demo
How Claude Code built a narrated screen recording of a bash script — by failing at screen capture, tab naming, PATH resolution, and coordinate math before getting it right. Every failure was isolated, diagnosed, and fixed without contaminating the next step.
Most people think AI either works on the first try or doesn't work at all. The interesting part is what happens between those two states — the iteration loop where each failure narrows the problem space until the solution is obvious.
This is a record of that loop, unedited. The task was simple: record a 15-second demo video of a bash script running in iTerm, with voice narration. The execution required chaining five tools — AppleScript, screencapture, ffmpeg, TTS, and bash — across a dual-monitor Mac. Here's what actually happened.
1
Position a window at a known rect
Can we put an iTerm window exactly where we want it?
AppleScript's set bounds places a window at pixel coordinates. We set it to {100, 100, 900, 600} and captured a screenshot of that region.
$ osascript -e 'tell application "iTerm2" ...
set bounds to {100, 100, 900, 600}'
$ screencapture -R 100,100,800,500 step1-test.png
captured
PNG image data, 800 x 500
PASS
Window at exact coords, clean terminal, prompt ready
The screenshot showed exactly the iTerm window — title bar, dark background, zsh prompt. No bleed from other apps. First try.
Lesson
Start with the step that has the fewest unknowns. Window positioning is deterministic — if this fails, nothing downstream matters.
2
Capture video of that window
Can we record what's happening in the window as a video file?
This is where it got interesting. Three different approaches failed before we found one that worked.
ffmpeg crop
→
FAIL
↩
2x Retina
→
FAIL
↩
other screen
→
FAIL
↩
screencapture -l
→
PASS
FAIL 1
ffmpeg screen capture + crop — captured Messages app
ffmpeg's avfoundation captures by screen index, not window. We assumed screen 0 = main display and cropped to our window coordinates.
$ ffmpeg -f avfoundation -i "4:none" -vf "crop=800:500:100:100" ...
kb/s:134.39
The extracted frame showed iMessage conversations and a Telegram sidebar — completely wrong region. Screen index 4 ("Capture screen 0") was the second monitor.
FAIL 2
Assumed Retina 2x scaling — displays are 1:1
Hypothesis: maybe the coordinates need to be doubled for Retina. We tried crop=1600:1000:200:200.
$ system_profiler SPDisplaysDataType | grep Resolution
Resolution: 2560 x 1440 (QHD/WQHD)
UI Looks like: 2560 x 1440 @ 60.00Hz
Not Retina at all — both displays are 2560x1440 at 1:1. The Retina hypothesis was wrong, but we learned the display geometry.
FAIL 3
Tried the other screen index — black/wrong area
Captured one frame from each screen and cropped at the window position. Screen 5 was black. Screen 4 showed Messages again. The fundamental problem: ffmpeg screen indices don't map predictably to macOS display arrangement.
At this point, three attempts to make ffmpeg crop work. Time to pivot.
PASS
screencapture -l <windowID> — perfect isolation
macOS screencapture -l captures a specific window by its CGWindowID, regardless of position, display, or overlapping windows. We got the window ID via Swift:
$ swift -e '... CGWindowListCopyWindowInfo ...'
WindowID: 10281 X:1500 Y:800 W:1000 H:600
$ screencapture -l 10281 -o step2-window.png
captured
Clean capture. Just the terminal window, nothing else. For video: rapid screencapture -l in a loop (~5fps), then stitch with ffmpeg.
Lesson
When the "correct" approach keeps failing, switch to the one that already proved it works. Three attempts at ffmpeg crop vs. one successful screencapture -l. That's not giving up — that's engineering.
3
Send commands and record them
Can we script the demo and capture each command as it runs?
The script needed to show four commands: create, send, list, close. Two things broke before it worked.
send by name
→
FAIL
↩
full path
→
FAIL
↩
create w/o cmd
→
PASS
FAIL 4
iterm-tab.sh not on PATH in the demo shell
The demo window was a fresh shell. The script lived at ~/.claude/scripts/iterm-tab.sh, which isn't on PATH by default.
zsh: command not found: iterm-tab.sh
Fix: use the full path ~/.claude/scripts/iterm-tab.sh in every command.
FAIL 5
create with command — tab name gets overwritten
We ran create poller "echo polling...". The command ran in the new tab, and iTerm replaced the session name with the process name. When send poller ran later, the tab was named something else entirely.
execution error: Tab not found: poller (-2700)
Fix: create poller without a command (name sticks), then send poller "echo ..." separately.
PASS
All four commands succeed, captured in 44 frames
Tab 'poller' created # create
(no error) # send
* Ship log verification # list
* poller (-zsh)
Tab 'poller' closed # close
44 frames at ~5fps, stitched to an 8.8-second MP4. Every command visible, every output captured.
Lesson
Test the exact environment your demo runs in, not your development environment. A script on your PATH isn't on every shell's PATH. A tab name you set isn't the name iTerm keeps.
4
Generate narration audio
Can we create four TTS clips that describe each command?
Four clips, one per command, generated via xAI's TTS API with a consistent voice.
01-create.mp3: 3.024s "Create a named tab called poller."
02-send.mp3: 3.096s "Send it a command without switching tabs."
03-list.mp3: 3.480s "List shows every tab across all windows."
04-close.mp3: 3.000s "Close it by name. Clean exit."
PASS
All four clips generated and verified — first try
The TTS API is a single POST with a voice ID. No iteration needed because there's one API, one call shape, one output format. Simple interfaces don't need iteration loops.
5
Combine video + audio
Can we sync the narration to the terminal recording?
The video was 8.8s, the narration 14.9s. Solution: re-stitch frames at a slower framerate (2.95fps) to match the audio duration, then mux with ffmpeg.
# 44 frames / 14.9s = 2.95 fps
$ ffmpeg -framerate 2.95 -i frames/%04d.png \
-i narration.mp3 \
-c:v libx264 -c:a aac -shortest \
iterm-tabs-final.mp4
Duration: 14.9s Size: 236K
PASS
Video + audio synced, plays correctly
236KB final file. Each command appears on screen while its narration plays. The math is simple once you have frame count and audio duration — framerate is just division.
Scorecard
| Step |
Test |
Failures |
Result |
| 1. Position window | Screenshot at exact coords | 0 | PASS |
| 2. Capture video | Window-only recording | 3 | PASS |
| 3. Record demo | All 4 commands visible | 2 | PASS |
| 4. TTS clips | 4 clips play correctly | 0 | PASS |
| 5. Combine | Video + audio synced | 0 | PASS |
What this teaches
The five failures weren't bugs — they were the process working correctly. Each failure was isolated (it broke one step, not the whole pipeline), diagnosed (we knew exactly what went wrong), and educational (it revealed something about the system we didn't know).
If we'd tried to build the full pipeline in one shot — window positioning + screen recording + command scripting + TTS + video editing — failure #3 (wrong screen index) would have been invisible inside a 50-line script. We'd have debugged for an hour without knowing which layer was broken.
The method: build the smallest thing. Test it. Fix it. Then add the next piece. This isn't just how LLMs should work — it's how all complex systems get built. The LLM part just means you can watch the whole process in 15 minutes instead of 15 hours.
AppleScript
screencapture
ffmpeg
xAI TTS
Swift
bash
Five tools. Zero frameworks. Tested at every seam.