morganwilliscloud/billai-bass
Turn a Big Mouth Billy Bass into a real-time voice assistant powered by Strands BidiAgent + Amazon Nova 2 Sonic on a Raspberry Pi 5 - morganwilliscloud/billai-bass
1 stars · 0 forks · Python
README
🐟 BillAI Bass Builder Guide Powered by Strands Agents Bidirectional Streaming + Amazon Nova 2 Sonic
billai-bass-demo-strands.mp4
Turn a Big Mouth Billy Bass into a real-time voice assistant: you talk, the fish talks back — head swiveling, mouth lip-syncing to its own voice, tail flapping for emphasis. The fish runs a Strands Agents bidirectional streaming agent (BidiAgent) on a Raspberry Pi 5, streaming live audio to and from Amazon Nova 2 Sonic on Amazon Bedrock. No robotics experience needed. No prior soldering experience needed. The person this guide is based on had never plugged in a Raspberry Pi before and got a talking fish in a weekend.
Inspired by the excellent billy-b-assistant project, rebuilt on Strands.
What's in this repo
File What it is
README.md This guide — start here, work top to bottom
billy.py The final working Python — the talking fish
motors.py Standalone motor test rig — run this before billy.py to verify wiring
asoundrc.example The ALSA config that pins the USB mic + speaker as defaults
requirements-frozen.txt Exact dependency versions from a known-good Pi build — pip install -r for guaranteed-working setup
iot-identity/ Optional production-grade credentials: CloudFormation template + walkthrough for X.509-cert auth instead of access keys
How to use this with an AI assistant Paste this README into Claude (or another AI assistant) and say: "I'm building this. Walk me through it one step at a time — wait for me to confirm each step before giving the next." That's the way it was built; that's the way it builds best. See the Build this WITH an AI Assistant section below for what to send when stuck.
🤝 Build this WITH an AI Assistant This project was created with help from Claude Code and you can use any AI assistant to help you get through. Paste this entire guide into your AI and say: "I'm building this. Walk me through it one step at a time — wait for me to confirm each step before giving the next." Your AI will pace the build to you, debug your exact error messages, and adapt when your hardware differs slightly from this guide. Things AI is great at during this build:
Paste your exact error message — the scary 50-line Python traceback usually has one meaningful line, and AI will find it. Send photos — stuck identifying a motor? Can't tell which pin is which? Photograph it and ask. (In Claude Code on a Mac: drop the image file into the terminal, or say "look at ~/Downloads/IMG_1234.jpg".) Say "I know nothing about X" — AI will back up and explain. There are no dumb questions. Ask before cutting/soldering anything you're unsure about.
If you hit something this guide doesn't cover, that's not a failure of you. It's probably a gap in the guide. AI + your error message will get you through.
🛒 Shopping list If you got a kit: you have everything in this table already. Building from scratch: this is the original Amazon order (~$240 total, January 2026 prices).
Item What it's for ~Price
Big Mouth Billy Bass (current "Gemmy" version) The fish. The modern version has 2 motors, which this guide assumes $30
Raspberry Pi 5, 8GB The brain. 4GB would also work $80
27W USB-C PD power supply (official Pi or RasTech GaN) Don't substitute a phone charger — the Pi limits power to USB ports without a proper supply $15
Official Raspberry Pi 5 Active Cooler Heatsink + fan. The Pi runs hot without it $10
MicroSD card, 64GB (e.g. Transcend USD340, any name brand) The Pi's hard drive $12
USB mini speaker (e.g. HONKYOB) Billy's voice $15
USB gooseneck mini microphone (e.g. CGS-M1) Billy's ears $13
MX1508 dual H-bridge motor driver, 5-pack (e.g. Aideepen, sold as "L298N replacement") Lets the Pi safely drive the motors. You need 1; the spares forgive mistakes $9
DuPont jumper wire kit, 120pc male/female mix (e.g. ELEGOO) Connects driver board to Pi. You'll use ~6 female-to-female $7
Soldering iron kit, 60W adjustable with tips + solder 10 solder joints total. Cheapest kit is fine $20
Flush cutters (e.g. KATA 2-pack) Snipping and stripping wires $9
JST XH connector kit Optional — handy for tidy splices, but the build works without it $9
Also needed, not in the kit:
A personal computer (Mac/Windows/Linux) to flash the SD card — ⚠️ corporate laptops often block USB storage; use a home machine A way to plug a microSD into that computer (most cards include an SD adapter) An AWS account (personal, not your employer's) with a payment method — you'll create a locked-down, fish-only IAM user in the prerequisites below Tape (electrical ideally, Scotch works), a small Phillips screwdriver Home WiFi that both your computer and the Pi can join
✅ Prerequisites (do these before build day) 1. AWS / Amazon Bedrock setup (least privilege — do it right) 💸 No AWS account yet? The free tier covers this project. New AWS accounts get $100 in credits (up to $200 with activities), and Amazon Bedrock is usable on the free plan with credits applied to Nova usage. Voice chat is cheap — roughly a penny per few minutes of conversation — so credits buy on the order of 100+ hours of fish banter. Two free-plan caveats:
Brand-new accounts sometimes start with Bedrock invocation quotas set to zero (no payment history yet). If model access is granted but every call fails with ValidationException: Operation not allowed, that's the quota — file a (free) AWS support case asking for Nova Sonic on-demand quota, or upgrade to the paid plan (your remaining credits carry over). Free-plan accounts auto-close after 6 months (or when credits run out). Fine for a demo fish; if Billy's becoming a permanent housemate, upgrade to paid before the deadline — unused credits survive the upgrade.
Your AWS keys will live in a plaintext file on a Pi inside a fish. Fish get demoed, lent out, and left on desks; SD cards pop out. So the keys must be able to do exactly one thing: talk to Nova Sonic. If they leak, the worst case is someone chats with a fish on your dime — not someone mining crypto in your account. Step 1 — Enable model access:
In the AWS console, switch region to us-east-1 (N. Virginia) — Nova Sonic lives there. Go to Amazon Bedrock → Model access and request access to Nova 2 Sonic (amazon.nova-2-sonic-v1:0). Usually instant. (Not v1 — original Nova Sonic hits end-of-life September 2026.)
Step 2 — Create a dedicated IAM user:
Console → IAM → Users → Create user. Name it something honest like billy-bass. Do NOT check "Provide user access to the AWS Management Console" — this user is a robot fish; it never logs into anything. On the permissions screen, choose "Attach policies directly" but don't select any of the AWS-managed policies (no AmazonBedrockFullAccess — that grants far more than the fish needs). Just click through and create the user with zero permissions.
Step 3 — Attach a policy that allows only Nova Sonic:
Open the new user → Permissions → Add permissions → Create inline policy → JSON tab. Paste: { "Version": "2012-10-17", "Statement": [ { "Sid": "BillyTalksToNovaSonicOnly", "Effect": "Allow", "Action": "bedrock:InvokeModelWithBidirectionalStream", "Resource": "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-2-sonic-v1:0" } ] }
Name it billy-nova-sonic-only, save.
That's one action on one model in one region. No S3, no EC2, no other Bedrock models, no ability to see or change anything else in your account. (If you later give Billy tools that touch AWS — say, reading a DynamoDB table — add that specific permission to this policy then, not broad access now.) Step 4 — Create the access key:
User → Security credentials → Create access key → choose "Application running outside AWS". Copy the access key ID + secret somewhere safe — the secret is shown exactly once. These go on the Pi later (§1.6).
Step 5 — Optional but smart: set a billing alarm. Console → Billing → Budgets → create a $10/month budget with an email alert. Casual Nova Sonic chat costs cents; the alarm is there so a stuck always-listening fish (or leaked key) can't surprise you. If the keys ever leak or a kit fish goes missing: IAM → user billy-bass → Security credentials → deactivate the access key. One click, fish bricked, account safe. 2. Know the two paste gotchas (you WILL hit these) When you paste code into a terminal/nano over SSH, two things love to go wrong:
Extra indentation: every pasted line gains leading spaces. ALSA config shrugs; Python explodes (IndentationError: unexpected indent on line 1 is the signature). Fix: sed -i 's/^ //' yourfile.py, or re-paste into nano carefully. Dropped/mangled characters: a missing comma, or quotes turned into “smart quotes.”
The habit that saves you: after creating any Python file, run python -m py_compile yourfile.py && echo OK before running it for real. If it doesn't say OK, paste the error to Claude.
Part 1 — The Brain (Raspberry Pi, no fish yet)
Strategy note: we get the entire voice assistant working on the bare Pi first, and only then open the fish. Software problems and hardware problems are much easier to debug when they can't be each other.
1.1 Flash the SD card On your personal computer:
Download Raspberry Pi Imager from raspberrypi.com/software, insert the microSD.
Choose: Device = Raspberry Pi 5 · OS = Raspberry Pi OS Lite (64-bit) · Storage = your card.
⚠️ OS version matters: you need an image with Python 3.12+ (Nova Sonic requirement). The current "Trixie"-based Raspberry Pi OS ships 3.13 ✅. Old "Bookworm" images ship 3.11 ❌. "Lite" = no desktop. Correct — you'll never plug in a monitor.
⚠️ Pick "Lite" carefully. In the OS list, expand "Raspberry Pi OS (other)" and choose Raspberry Pi OS Lite (64-bit) — not the regular "Raspberry Pi OS (64-bit)" at the top of the list, which is the Desktop edition. The Desktop edition ships a Wayfire panel that aggressively rewrites ~/.asoundrc on every login, breaking your audio config repeatedly with no obvious cause. Lite has no GUI and no panel. Already installed Desktop by mistake? It's fixable without re-flashing — sudo systemctl set-default multi-user.target && sudo systemctl disable lightdm.service && sudo reboot makes Desktop boot like Lite. But Lite from the start is cleaner.
When asked about OS customisation, say yes — this is the magic step:
Hostname: billy Username + password: pick and remember them WiFi: your network name + password, exactly right (a typo here = mystery no-show later) Services tab: enable SSH (password authentication)
Write, wait, eject. ("Raspberry Pi Connect" prompt: skip it — you can add remote access later with one command if you ever demo this somewhere.)
1.2 First boot
MicroSD into the slot on the Pi's underside. Plug in the USB-C power supply — there is no power button; plugging in = on. Wait 2–3 minutes (first boot resizes things and joins WiFi). From your computer's terminal: ssh yourusername@billy.local Type yes (lowercase) at the fingerprint prompt, then your password (the cursor won't move while typing a password — normal).
Gotchas:
Connection reset right after the fingerprint prompt → the Pi is still finishing first-boot setup. Wait 1–2 minutes, try again. (If it then complains REMOTE HOST IDENTIFICATION HAS CHANGED: run ssh-keygen -R billy.local and retry — the Pi regenerated its key mid-setup, this is expected.) billy.local not found → wait longer; then check your router's device list for "billy" and SSH to its IP. Persistent no-show usually = WiFi password typo → re-flash. When done for the day: run sudo shutdown now, wait ~20s, then unplug. Don't just yank power.
1.3 System packages sudo apt update && sudo apt full-upgrade -y sudo apt install -y python3-dev portaudio19-dev swig git (portaudio19-dev → microphone/speaker access for Python. swig → needed to build the GPIO library later; installing it now avoids a confusing error.) 1.4 Audio devices Plug the USB mic and USB speaker into the Pi, then: arecord -l # lists MICROPHONES — note your card name, e.g. "Device" aplay -l # lists SPEAKERS — note yours, e.g. "UACDemoV10" (ignore the two vc4hdmi entries) ⚠️ Use card NAMES, never numbers — the numbers shuffle between reboots (this bit the original build). Now make your USB devices the system default. Create the config with nano (heredocs + SSH paste = pain; nano is your friend): nano ~/.asoundrc Paste, substituting YOUR two card names: pcm.!default { type asym playback.pcm "plug:hw:UACDemoV10" capture.pcm "plug:hw:Device" } ctl.!default { type hw card UACDemoV10 }
Save (Ctrl+O, Enter) and exit (Ctrl+X). Then verify the file actually exists — cat ~/.asoundrc should echo it back. (In the original build this file silently failed to save, which surfaced as a baffling crash days later. Trust nothing; cat everything.) The round-trip test — record 3 seconds of yourself, play it back: arecord -d 3 -f S16_LE -r 16000 test.wav && aplay test.wav Hear yourself? The hardest Pi-specific problem is now behind you. (16000 Hz is deliberate — that's Nova Sonic's rate.) 1.5 Python environment + Strands python3 -m venv ~/billy/.venv source ~/billy/.venv/bin/activate pip install "strands-agents[bidi,bidi-io]" gpiozero lgpio Notes:
(.venv) appearing in your prompt = venv active. You must re-run the source line in every new SSH session — if Python suddenly can't find strands, that's why. If lgpio fails to build mentioning swig: you skipped step 1.3's swig install. sudo apt install -y swig, retry. Pin your version: this build uses Strands' experimental bidi API and subclasses a private class. Run pip freeze | grep strands and write the version down. If a future upgrade breaks things, you can return to it.
1.6 AWS credentials — in a file, not exports Use the keys from the dedicated billy-bass IAM user you created in the prerequisites (the one that can only call Nova Sonic) — never your personal/root AWS keys. Environment variables vanish every time you log out (this caused a crash mid-build), so do it permanently: mkdir -p ~/.aws nano ~/.aws/credentials [default] aws_access_key_id = YOUR_KEY_ID aws_secret_access_key = YOUR_SECRET
nano ~/.aws/config [default] region = us-east-1
chmod 600 ~/.aws/credentials 1.7 First conversation 🎉 nano ~/billy/billy.py: import asyncio from strands.experimental.bidi import BidiAgent, BidiAudioIO from strands.experimental.bidi.models import BidiNovaSonicModel
model = BidiNovaSonicModel( model_id="amazon.nova-2-sonic-v1:0", provider_config={ "audio": { "input_rate": 16000, "output_rate": 16000, "voice": "matthew", "channels": 1, "format": "pcm", } }, )
agent = BidiAgent( model=model, system_prompt=( "You are Billy, a wisecracking animatronic singing bass mounted on a " "wooden plaque. Keep responses short, punchy, and conversational." ), )
audio_io = BidiAudioIO()
asyncio.run(agent.run(inputs=[audio_io.input()], outputs=[audio_io.output()])) python -m py_compile ~/billy/billy.py && echo OK python ~/billy/billy.py Wait a few seconds, then just talk. You should have a voice conversation. Ctrl+C to stop. Gotchas:
A wall of ALSA lib... warnings at startup is normal noise — ignore anything that isn't a Python Traceback. AccessDeniedException → Bedrock model access not granted, or wrong region. no AWS credentials found → step 1.6 not done (or typo'd). OSError: Invalid sample rate → your ~/.asoundrc is missing or has wrong card names (run cat ~/.asoundrc and arecord -l and compare). Billy interrupts himself → the mic hears the speaker and the model thinks you're barging in. Point the mic away from the speaker, lower the volume. Physical placement fixes this.
Part 2 — The Body (open the fish) 2.1 Surgery 101 Remove the ~6 screws on Billy's backplate. Take the batteries out and leave them out. Inside you'll find a small zoo of black lumps; here's the field guide:
Looks like Is Fate
Thumb-sized silver cylinder + black gearbox, 2 wires A motor KEEP
Round disc with grille holes (on the backplate) Original speaker retire
Coin-sized flat silver disc, 2 wires Piezo buzzer (not a motor!) retire
Small board where every wire terminates Original control board retire
Wires to a button on the front The push button KEEP (future "talk to me" trigger)
Wires to a little window/sensor on the front Light/motion sensor retire
Count the motors. Modern Billy (sold today) = 2 motors: one for the mouth, one shared motor that turns the head when spun one way and flaps the tail when spun the other. Classic Billy (1999–2005) = 3 separate motors (this guide covers Modern; for Classic, ask Claude to adapt — you'll need a second driver board and one more GPIO pin). ⚠️ Trace wires with your finger; don't trust colors. Editions vary. In the original build, the "obvious" guess about which motor was which was backwards — the motor visible near the tail was the body motor, the mouth motor was buried near the head. Optional sanity check: with batteries in (fish closed), press the button — the factory song-and-dance proves both motors work. Do this once before surgery if you want peace of mind. 2.2 Disconnect & label
Free the motor wires and button wires from the original board — unplug if they're on connectors, otherwise snip close to the board (maximize wire length on the side you keep). Label each pair with tape flags immediately: MOUTH, BODY, BUTTON. The colors are your only documentation once the board's gone. The board, piezo, speaker, and sensor come out. Keep the button.
Part 3 — Connecting Brain to Body 3.1 Meet the motor driver The Pi's GPIO pins can whisper (3.3V signals) but motors need a shout (real current) — wiring a motor straight to the Pi kills the pin. The MX1508 driver board is the translator: the Pi whispers commands at its inputs, it shouts 5V at the motors. It has two channels (A and B) — one per motor. Convenient. The board has bare holes, so this is where you solder: 10 joints total, the easiest kind there is. Watch any 3-minute "how to solder" video first if you've never done it. You have 5 boards — mistakes are free. 3.2 Prepare the wires Take 6 female-to-female DuPont wires in six distinct colors. For each: snip one plastic end off, strip ~5mm of insulation (bite gently with flush cutters, slide the sleeve off), twist the strands tight. The color map used in this build (adjust to your colors, but write yours down):
Wire Driver hole Job
yellow + 5V power
black − ground
white IN1 mouth (one direction)
grey IN2 mouth (other direction)
purple IN3 body motor → head
blue IN4 body motor → tail
3.3 Solder For each wire: poke the bare end down through the top of its hole (label side up), bend the protruding tip slightly so it stays, flip the board, then: iron tip touching both the wire and the metal ring, count two seconds, feed in solder, pull solder away then iron away. You want a small shiny cone gripping the wire. Then the motor wires into the output holes (strip/twist/solder the same way):
mouth motor pair → MOTOR A holes body motor pair → MOTOR B holes (Within a pair, either hole — backwards just reverses direction, which is fixed in software.)
Quality control — do all three, on every joint:
Tug test: pull each wire firmly. It must not budge. Wiggle test: push it side to side. If the solder blob itself pivots on the board, it's not bonded — reheat 3 full seconds touching blob AND ring. Bridge check: in raking light, confirm a gap between every neighboring blob — especially between + and − (that bridge = short circuit).
Known failure modes from the original build (both fixable in 2 minutes):
Wire pops out leaving a solder-plugged hole → reheat the blob till liquid, push the wire back in, hold still 2 seconds. "Surface tack" — wire stuck onto the top of the blob instead of through the hole. It conducts… until it pops off inside the fish. Redo it properly.
3.4 Plug into the Pi Pi shut down (su