Muse — Songwriting Assistant — Design Spec

Date: 2026-06-27 Status: Approved (design) — pending implementation plan Owner: Joshua

1. Purpose

A personal songwriting companion that helps Joshua write deep, moving, catchy, and harmonically unique songs. It is built around his actual process — melody/lyrics-first, piano-literate, strong intuition, still building chord/scale fluency — and its job is to expand the harmonic palette his intuition can reach for, catch fragments before they evaporate, and grow them into full songs.

Working name: Muse.

The core creative loop

Joshua names a feeling (text now; a hummed voice memo later) → Muse replies with a genuinely interesting, non-cliché chord progression that fits that feeling, with every chord spelled out and a one-line “why it feels that way,” plus a piano audio clip he can hear instantly → he writes words/melody on top → he iterates in chat → Muse helps grow it into a full song → everything is saved to a library.

Priorities (from Joshua, in order)

1. Capture — fragments evaporate; nothing should be lost.
2. Development — grow a verse/hook into a full song.
3. Harmonic ceiling — break out of habitual chords/keys into unique territory.

Deferred chapters (explicitly out of scope for now)

• Lyrics / meaning assistance.
• Full DAW production — turning a composition into a produced, mixed recording. This is a genuinely separate, large project. We revisit it once Joshua has a pile of finished compositions and we know what production support he actually needs. (Phase 1’s audio-clip rendering is the first small bridge toward it.)

2. Experience

Front door: Discord (Joshua’s phone is always in reach; great for voice memos and async, low-friction capture). Grows into a web dashboard later, once the song library is big enough that browsing it becomes the bottleneck.

A Muse reply looks like:

Fmaj7 (F A C E) → A♭maj7 (A♭ C E♭ G) → E♭/G (E♭ G B♭, G in bass) → D♭maj7 (D♭ F A♭ C) Why it floats: A♭maj7 and D♭maj7 are borrowed from F minor — the major-7th color keeps it dreamy while the borrowed chords pull it bittersweet. 🎵 [piano clip attached]

Three layers, every time: 1. The progression, written so Joshua can read and play it. 2. Each chord spelled out (e.g. Fmaj7 = F A C E) — because he’s still building chord fluency, every reply quietly teaches vocabulary in the context of his own song. 3. A one-line “why it feels that way” — the layer that raises his harmonic ceiling over time. 4. A piano audio clip — so he hears the idea instantly, phone in hand, no piano required.

Iteration is conversational: “more tension on chord 3,” “make the bridge lift,” “give me a darker version” → Muse replies with a new card.

3. Architecture

Muse is a single always-on Node.js / TypeScript process running on Joshua’s Mac (which is always on). It listens to Discord, routes each message, and replies with a rendered card.

Discord message (text or voice memo)
        │
        ▼
  ┌─────────────┐
  │  Bot (I/O)  │  discord.js v14 — listens, downloads attachments, sends replies
  └─────┬───────┘
        │
        ▼
  ┌─────────────┐
  │   Router    │  text → Brain;  voice memo → (Phase 2) melody extraction
  └─────┬───────┘
        │
        ▼
  ┌─────────────┐
  │    Brain    │  LLM (Claude) emits a progression as Roman numerals + the
  │ (theory-    │  "why it feels" explanation, conditioned on the feeling/intent
  │  guarded)   │  and (later) Joshua's history/habits
  └─────┬───────┘
        │ Roman numerals + key + intent
        ▼
  ┌─────────────┐
  │   Theory    │  tonal — validates the LLM's chords, realizes them in the key,
  │  (tonal)    │  spells every chord (Fmaj7 → F A C E). Guardrail: no invalid chords.
  └─────┬───────┘
        │ concrete spelled chords + voicing
        ▼
  ┌─────────────┐
  │    MIDI     │  midi-writer-js — build a MIDI file with voicing/rhythm/tempo
  └─────┬───────┘
        │ .mid
        ▼
  ┌─────────────┐
  │   Render    │  FluidSynth (native) + piano soundfont → .wav (→ .mp3 via ffmpeg)
  └─────┬───────┘
        │ audio clip
        ▼
  ┌─────────────┐
  │  Library    │  every fragment, progression, and clip saved (SQLite/local files)
  └─────┬───────┘
        │
        ▼
   Reply card back to Discord (chords + spelled notes + why + audio clip)

Components (each independently understandable and testable)

• Bot (I/O) — discord.js v14. Listens to MessageCreate (plain message-listening, not slash commands — simpler for a single user). Downloads attachments via attachment.url. Replies with text + an audio file attachment. Requires the MESSAGE CONTENT privileged intent (one toggle in the Discord developer portal).
• Router — decides: text → Brain; audio attachment → capture-and-(Phase 2)-harmonize.
• Brain (theory-guarded LLM) — Claude generates the progression as Roman numerals plus the human “why it feels” sentence, conditioned on the feeling/intent (and, in Phase 4, on Joshua’s saved history). A small rule-based emotion→mode/palette map (e.g. melancholy → Aeolian + borrowed iv/♭VI) seeds and sanity-bounds the LLM. Hooktheory Trends API may be added later as a “what real songs do next” enrichment.
• Theory (tonal) — the guardrail. Validates every chord the LLM emits, realizes the Roman numerals into the chosen key (Progression.fromRomanNumerals), and spells each chord (Chord.get("Fmaj7").notes → ["F","A","C","E"]). If the LLM emits something invalid, this layer catches it and asks for a fix.
• MIDI (midi-writer-js) — turns spelled chords into a MIDI file with chosen voicing, rhythm, and tempo.
• Render (FluidSynth) — shells out to the FluidSynth CLI with a free piano soundfont (FluidR3_GM, MIT, or Salamander Grand, public domain) → .wav; optional ffmpeg → .mp3. Both are native binaries installed on the Mac.
• Library — persists every captured fragment (text or voice memo), every generated progression, and every rendered clip. Local SQLite + a files directory. This is the foundation of priority #1 (capture) and of Phase 4 (learning Joshua’s habits).

4. Build vs. assemble

Research confirms ~70% of this is assembly of mature, permissively-licensed parts; ~30% is custom glue. The one genuinely custom, taste-driven component is the emotion → harmony brain — which is a feature, not a gap: it’s where Joshua’s taste and a good LLM live.

5. Hosting

Runs as an always-on Node process on Joshua’s Mac (which is always on and online). Free, immediate, no Pi or cloud container needed. A cheap cloud container (Railway ~$5/mo) remains a someday option if Muse ever needs to be fully independent of the Mac.

Why not Cloudflare: the site’s normal host won’t work here, for two structural reasons. (1) A Discord bot must hold a persistent connection and be always listening; Cloudflare Workers/Pages are serverless — they wake per-request and vanish, leaving nothing to catch a message. (2) FluidSynth is a native binary that must be installed into an OS; serverless sandboxes can’t run arbitrary binaries. Neither is FluidSynth-specific — any audio renderer (Timidity, etc.) is also native, and the pure-JS rendering path (Tone.js) still doesn’t solve the always-listening problem. The real requirement is simply a regular, always-on computer — which the Mac is.

6. Memory / “learns you over time” (staged)

Muse is designed as a companion that builds a picture of Joshua, not a stateless vending machine — built up in stages:

• Stage A (Phase 1): save everything — every fragment, progression, and clip. This alone serves priority #1 (capture).
• Stage B (Phase 4): notice harmonic habits from the saved history (“you’ve leaned on minor-iv three times this month — want a Lydian lift instead?”) and deliberately nudge Joshua past his ceiling (priority #3).

7. Phasing (each phase ships independently)

• Phase 1 — MVP: the magic loop + raw capture. Text in (“dark, hopeful, floating underwater”) → progression + spelled chords + “why” + piano clip → iterate in chat. Every message (text and voice memo) is saved raw to the library from day one (capture #1, the cheap part — just download + store). No melody extraction yet. Delivers the core “give me an interesting progression I can hear” magic.
• Phase 2 — Hum harmonization. Voice memo → Basic Pitch → extracted melody/key → suggest chords that fit the hum. Closes the away-from-piano capture loop.
• Phase 3 — Development. Grow a saved fragment into a full song: structure, bridge, where it goes next (priority #2).
• Phase 4 — Learns you. Habit-noticing and ceiling-nudging over the saved library (priority #3, Stage B memory).
• Phase 5 — Web dashboard. Browse/develop the library visually once it’s large enough to need it.

8. Success criteria

• Phase 1 is a success when: Joshua can send a feeling from his phone and, within seconds, get back a progression he finds genuinely interesting (not cliché), correctly spelled, with a one-line reason he learns from, and a piano clip he can hear — and every idea he sends is saved and retrievable. He reports it actually produces songs/fragments he wouldn’t have written alone.

9. Open questions (for later phases, not blocking Phase 1)

• Voicing/rhythm style for the rendered clip (block chords vs. gentle arpeggio vs. pad) — start simple (block chords + light arpeggio), refine by ear.
• Whether to add the Hooktheory Trends API as a “real songs” enrichment to the Brain.
• Dashboard scope and when the library is big enough to warrant it.