_grammar_bias returned a shared module-level singleton that the loop
mutated in place (EOS block + repetition penalty). The penalty thus
accumulated across positions within a call and persisted across calls,
collapsing output to HOLD/NC until process restart. Clone the bias each
step so edits stay local. Add regression tests guarding the invariant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Disable the run button and show a "Генерация…" status while a request is
processing, then re-enable it. Repeated generations previously gave no
visible feedback once the model was cached, so it was unclear whether a
request was running.
- Append elapsed time to the final status so identical re-runs differ visibly.
- Rename the H1 heading and browser/tab title to
"генератор аккордовых последовательностей".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Runnable end-to-end report combining narrative, code, and inline figures:
data and .chord format, transformer working principle, two-stage training
curves, perplexity (3.58 -> 2.15), distribution-shift plot with a reading
legend, qualitative examples, and a generation demo. Written in a
first-person student voice.
- CLAUDE.md: report is now a Jupyter notebook; GOST formatting dropped
- requirements.txt: add nbconvert + ipykernel (optional, for the notebook)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Single-page form wrapping src.generate.generate_period: pick model, mode,
key, style, function, time, sampling params and optional prefix; returns
the chord grid plus downloadable .chord and .mid files. Russian usage
instructions are embedded on the same page.
Auto-length output is capped at 16 bars (the period maximum) so a model
that never emits EOS can't run away into dozens of NC/hold bars.
Added per the author's explicit request — web UI was previously out of
scope; updated CLAUDE.md and README accordingly. Choices for style/
function/time are derived from VOCAB so the form can't drift from the
tokenizer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tracks ROOT-level bigrams (prev_root → curr_root) across chord-change events.
At each FREE position, subtracts penalty * count(prev→root) from ROOT logits,
capped at 3.0 to prevent NC/HOLD flooding at extreme values.
Practical range: 0.5 (mild, breaks loops after 2 occurrences) to 1.0
(aggressive). Default 0.0 keeps backward compatibility.
Added --repetition-penalty flag to scripts/generate.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A sequence of only metadata tokens followed by EOS would silently return a
ChordPeriod with bars=[], which would later crash or produce an empty .chord
file. Now raises immediately with a descriptive message. Added a failing-then-
passing test to cover this path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements perplexity computation, chord distribution extraction (qualities,
extensions, inversions, root-motion intervals), 4-panel comparison plot, and
paired qualitative example generation for pretrained vs finetuned model.
Results on user val set: pretrained PPL 3.58 → finetuned PPL 2.15 (−40 %).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed checkpoints/*.pt from .gitignore; files are now stored as LFS
objects (pretrained.pt 17 MB, finetuned.pt 17 MB). LFS attributes were
already in place in .gitattributes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Added the official DDMAL link at the first prose mention in README.md,
docs/architecture.md, and docs/glossary.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Matches the configuration that produced finetuned.pt (val ppl 2.15,
best epoch 20, early stopped at 30).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Renamed and expanded to cover the full generation workflow:
- §1 quick start examples
- §2 complete CLI reference table (all 16 parameters)
- §3 corpus coverage by key and function (from actual training data)
- §4 prefix design patterns with templates and recommendations
- §5 temperature/top-p (content from sampling_guide, condensed)
- §6 technical implementation details
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Explains temperature and nucleus sampling mechanics, their interaction,
and provides a task-based cheatsheet with symptom/remedy table and
example CLI invocations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
val loss 1.19 → 0.77, val perplexity 3.29 → 2.15.
Best epoch 20, early stop at epoch 30 (patience=10).
Improvement over previous lr=1e-5 run (best val ppl 2.22).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the model could emit EOS before reaching n_bars because the
EOS-suppression was only applied via the n_bars break, not the grammar
bias. Fixed by masking EOS to -inf in the logit bias while
bars_completed < n_bars.
Added _EosHungryModel fixture and test_generate_bars_overrides_early_eos
to catch this regression class.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
generate_period() now accepts n_bars=N to stop after exactly N complete
bars. bars_completed is seeded from the prefix length so --bars counts
the full output, not just the generated tail.
scripts/generate.py exposes this as --bars (default: None = model decides).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_encode_prefix now handles hold ('.') and no-chord ('NC') tokens
alongside chord symbols, and returns (ids, n_positions) so that
pos_in_bar is tracked correctly regardless of token type.
Fixes ChordParseError when dots were passed in --prefix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
val loss 1.24 → 0.80, val perplexity 3.47 → 2.22.
Best epoch 50 (no early stop); convergence epoch 30.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Moved data/processed/{train,val,holdout}/ → data/processed/mcgill/{train,val,holdout}/
so both corpora have their own namespace under data/processed/.
Updated PRETRAIN_DATA paths in make_colab_zip.py accordingly
(path remap workaround no longer needed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
make_colab_zip.py now accepts --mode pretrain|finetune (default: pretrain).
Finetune mode bundles scripts/train.py + data/processed/user/{train,val}/*.pt
plus an optional --include-checkpoint flag for pretrained.pt.
notebooks/colab_finetune.ipynb covers the full Colab fine-tuning workflow:
upload zip → upload pretrained.pt → verify data → train → inspect → download.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace five narrow patterns (*.pt, *.pkl, train/, val/, holdout/) with
a single data/processed/ rule that also covers data/processed/user/.
All processed tensors are reproducible from committed .chord files.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
scripts/train.py: fix max_seq_len 256→320 (must match pretrained checkpoint);
increase epochs 15→50 and patience 5→10 to give the small corpus enough
gradient steps; reduce warmup 20→10 (was 22% of total steps).
scripts/generate.py: default to prepending the tonic chord when --prefix is
not given; add --no-tonic-anchor to opt out.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
src/generate.py: autoregressive generation with top-p sampling, grammar
masking (ROOT→QUAL→EXT→BASS; EOS only at bar boundary), key transposition,
and optional chord prefix. Partial bars on context truncation are padded
with HOLDs rather than discarded.
scripts/generate.py: CLI wrapping generate_period — accepts mode, key,
time, subdivision, style, function, prefix, temperature, top-p, seed,
tempo; writes .chord and optional MIDI.
src/tokenizer.py: fix docstring vocab size (81→84); normalize redundant
BASS_<note>==root to no slash in _tokens_to_symbol.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-run pre-training results with the corrected 84-token vocabulary and
max_seq_len=320. Previous checkpoint was trained on stale data with BAR
tokens and a corrupted tokenizer.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bar boundaries are now implicit — the detokenizer counts positions per bar
using TIME × SUB, and the generator gates EOS to bar boundaries only.
Removing the deterministic BAR token reduces vocab size from 85 to 84 and
lets the model focus on meaningful predictions.
- src/tokenizer.py: drop BAR from VOCAB (85→84); replace BAR-based
detokenize_to_period with position-counting logic; add write_chord_file;
fix _tokens_to_symbol for add9/m(add9) qualities
- tests/test_tokenizer.py: update vocab-size assertions to 84, structural
token test, remove bar-count test, add test_no_bar_token_in_vocab
- docs/chord_format_spec.md: bump to v2.3; document BAR removal in §5.2,
§5.3, §5.4, §5.5, §5.6, §6.2, and changelog
- CLAUDE.md: remove stale BAR reference, update vocab size to 84
- scripts/pretrain.py: raise max_seq_len 256→320 to cover regenerated
McGill data (mean=83, max=283 tokens with BAR-free tokenizer)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
scripts/make_colab_zip.py packages src/, scripts/pretrain.py,
requirements.txt, and processed .pt files into hamori_colab.zip,
remapping data/processed/{train,val}/ -> data/processed/mcgill/{train,val}/
so pretrain.py finds the data without modification.
notebooks/colab_pretrain.ipynb guides through upload, extraction,
dependency install, training run, report display, and results download.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
README:
- processed/ tree now shows mcgill/ and user/ subdirs
- --style user -> --style H1K0 in quick-start prefix example
- pretrained.report.txt and finetuned.report.txt added to artifact tables
architecture.md (-> v1.1):
- remove stale music21 fallback mention from chord_parser section
- fix ChordDataset: on-demand loading, not eager; remove non-existent
make_dataloader from public interface
- fix train function name: train_model -> train
- update logging description: report goes to .report.txt, not stdout
- note that scripts use max_seq_len=256 (sequences top out at 195 tokens)
requirements.md (-> v1.1):
- FT-12: update from unified script to pretrain.py + train.py pair
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pretrain.py -> checkpoints/pretrained.report.txt
train.py -> checkpoints/finetuned.report.txt
Single-line [report] saved -> <path> printed to stdout instead.
Also fix arrow character incompatible with Windows cp1251 console.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- scripts/run_pretrain.py -> scripts/pretrain.py: pre-trains on McGill
corpus (data/processed/mcgill/), saves checkpoints/pretrained.pt.
- scripts/train.py: rewritten as high-level fine-tune wrapper; loads
pretrained.pt, trains on data/processed/user/, saves finetuned.pt.
Both scripts include timing estimate, loss-curve plot, per-epoch report,
and --skip-training flag.
- README: updated section 7 to reflect new script names and separate
data directories.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- scripts/run_pretrain.py: single-command pre-training runner with
timing estimate, loss-curve plot (matplotlib), and per-epoch report.
Sets max_seq_len=256 (McGill sequences max out at 195 tokens, ~4x
faster attention than the 512 default).
- src/train.py: normalise --output so pretrained.pt and pretrained both
produce pretrained.pt + pretrained.log.csv (not pretrained.pt.log.csv).
Serialize Path fields as strings in checkpoint to satisfy weights_only.
- requirements.txt: drop unused pandas/music21, add mido (pretty_midi dep).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
AdamW + cosine-with-warmup schedule, PAD-ignoring cross-entropy, per-epoch
CSV logging, best-val-loss checkpointing, early stopping (patience=5).
Same script handles both pre-training and fine-tuning via --init-from.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
H1K0