fix: clone grammar bias per step in generate_period

_grammar_bias returned a shared module-level singleton that the loop
mutated in place (EOS block + repetition penalty). The penalty thus
accumulated across positions within a call and persisted across calls,
collapsing output to HOLD/NC until process restart. Clone the bias each
step so edits stay local. Add regression tests guarding the invariant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-04 21:14:04 +03:00
parent d1af7bceb8
commit dc037b0895
2 changed files with 54 additions and 1 deletions
+4 -1
View File
@@ -322,7 +322,10 @@ def generate_period(
break
inp = torch.tensor([ids], dtype=torch.long, device=device)
logits = model(inp)[0, -1] # [vocab_size]
bias = _grammar_bias(last_id, pos_in_bar, positions_per_bar)
# _grammar_bias returns a shared module-level singleton; clone before
# any in-place edit so EOS-blocking and the repetition penalty never
# leak across positions or across calls.
bias = _grammar_bias(last_id, pos_in_bar, positions_per_bar).clone()
if n_bars is not None and bars_completed < n_bars:
bias[_EOS] = float("-inf") # don't let the model stop early