fix: clone grammar bias per step in generate_period
_grammar_bias returned a shared module-level singleton that the loop mutated in place (EOS block + repetition penalty). The penalty thus accumulated across positions within a call and persisted across calls, collapsing output to HOLD/NC until process restart. Clone the bias each step so edits stay local. Add regression tests guarding the invariant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+4
-1
@@ -322,7 +322,10 @@ def generate_period(
|
||||
break
|
||||
inp = torch.tensor([ids], dtype=torch.long, device=device)
|
||||
logits = model(inp)[0, -1] # [vocab_size]
|
||||
bias = _grammar_bias(last_id, pos_in_bar, positions_per_bar)
|
||||
# _grammar_bias returns a shared module-level singleton; clone before
|
||||
# any in-place edit so EOS-blocking and the repetition penalty never
|
||||
# leak across positions or across calls.
|
||||
bias = _grammar_bias(last_id, pos_in_bar, positions_per_bar).clone()
|
||||
if n_bars is not None and bars_completed < n_bars:
|
||||
bias[_EOS] = float("-inf") # don't let the model stop early
|
||||
|
||||
|
||||
Reference in New Issue
Block a user