tinker_cookbook.rl.InitialObservationOverflow
class tinker_cookbook.rl.InitialObservationOverflow()
Sentinel returned by Env.initial_observation instead of an
(observation, stop_condition) pair when the initial prompt already
exceeds the environment's token budget.
The rollout loop (do_single_rollout) converts this into an immediate
graceful stop: a trajectory with a single synthetic transition (empty
observation, empty action, episode_done=True) carrying reward,
metrics, and logs, with Trajectory.stop_reason mirrored from
the stop/<reason> metric (StopReason.MAX_TOKENS for prompt
overflow). The synthetic transition contributes no training tokens
(trajectory_to_data emits no datum for it) but its reward counts
toward the trajectory's total, so group reward centering sees the
overflowed member.
Returning this sentinel (rather than raising) keeps one oversized prompt
from failing the whole group under the FailFast rollout strategy.
Fields:
- reward (float)
- metrics (Metrics, default:
field(default_factory=dict)) - logs (Logs, default:
field(default_factory=dict))