Copy as Markdown View as Markdown

tinker_cookbook.hyperparam_utils.get_lr

Generated from tinker-cookbook at commit 8486b8d. Source links point at that snapshot.

tinker_cookbook.hyperparam_utils.get_lr(model_name, is_lora)

Get a recommended learning rate for the given model.

Applies model-family-specific scaling based on hidden size. Only Llama and Qwen families have calibrated formulas; other models raise NotImplementedError.

Parameters:

• model_name (str) – HuggingFace model identifier.
• is_lora (bool) – If True, scale the base LR by the LoRA multiplier (10x).

Returns: The recommended learning rate.