pytorch_pfn_extras.training.extensions.lr_scheduler.LRScheduler#
- class pytorch_pfn_extras.training.extensions.lr_scheduler.LRScheduler(scheduler, *, stepper=<function _default_stepper>, trigger=(1, 'epoch'), wait_for_first_optimizer_step=False, is_async=True)#
Bases:
Extension
Trainer extension to adjust the learning rate using PyTorch’s learning rate scheduler.
This extension calls step() method of the given LR scheduler. (torch.option.lr_scheduler.*). When using ReduceLROnPlateau, the latest reported val/loss value will be used. This behavior can be customized by passing a custom stepper function.
- Parameters:
scheduler (_LRScheduler or ReduceLROnPlateau) – Any instance of torch.optim.lr_scheduler.*.
stepper (callable) – Function that performs the step on the scheduler.
trigger (Optional[Union[Trigger, Callable[[ExtensionsManagerProtocol], bool], Tuple[float, str]]]) – Frequency to call this extension.
wait_for_first_optimizer_step (bool) – Wait until optimizer.step() is called before invoking scheduler.step(). This can address the issue where optimizer.step() is not called from the first iteration when using GradScaler.
is_async (bool) –
Methods
__init__
(scheduler, *[, stepper, trigger, ...])finalize
(manager)Finalizes the extension.
initialize
(manager)Initializes up the manager state.
load_state_dict
(state)on_error
(manager, exc, tb)Handles the error raised during training before finalization.
Serializes the extension state.
step_by_value
(key)Attributes
default_name
Default name of the extension.
is_async
name
needs_model_state
priority
trigger
- __call__(manager)#
Invokes the extension.
Implementations should override this operator. This method is called at iterations which the corresponding trigger accepts.
- Parameters:
manager (ExtensionsManager) – Manager object to call this operator.
- Return type:
None
- __init__(scheduler, *, stepper=<function _default_stepper>, trigger=(1, 'epoch'), wait_for_first_optimizer_step=False, is_async=True)#
- Parameters:
scheduler (Any) –
stepper (Any) –
trigger (Optional[Union[Trigger, Callable[[ExtensionsManagerProtocol], bool], Tuple[float, str]]]) –
wait_for_first_optimizer_step (bool) –
is_async (bool) –
- Return type:
None
- load_state_dict(state)#
- Parameters:
state (Dict[str, Any]) –
- Return type:
None
- state_dict()#
Serializes the extension state.
It is called when a manager that owns this extension is serialized. It serializes nothing by default.
- Return type:
Dict[str, Any]
- static step_by_value(key)#
- Parameters:
key (Optional[str]) –
- Return type:
Any