pytorch_pfn_extras.training.extensions.LogReport#

class pytorch_pfn_extras.training.extensions.LogReport(keys=None, trigger=(1, 'epoch'), postprocess=None, filename=None, append=False, format=None, **kwargs)#

Bases: Extension

An extension to output the accumulated results to a log file.

This extension accumulates the observations of the manager to DictSummary at a regular interval specified by a supplied trigger, and writes them into a log file in JSON format.

There are two triggers to handle this extension. One is the trigger to invoke this extension, which is used to handle the timing of accumulating the results. It is set to 1, 'iteration' by default. The other is the trigger to determine when to emit the result. When this trigger returns True, this extension appends the summary of accumulated values to the list of past summaries, and writes the list to the log file. Then, this extension makes a new fresh summary object which is used until the next time that the trigger fires.

It also adds some entries to each result dictionary.

  • 'epoch' and 'iteration' are the epoch and iteration counts at the output, respectively.

  • 'elapsed_time' is the elapsed time in seconds since the training begins. The value is taken from ExtensionsManager.elapsed_time.

Parameters:

  • keys (iterable of strs) – Keys of values to accumulate. If this is None, all the values are accumulated and output to the log file.

  • trigger (Optional[Union[Trigger, Callable[[ExtensionsManagerProtocol], bool], Tuple[float, str]]]) – Trigger that decides when to aggregate the result and output the values. This is distinct from the trigger of this extension itself. If it is a tuple in the form <int>, 'epoch' or <int>, 'iteration', it is passed to IntervalTrigger.

  • postprocess (Optional[Callable[[Mapping[str, Any]], None]]) – Callback to postprocess the result dictionaries. Each result dictionary is passed to this callback on the output. This callback can modify the result dictionaries, which are used to output to the log file.

  • filename (str) – Name of the log file under the output directory. It can be a format string: the last result dictionary is passed for the formatting. For example, users can use ‘{iteration}’ to separate the log files for different iterations. (default: 'log')

  • append (bool, optional) – If the file is JSON Lines or YAML, contents will be appended instead of rewritting the file every call.

  • format (str, optional) – accepted values are ‘json’, ‘json-lines’ and ‘yaml’.

  • writer (writer object, optional) – must be callable. object to dump the log to. If specified, it needs to have a correct savefun defined. The writer can override the save location in the pytorch_pfn_extras.training.ExtensionsManager object

  • kwargs (Any) –

Note

Enabling append=True reduces size of snapshots (and thus reduces the time needed to take snapshots). Note that extensions relying on the logs from past iterations may behave differently; for example, when resuming from a snapshot, PrintReport will not show logs of iterations already done.

Methods

__init__([keys, trigger, postprocess, ...])

finalize(manager)

Finalizes the extension.

initialize(manager)

Initializes up the manager state.

load_state_dict(to_load)

on_error(manager, exc, tb)

Handles the error raised during training before finalization.

state_dict()

Serializes the extension state.

to_dataframe()

Attributes

default_name

Default name of the extension.

is_async

log

The current list of observation dictionaries.

name

needs_model_state

priority

trigger

__call__(manager)#

Invokes the extension.

Implementations should override this operator. This method is called at iterations which the corresponding trigger accepts.

Parameters:

manager (ExtensionsManager) – Manager object to call this operator.

Return type:

None

__init__(keys=None, trigger=(1, 'epoch'), postprocess=None, filename=None, append=False, format=None, **kwargs)#
Parameters:

  • keys (Optional[Iterable[str]]) –

  • trigger (Optional[Union[Trigger, Callable[[ExtensionsManagerProtocol], bool], Tuple[float, str]]]) –

  • postprocess (Optional[Callable[[Mapping[str, Any]], None]]) –

  • filename (Optional[str]) –

  • append (bool) –

  • format (Optional[str]) –

  • kwargs (Any) –

finalize(manager)#

Finalizes the extension.

This method is called at the end of the training loop.

Parameters:

manager (ExtensionsManagerProtocol) –

Return type:

None

load_state_dict(to_load)#
Parameters:

to_load (Dict[str, Any]) –

Return type:

None

property log: List[Mapping[str, Any]]#

The current list of observation dictionaries.

state_dict()#

Serializes the extension state.

It is called when a manager that owns this extension is serialized. It serializes nothing by default.

Return type:

Dict[str, Any]

to_dataframe()#
Return type:

pandas.DataFrame