pytorch_pfn_extras.writing.StandardWriter#

class pytorch_pfn_extras.writing.StandardWriter(savefun=<function save>, fs=None, out_dir='', **kwds)#

Bases: Writer, Generic[_Worker]

Base class of snapshot writers which use thread or process.

This class creates a new thread or a process every time when __call__ is invoked.

Parameters:

  • savefun – Callable object. It takes three arguments: the output file path, the serialized dictionary object, and the optional keyword arguments.

  • fs – FileSystem abstracting interface to implement all the operations. optional, defaults to None

  • out_dir – str. Specifies the directory this writer will use. It takes precedence over the one specified in __call__ optional, defaults to ''

  • kwds – Keyword arguments for the savefun.

See also

Methods

__init__([savefun, fs, out_dir])

create_worker(filename, out_dir, target, *)

Creates a worker for the snapshot.

finalize()

Finalizes the writer.

initialize(out_dir)

save(filename, out_dir, target, savefun, ...)

__call__(filename, out_dir, target, *, savefun=None, append=False)#

Does the actual writing to the file.

This method is invoked by a Snapshot object every time it takes a snapshot.

Parameters:

  • filename (str) – Name of the file into which the serialized target is saved. It is a concrete file name, i.e. not a pre-formatted template string.

  • out_dir (str) –

    Output directory. Corresponds to :py:attr:`ExtensionsManager.out

    <pytorch_pfn_extras.training.ExtensionsManager.out>`.

  • target (dict) – Serialized object which will be saved.

  • savefun (callable) – A callable that accepts a two positional arguments (an object to be serialized, file path) like torch.save.

  • append (bool) – Mode used to open the file. True to use the append mode, False to use the write mode (truncates the file if it already exists).

Return type:

None

__init__(savefun=<function save>, fs=None, out_dir='', **kwds)#
Parameters:

  • savefun (Callable[[...], None]) –

  • fs (Optional[Any]) –

  • out_dir (str) –

  • kwds (Any) –

Return type:

None

create_worker(filename, out_dir, target, *, savefun=None, append=False, **savefun_kwargs)#

Creates a worker for the snapshot.

This method creates a thread or a process to take a snapshot. The created worker must have start() and join() methods. If the worker has an exitcode attribute (e.g., multiprocessing.Process), the value will be tested.

Parameters:

  • filename (str) –

  • out_dir (str) –

  • target (Union[Sequence[Any], Mapping[str, Any]]) –

  • savefun (Optional[Callable[[...], None]]) –

  • append (bool) –

  • savefun_kwargs (Any) –

Return type:

_Worker

finalize()#

Finalizes the writer.

Calling this method on already-finalized Writer does nothing.

Return type:

None