pytorch_pfn_extras.training.extensions.snapshot_writers.ProcessWriter#

class pytorch_pfn_extras.training.extensions.snapshot_writers.ProcessWriter(savefun=<function save>, fs=None, out_dir='', **kwds)#

Bases: StandardWriter[Process]

Snapshot writer that uses a separate process.

This class creates a new process that invokes the actual saving function.

Note

Forking a new process from a MPI process might be danger. Consider using ThreadWriter instead of ProcessWriter if you are using MPI.

See also

Methods

__init__([savefun, fs, out_dir])

create_worker(filename, out_dir, target, *)

Creates a worker for the snapshot.

finalize()

Finalizes the writer.

initialize(out_dir)

save(filename, out_dir, target, savefun, ...)

Parameters:

  • savefun (Callable[[...], None]) –

  • fs (Any) –

  • out_dir (str) –

  • kwds (Any) –

__init__(savefun=<function save>, fs=None, out_dir='', **kwds)#
Parameters:

  • savefun (Callable[[...], None]) –

  • fs (Any) –

  • out_dir (str) –

  • kwds (Any) –

Return type:

None

create_worker(filename, out_dir, target, *, savefun=None, append=False, **savefun_kwargs)#

Creates a worker for the snapshot.

This method creates a thread or a process to take a snapshot. The created worker must have start() and join() methods. If the worker has an exitcode attribute (e.g., multiprocessing.Process), the value will be tested.

Parameters:

  • filename (str) –

  • out_dir (str) –

  • target (Union[Sequence[Any], Mapping[str, Any]]) –

  • savefun (Optional[Callable[[...], None]]) –

  • append (bool) –

  • savefun_kwargs (Any) –

Return type:

Process