pytorch_pfn_extras.writing.ProcessWriter#
- class pytorch_pfn_extras.writing.ProcessWriter(savefun=<function save>, fs=None, out_dir='', **kwds)#
Bases:
StandardWriter
[Process
]Snapshot writer that uses a separate process.
This class creates a new process that invokes the actual saving function.
Note
Forking a new process from a MPI process might be danger. Consider using
ThreadWriter
instead ofProcessWriter
if you are using MPI.Methods
__init__
([savefun, fs, out_dir])create_worker
(filename, out_dir, target, *)Creates a worker for the snapshot.
finalize
()Finalizes the writer.
initialize
(out_dir)save
(filename, out_dir, target, savefun, ...)- __init__(savefun=<function save>, fs=None, out_dir='', **kwds)#
- Parameters:
savefun (Callable[[...], None]) –
fs (Optional[Any]) –
out_dir (str) –
kwds (Any) –
- Return type:
None
- create_worker(filename, out_dir, target, *, savefun=None, append=False, **savefun_kwargs)#
Creates a worker for the snapshot.
This method creates a thread or a process to take a snapshot. The created worker must have
start()
andjoin()
methods. If the worker has anexitcode
attribute (e.g.,multiprocessing.Process
), the value will be tested.- Parameters:
filename (str) –
out_dir (str) –
target (Union[Sequence[Any], Mapping[str, Any]]) –
savefun (Optional[Callable[[...], None]]) –
append (bool) –
savefun_kwargs (Any) –
- Return type:
Process