Shortcuts

Core

Runner

IRunner

class catalyst.core.runner.IRunner(model: Optional[Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]]] = None, engine: Optional[catalyst.core.engine.Engine] = None)[source]

Bases: catalyst.core.callback.ICallback, catalyst.core.logger.ILogger, abc.ABC

An abstraction that contains all the logic of how to run the experiment, epochs, loaders and batches. Please check examples.

Parameters
  • model – Torch model object

  • engine – Engine instance

Abstraction, please check out implementations for more details:

Note

To learn more about Catalyst Core concepts, please check out

Note

Please follow the minimal examples sections for use cases.

get_callbacks() OrderedDict[str, Callback][source]

Returns the callbacks for the experiment.

get_criterion() Optional[Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]]][source]

Returns the criterion for the experiment.

abstract get_engine() catalyst.core.engine.Engine[source]

Returns the engine for the experiment.

abstract get_loaders() OrderedDict[str, DataLoader][source]

Returns the loaders for the experiment.

get_loggers() Dict[str, catalyst.core.logger.ILogger][source]

Returns the loggers for the experiment.

abstract get_model() Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]][source]

Returns the model for the experiment.

get_optimizer(model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]]) Optional[Union[torch.optim.optimizer.Optimizer, Dict[str, torch.optim.optimizer.Optimizer]]][source]

Returns the optimizer for the experiment.

get_scheduler(optimizer: Union[torch.optim.optimizer.Optimizer, Dict[str, torch.optim.optimizer.Optimizer]]) Optional[Union[torch.optim.lr_scheduler._LRScheduler, torch.optim.lr_scheduler.ReduceLROnPlateau, Dict[str, Union[torch.optim.lr_scheduler._LRScheduler, torch.optim.lr_scheduler.ReduceLROnPlateau]]]][source]

Returns the scheduler for the experiment.

abstract handle_batch(batch: Mapping[str, Any]) None[source]

Inner method to handle specified data batch. Used to make a train/valid/infer step during Experiment run.

Parameters

batch (Mapping[str, Any]) – dictionary with data batches from DataLoader.

property hparams: collections.OrderedDict

Returns hyper-parameters for current run.

Example::
>>> runner.hparams
OrderedDict([('optimizer', 'Adam'),
 ('lr', 0.02),
 ('betas', (0.9, 0.999)),
 ('eps', 1e-08),
 ('weight_decay', 0),
 ('amsgrad', False),
 ('train_batch_size', 32)])
Returns

dictionary with hyperparameters

log_hparams(*args, **kwargs) None[source]

Logs hyperparameters to available loggers.

log_image(*args, **kwargs) None[source]

Logs image to available loggers.

log_metrics(*args, **kwargs) None[source]

Logs batch, loader and epoch metrics to available loggers.

property num_epochs: int

Returns the number of epochs in the experiment.

run() catalyst.core.runner.IRunner[source]

Runs the experiment.

Returns

self, IRunner instance after the experiment

property seed: int

Experiment’s seed for reproducibility.

IRunnerError

class catalyst.core.runner.IRunnerError[source]

Bases: Exception

Exception class for all runner errors.

Engine

class catalyst.core.engine.Engine(device_placement: bool = True, split_batches: bool = False, fp16: Optional[bool] = None, mixed_precision: Optional[str] = None, cpu: bool = False, deepspeed_plugin: Optional[accelerate.utils.DeepSpeedPlugin] = None, rng_types: Optional[List[Union[str, accelerate.utils.RNGType]]] = None, dispatch_batches: Optional[bool] = None, kwargs_handlers: Optional[List[accelerate.kwargs_handlers.KwargsHandler]] = None)[source]

Bases: accelerate.accelerator.Accelerator

An abstraction that syncs experiment run with different hardware-specific configurations. - CPU - GPU - DataParallel (deepspeed, torch) - AMP (deepspeed, torch) - DDP (deepspeed, torch) - XLA

Please check out implementations for more details:
cleanup()[source]

Cleans DDP variables and processes.

property is_ddp

Boolean flag for distributed type.

mean_reduce_ddp_metrics(metrics: Dict) Dict[source]

Syncs metrics over world_size in the distributed mode.

setup(local_rank: int, world_size: int)[source]

Initialize DDP variables and processes if required.

Parameters
  • local_rank – process rank. Default is -1.

  • world_size – number of devices in netwok to expect for train. Default is 1.

spawn(fn: Callable, *args, **kwargs)[source]

Spawns processes with specified fn and args/kwargs.

Parameters
  • fn (function) – Function is called as the entrypoint of the spawned process. This function must be defined at the top level of a module so it can be pickled and spawned. This is a requirement imposed by multiprocessing. The function is called as fn(i, *args), where i is the process index and args is the passed through tuple of arguments.

  • *args – Arguments passed to spawn method.

  • **kwargs – Keyword-arguments passed to spawn method.

Returns

wrapped function (if needed).

Callback

ICallback

class catalyst.core.callback.ICallback[source]

Bases: object

A callable abstraction for deep learning runs.

on_batch_end(runner: IRunner) None[source]

Event handler for batch end.

on_batch_start(runner: IRunner) None[source]

Event handler for batch start.

on_epoch_end(runner: IRunner) None[source]

Event handler for epoch end.

on_epoch_start(runner: IRunner) None[source]

Event handler for epoch start.

on_exception(runner: IRunner) None[source]

Event handler for exception case.

on_experiment_end(runner: IRunner) None[source]

Event handler for experiment end.

on_experiment_start(runner: IRunner) None[source]

Event handler for experiment start.

on_loader_end(runner: IRunner) None[source]

Event handler for loader end.

on_loader_start(runner: IRunner) None[source]

Event handler for loader start.

CallbackOrder

class catalyst.core.callback.CallbackOrder(value)[source]

Bases: enum.IntFlag

Callback usage order during training.

Catalyst executes Callbacks with low CallbackOrder before Callbacks with high CallbackOrder.

Predefined orders:

  • Internal (0) - some Catalyst Extras, like PhaseCallbacks (used in GANs).

  • Metric (10) - Callbacks with metrics and losses computation.

  • MetricAggregation (20) - metrics aggregation callbacks, like sum different losses into one.

  • Backward (30) - backward step.

  • Optimizer (40) - optimizer step, requires computed metrics for optimization.

  • Scheduler (50) - scheduler step, in ReduceLROnPlateau case requires computed validation metrics for optimizer schedule.

  • Checkpoint (60) - checkpoint step.

  • External (100) - additional callbacks with custom logic.

Nevertheless, you always can create CustomCallback with any order, for example:

>>> class MyCustomCallback(Callback):
>>>     def __init__(self):
>>>         super().__init__(order=13)
>>>     ...
# MyCustomCallback will be executed after all `Metric`-Callbacks
# but before all `MetricAggregation`-Callbacks.
Backward = 30
Checkpoint = 50
External = 100
Internal = 0
Metric = 10
MetricAggregation = 20
Optimizer = 40
Scheduler = 50
backward = 30
checkpoint = 50
external = 100
internal = 0
metric = 10
metric_aggregation = 20
optimizer = 40
scheduler = 50

Callback

class catalyst.core.callback.Callback(order: int)[source]

Bases: catalyst.core.callback.ICallback

An abstraction that lets you customize your experiment run logic.

Parameters

order – flag from CallbackOrder

To give users maximum flexibility and extensibility Catalyst supports callback execution anywhere in the training loop:

-- experiment start
---- epoch start
------ loader start
-------- batch start
---------- batch handler (Runner logic)
-------- batch end
------ loader end
---- epoch end
-- experiment end

exception – if an Exception was raised

Abstraction, please check out implementations for more details:

Note

To learn more about Catalyst Core concepts, please check out

__init__(order: int)[source]

Init.

CallbackWrapper

class catalyst.core.callback.CallbackWrapper(base_callback: catalyst.core.callback.Callback, enable_callback: bool = True)[source]

Bases: catalyst.core.callback.Callback

Enable/disable callback execution.

Parameters
  • base_callback – callback to wrap

  • enable_callback – indicator to enable/disable callback, if True then callback will be enabled, default True

__init__(base_callback: catalyst.core.callback.Callback, enable_callback: bool = True)[source]

Init.

on_exception(runner: IRunner) None[source]

Event handler for exception case.

ILogger

class catalyst.core.logger.ILogger(log_batch_metrics: bool, log_epoch_metrics: bool)[source]

Bases: object

An abstraction that syncs experiment run with monitoring tools.

Parameters
  • log_batch_metrics – boolean flag to log batch metrics.

  • log_epoch_metrics – boolean flag to log epoch metrics.

Abstraction, please check out implementations for more details:

close_log() None[source]

Closes the logger.

flush_log() None[source]

Flushes the logger.

log_artifact(tag: str, runner: IRunner, artifact: object = None, path_to_artifact: str = None, scope: str = None) None[source]

Logs artifact (arbitrary file like audio, video, etc) to the logger.

property log_batch_metrics: bool

Boolean flag to log batch metrics.

Returns

boolean flag to log batch metrics.

Return type

bool

property log_epoch_metrics: bool

Boolean flag to log epoch metrics.

Returns

boolean flag to log epoch metrics.

Return type

bool

log_hparams(hparams: Dict, runner: IRunner = None) None[source]

Logs hyperparameters to the logger.

log_image(tag: str, image: numpy.ndarray, runner: IRunner, scope: str = None) None[source]

Logs image to the logger.

log_metrics(metrics: Dict[str, float], scope: str, runner: IRunner) None[source]

Logs metrics to the logger.

property logger: Any

DAR401

Returns: # noqa: DAR201, DAR202

Any: internal logger/experiment/etc. from the monitoring system.

Type

Internal logger/experiment/etc. from the monitoring system. # noqa