Core

Runner

IRunner

class catalyst.core.runner.IRunner(model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]] = None, engine: catalyst.core.engine.Engine = None)

Bases: catalyst.core.callback.ICallback, catalyst.core.logger.ILogger, abc.ABC

An abstraction that contains all the logic of how to run the experiment, epochs, loaders and batches. Please check examples.

Parameters

• model – Torch model object

• engine – Engine instance

Abstraction, please check out implementations for more details:

• catalyst.runners.runner.Runner

• catalyst.runners.config.ConfigRunner

Note

To learn more about Catalyst Core concepts, please check out

• catalyst.core.runner.IRunner

• catalyst.core.engine.Engine

• catalyst.core.callback.Callback

Note

Please follow the minimal examples sections for use cases.

get_callbacks() → OrderedDict[str, Callback]

Returns the callbacks for the experiment.

get_criterion() → Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module], None]

Returns the criterion for the experiment.

abstract get_engine() → catalyst.core.engine.Engine

Returns the engine for the experiment.

abstract get_loaders() → OrderedDict[str, DataLoader]

Returns the loaders for the experiment.

get_loggers() → Dict[str, catalyst.core.logger.ILogger]

Returns the loggers for the experiment.

abstract get_model() → Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]]

Returns the model for the experiment.

get_optimizer(model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]]) → Union[torch.optim.optimizer.Optimizer, Dict[str, torch.optim.optimizer.Optimizer], None]

Returns the optimizer for the experiment.

get_scheduler(optimizer: Union[torch.optim.optimizer.Optimizer, Dict[str, torch.optim.optimizer.Optimizer]]) → Union[torch.optim.lr_scheduler._LRScheduler, torch.optim.lr_scheduler.ReduceLROnPlateau, Dict[str, Union[torch.optim.lr_scheduler._LRScheduler, torch.optim.lr_scheduler.ReduceLROnPlateau]], None]

Returns the scheduler for the experiment.

abstract handle_batch(batch: Mapping[str, Any]) → None

Inner method to handle specified data batch. Used to make a train/valid/infer step during Experiment run.

Parameters

batch (Mapping[str, Any]) – dictionary with data batches from DataLoader.

property hparams

Returns hyper-parameters for current run.

Example::

>>> runner.hparams
OrderedDict([('optimizer', 'Adam'),
 ('lr', 0.02),
 ('betas', (0.9, 0.999)),
 ('eps', 1e-08),
 ('weight_decay', 0),
 ('amsgrad', False),
 ('train_batch_size', 32)])

Returns

dictionary with hyperparameters

log_hparams(*args, **kwargs) → None

Logs hyperparameters to available loggers.

log_image(*args, **kwargs) → None

Logs image to available loggers.

log_metrics(*args, **kwargs) → None

Logs batch, loader and epoch metrics to available loggers.

property num_epochs

Returns the number of epochs in the experiment.

run() → catalyst.core.runner.IRunner

Runs the experiment.

Returns

self, IRunner instance after the experiment

property seed

Experiment’s seed for reproducibility.

IRunnerError

class catalyst.core.runner.IRunnerError

Bases: Exception

Exception class for all runner errors.

Engine

class catalyst.core.engine.Engine(device_placement: bool = True, split_batches: bool = False, fp16: bool = None, mixed_precision: Union[accelerate.utils.PrecisionType, str] = None, cpu: bool = False, deepspeed_plugin: accelerate.utils.DeepSpeedPlugin = None, fsdp_plugin: accelerate.utils.FullyShardedDataParallelPlugin = None, rng_types: Optional[List[Union[str, accelerate.utils.RNGType]]] = None, log_with: Optional[List[Union[str, accelerate.utils.LoggerType, accelerate.tracking.GeneralTracker]]] = None, logging_dir: Union[str, os.PathLike, None] = None, dispatch_batches: Optional[bool] = None, step_scheduler_with_optimizer: bool = True, kwargs_handlers: Optional[List[accelerate.kwargs_handlers.KwargsHandler]] = None)

Bases: accelerate.accelerator.Accelerator

An abstraction that syncs experiment run with different hardware-specific configurations. - CPU - GPU - DataParallel (deepspeed, torch) - AMP (deepspeed, torch) - DDP (deepspeed, torch) - XLA

Please check out implementations for more details:

• catalyst.engines.torch.CPUEngine

• catalyst.engines.torch.GPUEngine

• catalyst.engines.torch.DataParallelEngine

• catalyst.engines.torch.DistributedDataParallelEngine

• catalyst.engines.torch.DistributedXLAEngine

cleanup()

Cleans DDP variables and processes.

property is_ddp

Boolean flag for distributed type.

mean_reduce_ddp_metrics(metrics: Dict) → Dict

Syncs metrics over world_size in the distributed mode.

setup(local_rank: int, world_size: int)

Initialize DDP variables and processes if required.

Parameters

• local_rank – process rank. Default is -1.

• world_size – number of devices in netwok to expect for train. Default is 1.

spawn(fn: Callable, *args, **kwargs)

Spawns processes with specified fn and args/kwargs.

Parameters

• fn (function) – Function is called as the entrypoint of the spawned process. This function must be defined at the top level of a module so it can be pickled and spawned. This is a requirement imposed by multiprocessing. The function is called as fn(i, *args), where i is the process index and args is the passed through tuple of arguments.

• *args – Arguments passed to spawn method.

• **kwargs – Keyword-arguments passed to spawn method.

Returns

wrapped function (if needed).

Callback

ICallback

class catalyst.core.callback.ICallback

Bases: object

A callable abstraction for deep learning runs.

on_batch_end(runner: IRunner) → None

Event handler for batch end.

on_batch_start(runner: IRunner) → None

Event handler for batch start.

on_epoch_end(runner: IRunner) → None

Event handler for epoch end.

on_epoch_start(runner: IRunner) → None

Event handler for epoch start.

on_exception(runner: IRunner) → None

Event handler for exception case.

on_experiment_end(runner: IRunner) → None

Event handler for experiment end.

on_experiment_start(runner: IRunner) → None

Event handler for experiment start.

on_loader_end(runner: IRunner) → None

Event handler for loader end.

on_loader_start(runner: IRunner) → None

Event handler for loader start.

CallbackOrder

class catalyst.core.callback.CallbackOrder

Bases: enum.IntFlag

Callback usage order during training.

Catalyst executes Callbacks with low CallbackOrder before Callbacks with high CallbackOrder.

Predefined orders:

• Internal (0) - some Catalyst Extras, like PhaseCallbacks (used in GANs).

• Metric (10) - Callbacks with metrics and losses computation.

• MetricAggregation (20) - metrics aggregation callbacks, like sum different losses into one.

• Backward (30) - backward step.

• Optimizer (40) - optimizer step, requires computed metrics for optimization.

• Scheduler (50) - scheduler step, in ReduceLROnPlateau case requires computed validation metrics for optimizer schedule.

• Checkpoint (60) - checkpoint step.

• External (100) - additional callbacks with custom logic.

Nevertheless, you always can create CustomCallback with any order, for example:

>>> class MyCustomCallback(Callback):
>>>     def __init__(self):
>>>         super().__init__(order=13)
>>>     ...
# MyCustomCallback will be executed after all `Metric`-Callbacks
# but before all `MetricAggregation`-Callbacks.

Backward = 30

Checkpoint = 50

External = 100

Internal = 0

Metric = 10

MetricAggregation = 20

Optimizer = 40

Scheduler = 50

backward = 30

checkpoint = 50

external = 100

internal = 0

metric = 10

metric_aggregation = 20

optimizer = 40

scheduler = 50

Callback

class catalyst.core.callback.Callback(order: int)

Bases: catalyst.core.callback.ICallback

An abstraction that lets you customize your experiment run logic.

Parameters

order – flag from CallbackOrder

To give users maximum flexibility and extensibility Catalyst supports callback execution anywhere in the training loop:

-- experiment start
---- epoch start
------ loader start
-------- batch start
---------- batch handler (Runner logic)
-------- batch end
------ loader end
---- epoch end
-- experiment end

exception – if an Exception was raised

Abstraction, please check out implementations for more details:

• catalyst.callbacks.criterion.CriterionCallback

• catalyst.callbacks.optimizer.OptimizerCallback

• catalyst.callbacks.scheduler.SchedulerCallback

• catalyst.callbacks.checkpoint.CheckpointCallback

Note

To learn more about Catalyst Core concepts, please check out

• catalyst.core.runner.IRunner

• catalyst.core.engine.Engine

• catalyst.core.callback.Callback

CallbackWrapper

class catalyst.core.callback.CallbackWrapper(base_callback: catalyst.core.callback.Callback, enable_callback: bool = True)

Bases: catalyst.core.callback.Callback

Enable/disable callback execution.

Parameters

• base_callback – callback to wrap

• enable_callback – indicator to enable/disable callback, if True then callback will be enabled, default True

on_exception(runner: IRunner) → None

Event handler for exception case.

ILogger

class catalyst.core.logger.ILogger(log_batch_metrics: bool, log_epoch_metrics: bool)

Bases: object

An abstraction that syncs experiment run with monitoring tools.

Parameters

• log_batch_metrics – boolean flag to log batch metrics.

• log_epoch_metrics – boolean flag to log epoch metrics.

Abstraction, please check out implementations for more details:

• catalyst.loggers.console.ConsoleLogger

• catalyst.loggers.mlflow.MLflowLogger

• catalyst.loggers.neptune.NeptuneLogger

• catalyst.loggers.tensorboard.TensorboardLogger

close_log() → None

Closes the logger.

flush_log() → None

Flushes the logger.

log_artifact(tag: str, runner: IRunner, artifact: object = None, path_to_artifact: str = None, scope: str = None) → None

Logs artifact (arbitrary file like audio, video, etc) to the logger.

property log_batch_metrics

Boolean flag to log batch metrics.

Returns

boolean flag to log batch metrics.

Return type

bool

property log_epoch_metrics

Boolean flag to log epoch metrics.

Returns

boolean flag to log epoch metrics.

Return type

bool

log_hparams(hparams: Dict, runner: IRunner = None) → None

Logs hyperparameters to the logger.

log_image(tag: str, image: numpy.ndarray, runner: IRunner, scope: str = None) → None

Logs image to the logger.

log_metrics(metrics: Dict[str, float], scope: str, runner: IRunner) → None

Logs metrics to the logger.

property logger

DAR401

Returns: # noqa: DAR201, DAR202

Any: internal logger/experiment/etc. from the monitoring system.

Type

Internal logger/experiment/etc. from the monitoring system. # noqa