Core

Core

Experiment

class catalyst.core.experiment._Experiment

Bases: abc.ABC

An abstraction that contains information about the experiment – a model, a criterion, an optimizer, a scheduler, and their hyperparameters. It also contains information about the data and transformations used. In general, the Experiment knows what you would like to run.

Note

To learn more about Catalyst Core concepts, please check out

• catalyst.core.experiment._Experiment

• catalyst.core.runner._Runner

• catalyst.core.callback.Callback

Abstraction, please check out the implementations:

• catalyst.dl.experiment.base.BaseExperiment

• catalyst.dl.experiment.config.ConfigExperiment

• catalyst.dl.experiment.supervised.SupervisedExperiment

abstract property distributed_params

Dictionary with the parameters for distributed and half-precision training.

Used in catalyst.utils.distributed.process_components to setup Nvidia Apex or PyTorch distributed.

Example:

>>> experiment.distributed_params
{"opt_level": "O1", "syncbn": True}  # Apex variant

abstract get_callbacks(stage: str) → OrderedDict[str, Callback]

Returns callbacks for a given stage.

Note

To learn more about Catalyst Callbacks mechanism, please follow catalyst.core.callback.Callback documentation.

Note

We need ordered dictionary to guarantee the correct dataflow and order of metrics optimization. For example, to compute loss before optimization, or to compute all the metrics before logging :)

Parameters

stage (str) – stage name of interest like “pretrain” / “train” / “finetune” / etc

Returns

Ordered dictionary with callbacks for current stage.

Return type

OrderedDict[str, Callback]

Note

To learn more about Catalyst Core concepts, please check out

• catalyst.core.experiment._Experiment

• catalyst.core.runner._Runner

• catalyst.core.runner.State

• catalyst.core.callback.Callback

abstract get_criterion(stage: str) → torch.nn.modules.module.Module

Returns the criterion for a given stage.

Example:

# for typical classification task
>>> experiment.get_criterion(stage="training")
nn.CrossEntropyLoss()

Parameters

stage (str) – stage name of interest like “pretrain” / “train” / “finetune” / etc

Returns

criterion for a given stage.

Return type

Criterion

get_datasets(stage: str, epoch: int = None, **kwargs) → OrderedDict[str, Dataset]

Returns the datasets for a given stage and epoch.

Note

For Deep Learning cases you have the same dataset during whole stage.

For Reinforcement Learning it common to change the dataset (experiment) every training epoch.

Parameters

• stage (str) – stage name of interest, like “pretrain” / “train” / “finetune” / etc

• epoch (int) – epoch index

• **kwargs (dict) – additional parameters to use during dataset creation

Returns

Ordered dictionary

with datasets for current stage and epoch.

Return type

OrderedDict[str, Dataset]

Note

We need ordered dictionary to guarantee the correct dataflow and order of our training datasets. For example, to run through train data before validation one :)

Example:

>>> experiment.get_datasets(
>>>     stage="training",
>>>     in_csv_train="path/to/train/csv",
>>>     in_csv_valid="path/to/valid/csv",
>>> )
OrderedDict({
    "train": CsvDataset(in_csv=in_csv_train, ...),
    "valid": CsvDataset(in_csv=in_csv_valid, ...),
})

get_experiment_components(stage: str, model: torch.nn.modules.module.Module = None) → Tuple[torch.nn.modules.module.Module, torch.nn.modules.module.Module, torch.optim.optimizer.Optimizer, torch.optim.lr_scheduler._LRScheduler]

Returns the tuple containing criterion, optimizer and scheduler by giving model and stage.

Aggregation method, based on,

• catalyst.core.experiment._Experiment.get_model

• catalyst.core.experiment._Experiment.get_criterion

• catalyst.core.experiment._Experiment.get_optimizer

• catalyst.core.experiment._Experiment.get_scheduler

Parameters

• stage (str) – stage name of interest, like “pretrain” / “train” / “finetune” / etc

• model (Model) – model to optimize with stage optimizer

Returns

model, criterion, optimizer, scheduler

for a given stage and model

Return type

tuple

abstract get_loaders(stage: str, epoch: int = None) → OrderedDict[str, DataLoader]

Returns the loaders for a given stage.

Note

Wrapper for catalyst.core.experiment._Experiment.get_datasets. For most of your experiments you need to rewrite get_datasets method only.

Parameters

• stage (str) – stage name of interest, like “pretrain” / “train” / “finetune” / etc

• epoch (int) – epoch index

• **kwargs (dict) – additional parameters to use during dataset creation

Returns

Ordered dictionary

with loaders for current stage and epoch.

Return type

OrderedDict[str, DataLoader]

abstract get_model(stage: str) → torch.nn.modules.module.Module

Returns the model for a given stage.

Example:

# suppose we have typical MNIST model, like
# nn.Sequential(nn.Linear(28*28, 128), nn.Linear(128, 10))
>>> experiment.get_model(stage="training")
Sequential(
  (0): Linear(in_features=784, out_features=128, bias=True)
  (1): Linear(in_features=128, out_features=10, bias=True)
)

Parameters

stage (str) – stage name of interest like “pretrain” / “train” / “finetune” / etc

Returns

model for a given stage.

Return type

Model

abstract get_optimizer(stage: str, model: torch.nn.modules.module.Module) → torch.optim.optimizer.Optimizer

Returns the optimizer for a given stage and model.

Example:

>>> experiment.get_optimizer(stage="training", model=model)
torch.optim.Adam(model.parameters())

Parameters

• stage (str) – stage name of interest like “pretrain” / “train” / “finetune” / etc

• model (Model) – model to optimize with stage optimizer

Returns

optimizer for a given stage and model.

Return type

Optimizer

abstract get_scheduler(stage: str, optimizer: torch.optim.optimizer.Optimizer) → torch.optim.lr_scheduler._LRScheduler

Returns the scheduler for a given stage and optimizer.

Example::

>>> experiment.get_scheduler(stage="training", optimizer=optimizer)
torch.optim.lr_scheduler.StepLR(optimizer)

Parameters

• stage (str) – stage name of interest like “pretrain” / “train” / “finetune” / etc

• optimizer (Optimizer) – optimizer to schedule with stage scheduler

Returns

scheduler for a given stage and optimizer.

Return type

Scheduler

abstract get_stage_params(stage: str) → Mapping[str, Any]

Returns State parameters for a given stage.

To learn more about State, please follow catalyst.core.runner.State documentation.

Example:

>>> experiment.get_stage_params(stage="training")
{
    "logdir": "./logs/training",
    "num_epochs": 42,
    "valid_loader": "valid",
    "main_metric": "loss",
    "minimize_metric": True,
    "checkpoint_data": {
        "comment": "break the cycle - use the Catalyst"
    }
}

Parameters

stage (str) – stage name of interest like “pretrain” / “train” / “finetune” / etc

Returns

State parameters for a given stage.

Return type

dict

get_transforms(stage: str = None, dataset: str = None)

Returns the data transforms for a given stage and dataset.

Parameters

• stage (str) – stage name of interest, like “pretrain” / “train” / “finetune” / etc

• dataset (str) – dataset name of interest, like “train” / “valid” / “infer”

Note

For datasets/loaders nameing please follow catalyst.core.runner.State documentation.

Returns

Data transformations to use for specified dataset.

abstract property initial_seed

Experiment’s initial seed, used to setup global seed at the beginning of each stage. Additionally, Catalyst Runner setups experiment.initial_seed + runner.global_epoch + 1 as global seed each epoch. Used for experiment reproducibility.

Example:

>>> experiment.initial_seed
42

abstract property logdir

Path to the directory where the experiment logs would be saved.

Example:

>>> experiment.logdir
./path/to/my/experiment/logs

abstract property stages

Experiment’s stage names.

Example:

>>> experiment.stages
["pretraining", "training", "finetuning"]

Note

To understand stages concept, please follow Catalyst documentation, for example, catalyst.core.callback.Callback

Runner

class catalyst.core.runner._Runner(model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]] = None, device: Union[str, torch.device] = None)

Bases: abc.ABC, catalyst.core.legacy._RunnerLegacy, catalyst.tools.frozen_class.FrozenClass

An abstraction that knows how to run an experiment. It contains all the logic of how to run the experiment, stages, epoch and batches.

Note

To learn more about Catalyst Core concepts, please check out

• catalyst.core.experiment._Experiment

• catalyst.core.runner._Runner

• catalyst.core.callback.Callback

Abstraction, please check out the implementations:

• catalyst.dl.runner.runner.Runner

• catalyst.dl.runner.supervised.SupervisedRunner

Runner also contains full information about experiment runner.

Runner section

runner.model - an instance of torch.nn.Module class, (should implement forward method); for example,

runner.model = torch.nn.Linear(10, 10)

runner.device - an instance of torch.device (CPU, GPU, TPU); for example,

runner.device = torch.device("cpu")

Experiment section

runner.criterion - an instance of torch.nn.Module class or torch.nn.modules.loss._Loss (should implement forward method); for example,

runner.criterion = torch.nn.CrossEntropyLoss()

runner.optimizer - an instance of torch.optim.optimizer.Optimizer (should implement step method); for example,

runner.optimizer = torch.optim.Adam()

runner.scheduler - an instance of torch.optim.lr_scheduler._LRScheduler (should implement step method); for example,

runner.scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau()

runner.callbacks - ordered dictionary with Catalyst.Callback instances; for example,

runner.callbacks = {
    "accuracy": AccuracyCallback(),
    "criterion": CriterionCallback(),
    "optim": OptimizerCallback(),
    "saver": CheckpointCallback()
}

Dataflow section

runner.loaders - ordered dictionary with torch.DataLoaders; for example,

runner.loaders = {
    "train": MnistTrainLoader(),
    "valid": MnistValidLoader()
}

Note

• “train” prefix is used for training loaders - metrics computations, backward pass, optimization

• “valid” prefix is used for validation loaders - metrics computations only

• “infer” prefix is used for inference loaders - dataset prediction

runner.input - dictionary, containing batch of data from currents DataLoader; for example,

runner.input = {
    "images": np.ndarray(batch_size, c, h, w),
    "targets": np.ndarray(batch_size, 1),
}

runner.output - dictionary, containing model output for current batch; for example,

runner.output = {"logits": torch.Tensor(batch_size, num_classes)}

Metrics section

runner.batch_metrics - dictionary, flatten storage for batch metrics; for example,

runner.batch_metrics = {"loss": ..., "accuracy": ..., "iou": ...}

runner.loader_metrics - dictionary with aggregated batch statistics for loader (mean over all batches) and global loader metrics, like AUC; for example,

runner.loader_metrics = {"loss": ..., "accuracy": ..., "auc": ...}

runner.epoch_metrics - dictionary with summarized metrics for different loaders and global epoch metrics, like lr, momentum; for example,

runner.epoch_metrics = {
    "train_loss": ..., "train_auc": ..., "valid_loss": ...,
    "lr": ..., "momentum": ...,
}

Validation metrics section

runner.main_metric - string, containing name of metric of interest for optimization, validation and checkpointing during training

runner.minimize_metric - bool, indicator flag

• True if we need to minimize metric during training, like Cross Entropy loss

• False if we need to maximize metric during training, like Accuracy or Intersection over Union

Validation section

runner.valid_loader - string, name of validation loader for metric selection, validation and model checkpoining

runner.valid_metrics - dictionary with validation metrics for currect epoch; for example,

runner.valid_metrics = {"loss": ..., "accuracy": ..., "auc": ...}

Note

subdictionary of epoch_metrics

runner.is_best_valid - bool, indicator flag

• True if this training epoch is best over all epochs

• False if not

runner.best_valid_metrics - dictionary with best validation metrics during whole training process

Distributed section

runner.distributed_rank - distributed rank of current worker

runner.is_distributed_master - bool, indicator flag

• True if is master node (runner.distributed_rank == 0)

• False if is worker node (runner.distributed_rank != 0)

runner.is_distributed_worker - bool, indicator flag

• True if is worker node (runner.distributed_rank > 0)

• False if is master node (runner.distributed_rank <= 0)

Experiment info section

runner.global_sample_step - int, numerical indicator, counter for all individual samples, that passes through our model during training, validation and inference stages

runner.global_batch_step - int, numerical indicator, counter for all batches, that passes through our model during training, validation and inference stages

runner.global_epoch - int, numerical indicator, counter for all epochs, that have passed during model training, validation and inference stages

runner.verbose - bool, indicator flag

runner.is_check_run - bool, indicator flag

• True if you want to check you pipeline and run only 2 batches per loader and 2 epochs per stage

• False (default) if you want to just the pipeline

runner.need_early_stop - bool, indicator flag used for EarlyStopping and CheckRun Callbacks

• True if we need to stop the training

• False (default) otherwise

runner.need_exception_reraise - bool, indicator flag

• True (default) if you want to show exception during pipeline and stop the training process

• False otherwise

Stage info section

runner.stage_name - string, current stage name, for example,

runner.stage_name = "pretraining" / "training" / "finetuning" / etc

runner.num_epochs - int, maximum number of epochs, required for this stage

runner.is_infer_stage - bool, indicator flag

• True for inference stages

• False otherwise

Epoch info section

runner.epoch - int, numerical indicator for current stage epoch

Loader info section

runner.loader_sample_step - int, numerical indicator for number of samples passed through our model in current loader

runner.loader_batch_step - int, numerical indicator for batch index in current loader

runner.loader_name - string, current loader name for example,

runner.loader_name = "train_dataset1" / "valid_data2" / "infer_golden"

runner.loader_len - int, maximum number of batches in current loader

runner.loader_batch_size - int, batch size parameter in current loader

runner.is_train_loader - bool, indicator flag

• True for training loaders

• False otherwise

runner.is_valid_loader - bool, indicator flag

• True for validation loaders

• False otherwise

runner.is_infer_loader - bool, indicator flag

• True for inference loaders

• False otherwise

Batch info section

runner.batch_size - int, length of the current batch

Logging section

runner.logdir - string, path to logging directory to save all logs, metrics, checkpoints and artifacts

runner.checkpoint_data - dictionary with all extra data for experiment tracking

Extra section

runner.exception - python Exception instance to raise (or not ;) )

__init__(model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]] = None, device: Union[str, torch.device] = None)

Parameters

• model (StateModel) – Torch model object

• device (Device) – Torch device

property device

Returns the runner’s device instance.

get_attr(key: str, inner_key: str = None) → Any

Alias for python getattr method. Useful for Callbacks preparation and cases with multi-criterion, multi-optimizer setup. For example, when you would like to train multi-task classification.

Used to get a named attribute from a State by key keyword; for example

# example 1
runner.get_attr("criterion")
# is equivalent to
runner.criterion

# example 2
runner.get_attr("optimizer")
# is equivalent to
runner.optimizer

# example 3
runner.get_attr("scheduler")
# is equivalent to
runner.scheduler

With inner_key usage, it suppose to find a dictionary under key and would get inner_key from this dict; for example,

# example 1
runner.get_attr("criterion", "bce")
# is equivalent to
runner.criterion["bce"]

# example 2
runner.get_attr("optimizer", "adam")
# is equivalent to
runner.optimizer["adam"]

# example 3
runner.get_attr("scheduler", "adam")
# is equivalent to
runner.scheduler["adam"]

Parameters

• key (str) – name for attribute of interest, like criterion, optimizer, scheduler

• inner_key (str) – name of inner dictionary key

property model

Returns the runner’s model instance.

run_experiment(experiment: catalyst.core.experiment._Experiment = None) → catalyst.core.runner._Runner

Starts the experiment.

Parameters

experiment (_Experiment) – Experiment instance to use for Runner.

class catalyst.core.runner._StageBasedRunner(model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]] = None, device: Union[str, torch.device] = None)

Bases: catalyst.core.runner._Runner

Runner abstraction that suppose to have constant datasources per stage.

Callback

class catalyst.core.callback.Callback(order: int, node: int = <CallbackNode.All: 0>, scope: int = <CallbackScope.Stage: 0>)

Bases: object

An abstraction that lets you customize your experiment run logic. To give users maximum flexibility and extensibility Catalyst supports callback execution anywhere in the training loop:

-- stage start
---- epoch start
------ loader start
-------- batch start
---------- batch handler (Runner logic)
-------- batch end
------ loader end
---- epoch end
-- stage end

exception – if an Exception was raised

All callbacks have

• order from CallbackOrder

• node from CallbackNode

• scope from CallbackScope

Note

To learn more about Catalyst Core concepts, please check out

• catalyst.core.experiment._Experiment

• catalyst.core.runner._Runner

• catalyst.core.callback.Callback

Abstraction, please check out the implementations:

• catalyst.core.callbacks.criterion.CriterionCallback

• catalyst.core.callbacks.optimizer.OptimizerCallback

• catalyst.core.callbacks.scheduler.SchedulerCallback

• catalyst.core.callbacks.logging.TensorboardLogger

• catalyst.core.callbacks.checkpoint.CheckpointCallback

__init__(order: int, node: int = <CallbackNode.All: 0>, scope: int = <CallbackScope.Stage: 0>)

Callback initializer.

Parameters

• order – flag from CallbackOrder

• node – flag from CallbackNode

• scope – flag from CallbackScope

on_batch_end(runner: _Runner)

Event handler for batch end.

Parameters

runner ("_Runner") – _Runner instance.

on_batch_start(runner: _Runner)

Event handler for batch start.

Parameters

runner ("_Runner") – _Runner instance.

on_epoch_end(runner: _Runner)

Event handler for epoch end.

Parameters

runner ("_Runner") – _Runner instance.

on_epoch_start(runner: _Runner)

Event handler for epoch start.

Parameters

runner ("_Runner") – _Runner instance.

on_exception(runner: _Runner)

Event handler for exception case.

Parameters

runner ("_Runner") – _Runner instance.

on_loader_end(runner: _Runner)

Event handler for loader end.

Parameters

runner ("_Runner") – _Runner instance.

on_loader_start(runner: _Runner)

Event handler for loader start.

Parameters

runner ("_Runner") – _Runner instance.

on_stage_end(runner: _Runner)

Event handler for stage end.

Parameters

runner ("_Runner") – _Runner instance.

on_stage_start(runner: _Runner)

Event handler for stage start.

Parameters

runner ("_Runner") – _Runner instance.

class catalyst.core.callback.CallbackNode

Bases: enum.IntFlag

Callback node usage flag during distributed training.

• All (0) - use on all nodes, botch master and worker.

• Master (1) - use only on master node.

• Worker (2) - use only in worker nodes.

All = 0

Master = 1

Worker = 2

class catalyst.core.callback.CallbackOrder

Bases: enum.IntFlag

Callback usage order during training.

Catalyst executes Callbacks with low CallbackOrder before Callbacks with high CallbackOrder.

Predefined orders:

• Internal (0) - some Catalyst Extras, like PhaseCallbacks (used in GANs).

• Metric (20) - Callbacks with metrics and losses computation.

• MetricAggregation (40) - metrics aggregation callbacks, like sum different losses into one.

• Optimizer (60) - optimizer step, requires computed metrics for optimization.

• Validation (80) - validation step, computes validation metrics subset based on all metrics.

• Scheduler (100) - scheduler step, in ReduceLROnPlateau case requires computed validation metrics for optimizer schedule.

• Logging (120) - logging step, logs metrics to Console/Tensorboard/Alchemy, requires computed metrics.

• External (200) - additional callbacks with custom logic, like InferenceCallbacks

Nevertheless, you always can create CustomCallback with any order, for example:

>>> class MyCustomCallback(Callback):
>>>     def __init__(self):
>>>         super().__init__(order=42)
>>>     ...
# MyCustomCallback will be executed after all `Metric`-Callbacks
# but before all `MetricAggregation`-Callbacks.

External = 200

Internal = 0

Logging = 120

Metric = 20

MetricAggregation = 40

Optimizer = 60

Scheduler = 100

Validation = 80

class catalyst.core.callback.CallbackScope

Bases: enum.IntFlag

Callback scope usage flag during training.

• Stage (0) - use Callback only during one experiment stage.

• Experiment (1) - use Callback during whole experiment run.

Experiment = 1

Stage = 0

Callbacks

Checkpoint

class catalyst.core.callbacks.checkpoint.CheckpointCallback(save_n_best: int = 1, resume: str = None, resume_dir: str = None, metrics_filename: str = '_metrics.json', load_on_stage_start: Union[str, Dict[str, str]] = None, load_on_stage_end: Union[str, Dict[str, str]] = None)

Bases: catalyst.core.callbacks.checkpoint.BaseCheckpointCallback

Checkpoint callback to save/restore your model/criterion/optimizer/scheduler.

__init__(save_n_best: int = 1, resume: str = None, resume_dir: str = None, metrics_filename: str = '_metrics.json', load_on_stage_start: Union[str, Dict[str, str]] = None, load_on_stage_end: Union[str, Dict[str, str]] = None)

Parameters

• save_n_best (int) – number of best checkpoint to keep, if 0 then store only last state of model and load_on_stage_end should be one of last or last_full.

• resume (str) – path to checkpoint to load and initialize runner state

• resume_dir (str) – directory with checkpoints, if specified in combination with resume than resume checkpoint will be loaded from resume_dir

• metrics_filename (str) – filename to save metrics in checkpoint folder. Must ends on .json or .yml

• load_on_stage_start (str or Dict[str, str]) –

  load specified state/model at stage start.

  If passed string then will be performed initialization from specified state (best/best_full/last/last_full) or checkpoint file.

  If passed dict then will be performed initialization only for specified parts - model, criterion, optimizer, scheduler.

  Example

  >>> # possible checkpoints to use:
  >>> #   "best"/"best_full"/"last"/"last_full"
  >>> #   or path to specific checkpoint
  >>> to_load = {
  >>>    "model": "path/to/checkpoint.pth",
  >>>    "criterion": "best",
  >>>    "optimizer": "last_full",
  >>>    "scheduler": "best_full",
  >>> }
  >>> CheckpointCallback(load_on_stage_start=to_load)
  

  All other keys instead of "model", "criterion", "optimizer" and "scheduler" will be ignored.

  If None or an empty dict (or dict without mentioned above keys) then no action is required at stage start and:

  • Config API - will be used best state of model

  • Notebook API - no action will be performed (will be used the last state)

  NOTE: Loading will be performed on all stages except first.

  NOTE: Criterion, optimizer and scheduler are optional keys and should be loaded from full checkpoint.

  Model state can be loaded from any checkpoint.

  When dict contains keys for model and some other part (for example {"model": "last", "optimizer": "last"}) and they match in prefix ("best" and "best_full") then will be loaded full checkpoint because it contains required states.

• load_on_stage_end (str or Dict[str, str]) –

  load specified state/model at stage end.

  If passed string then will be performed initialization from specified state (best/best_full/last/last_full) or checkpoint file.

  If passed dict then will be performed initialization only for specified parts - model, criterion, optimizer, scheduler. Logic for dict is the same as for load_on_stage_start.

  If None then no action is required at stage end and will be used the last runner.

  NOTE: Loading will be performed always at stage end.

get_checkpoint_suffix(checkpoint: dict) → str

Create checkpoint filename suffix based on checkpoint data.

Parameters

checkpoint (dict) – checkpoint dict, should contain stage_name and epoch keys.

on_epoch_end(runner: catalyst.core.runner._Runner) → None

Collect and save checkpoint after epoch.

Parameters

runner (_Runner) – current runner

on_stage_end(runner: catalyst.core.runner._Runner) → None

Show information about best checkpoints during the stage and load model specified in load_on_stage_end.

Parameters

runner (_Runner) – current runner

on_stage_start(runner: catalyst.core.runner._Runner) → None

Setup model for stage.

NOTE: If CheckpointCallback initialized with resume (as path to checkpoint file) or resume (as filename) and resume_dir (as directory with file) then will be performed loading checkpoint.

Parameters

runner (_Runner) – current runner

process_checkpoint(logdir: Union[str, pathlib.Path], checkpoint: Dict, is_best: bool, main_metric: str = 'loss', minimize_metric: bool = True) → None

Save checkpoint and metrics.

Parameters

• logdir (str or Path object) – directory for storing checkpoints

• checkpoint (dict) – dict with checkpoint data

• is_best (bool) – indicator to save best checkpoint, if true then will be saved two additional checkpoints - best and best_full.

• main_metric (str) – metric to use for selecting the best model

• minimize_metric (bool) – indicator for selecting best metric, if true then best metric will be the metric with the lowest value, otherwise with the greatest value.

process_metrics(last_valid_metrics: Dict[str, float]) → Dict

Add last validation metrics to list of previous validation metrics and keep save_n_best metrics.

Parameters

last_valid_metrics (dict) – dict with metrics from last validation step.

truncate_checkpoints(minimize_metric: bool) → None

Keep save_n_best checkpoints based on main metric.

Parameters

minimize_metric (bool) – if True then keep save_n_best checkpoints with the lowest/highest values of the main metric.

class catalyst.core.callbacks.checkpoint.IterationCheckpointCallback(save_n_last: int = 1, period: int = 100, stage_restart: bool = True, metrics_filename: str = '_metrics_iter.json', load_on_stage_end: str = 'best_full')

Bases: catalyst.core.callbacks.checkpoint.BaseCheckpointCallback

Iteration checkpoint callback to save your model/criterion/optimizer.

__init__(save_n_last: int = 1, period: int = 100, stage_restart: bool = True, metrics_filename: str = '_metrics_iter.json', load_on_stage_end: str = 'best_full')

Parameters

• save_n_last (int) – number of last checkpoint to keep

• period (int) – save the checkpoint every period

• stage_restart (bool) – restart counter every stage or not

• metrics_filename (str) – filename to save metrics in checkpoint folder. Must ends on .json or .yml

• load_on_stage_end (str) – name of the model to load at the end of the stage. You can use best, best_full (default) to load the best model according to validation metrics, or last last_full to use just the last one.

get_checkpoint_suffix(checkpoint: dict) → str

Create checkpoint filename suffix based on checkpoint data.

Parameters

checkpoint (dict) – checkpoint dict, should contain stage_name and epoch keys.

on_batch_end(runner: catalyst.core.runner._Runner)

Save checkpoint based on batches count.

Parameters

runner (_Runner) – current runner

on_stage_end(runner: catalyst.core.runner._Runner)

Load model specified in load_on_stage_end.

Parameters

runner (_Runner) – current runner

on_stage_start(runner: catalyst.core.runner._Runner)

Reset iterations counter.

Parameters

runner (_Runner) – current runner

process_checkpoint(logdir: Union[str, pathlib.Path], checkpoint: Dict, batch_metrics: Dict[str, float])

Save checkpoint and metrics.

Parameters

• logdir (str or Path object) – directory for storing checkpoints

• checkpoint (dict) – dict with checkpoint data

• batch_metrics (dict) – dict with metrics based on a few batches

process_metrics() → Dict

Update metrics with last save_n_last checkpoints.

truncate_checkpoints(**kwargs) → None

Keep save_n_best checkpoints based on main metric.

Criterion

class catalyst.core.callbacks.criterion.CriterionCallback(input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', prefix: str = 'loss', criterion_key: str = None, multiplier: float = 1.0, **metric_kwargs)

Bases: catalyst.core.callbacks.metrics._MetricCallback

Callback for that measures loss with specified criterion.

__init__(input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', prefix: str = 'loss', criterion_key: str = None, multiplier: float = 1.0, **metric_kwargs)

Parameters

• input_key (Union[str, List[str], Dict[str, str]]) – key/list/dict of keys that takes values from the input dictionary If ‘__all__’, the whole input will be passed to the criterion If None, empty dict will be passed to the criterion.

• output_key (Union[str, List[str], Dict[str, str]]) – key/list/dict of keys that takes values from the input dictionary If ‘__all__’, the whole output will be passed to the criterion If None, empty dict will be passed to the criterion.

• prefix (str) – prefix for metrics and output key for loss in runner.batch_metrics dictionary

• criterion_key (str) – A key to take a criterion in case there are several of them and they are in a dictionary format.

• multiplier (float) – scale factor for the output loss.

property metric_fn

Docs. Contribution is welcome.

Type

@TODO

on_stage_start(runner: catalyst.core.runner._Runner)

Checks that the current stage has correct criterion.

Early Stop

class catalyst.core.callbacks.early_stop.CheckRunCallback(num_batch_steps: int = 3, num_epoch_steps: int = 2)

Bases: catalyst.core.callback.Callback

@TODO: Docs. Contribution is welcome.

__init__(num_batch_steps: int = 3, num_epoch_steps: int = 2)

@TODO: Docs. Contribution is welcome.

on_batch_end(runner: catalyst.core.runner._Runner)

@TODO: Docs. Contribution is welcome.

on_epoch_end(runner: catalyst.core.runner._Runner)

@TODO: Docs. Contribution is welcome.

class catalyst.core.callbacks.early_stop.EarlyStoppingCallback(patience: int, metric: str = 'loss', minimize: bool = True, min_delta: float = 1e-06)

Bases: catalyst.core.callback.Callback

@TODO: Docs. Contribution is welcome.

__init__(patience: int, metric: str = 'loss', minimize: bool = True, min_delta: float = 1e-06)

@TODO: Docs. Contribution is welcome.

on_epoch_end(runner: catalyst.core.runner._Runner) → None

@TODO: Docs. Contribution is welcome.

Exception

class catalyst.core.callbacks.exception.ExceptionCallback

Bases: catalyst.core.callback.Callback

@TODO: Docs. Contribution is welcome.

__init__()

@TODO: Docs. Contribution is welcome.

on_exception(runner: catalyst.core.runner._Runner)

@TODO: Docs. Contribution is welcome.

Logging

class catalyst.core.callbacks.logging.ConsoleLogger

Bases: catalyst.core.callback.Callback

Logger callback, translates runner.*_metrics to console and text file.

__init__()

Init ConsoleLogger.

on_epoch_end(runner: catalyst.core.runner._Runner)

Translate runner.metric_manager to console and text file at the end of an epoch.

on_stage_end(runner: catalyst.core.runner._Runner)

Called at the end of each stage.

on_stage_start(runner: catalyst.core.runner._Runner)

Prepare runner.logdir for the current stage.

class catalyst.core.callbacks.logging.TensorboardLogger(metric_names: List[str] = None, log_on_batch_end: bool = True, log_on_epoch_end: bool = True)

Bases: catalyst.core.callback.Callback

Logger callback, translates runner.metric_manager to tensorboard.

__init__(metric_names: List[str] = None, log_on_batch_end: bool = True, log_on_epoch_end: bool = True)

Parameters

• metric_names (List[str]) – list of metric names to log, if none - logs everything

• log_on_batch_end (bool) – logs per-batch metrics if set True

• log_on_epoch_end (bool) – logs per-epoch metrics if set True

on_batch_end(runner: catalyst.core.runner._Runner)

Translate batch metrics to tensorboard.

on_epoch_end(runner: catalyst.core.runner._Runner)

Translate epoch metrics to tensorboard.

on_loader_start(runner: catalyst.core.runner._Runner)

Prepare tensorboard writers for the current stage.

on_stage_end(runner: catalyst.core.runner._Runner)

Close opened tensorboard writers.

on_stage_start(runner: catalyst.core.runner._Runner)

@TODO: Docs. Contribution is welcome.

class catalyst.core.callbacks.logging.VerboseLogger(always_show: List[str] = None, never_show: List[str] = None)

Bases: catalyst.core.callback.Callback

Logs the params into console.

__init__(always_show: List[str] = None, never_show: List[str] = None)

Parameters

• always_show (List[str]) – list of metrics to always show if None default is ["_timer/_fps"] to remove always_show metrics set it to an empty list []

• never_show (List[str]) – list of metrics which will not be shown

on_batch_end(runner: catalyst.core.runner._Runner)

Update tqdm progress bar at the end of each batch.

on_exception(runner: catalyst.core.runner._Runner)

Called if an Exception was raised.

on_loader_end(runner: catalyst.core.runner._Runner)

Cleanup and close tqdm progress bar.

on_loader_start(runner: catalyst.core.runner._Runner)

Init tqdm progress bar.

Metrics

class catalyst.core.callbacks.metrics._MetricCallback(prefix: str, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metrics_kwargs)

Bases: abc.ABC, catalyst.core.callback.Callback

@TODO: Docs. Contribution is welcome.

__init__(prefix: str, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metrics_kwargs)

@TODO: Docs. Contribution is welcome.

abstract property metric_fn

Docs. Contribution is welcome.

Type

@TODO

on_batch_end(runner: catalyst.core.runner._Runner) → None

Computes the metric and add it to batch metrics.

class catalyst.core.callbacks.metrics.MetricCallback(prefix: str, metric_fn: Callable, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metric_kwargs)

Bases: catalyst.core.callbacks.metrics._MetricCallback

A callback that returns single metric on runner.on_batch_end.

__init__(prefix: str, metric_fn: Callable, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metric_kwargs)

@TODO: Docs. Contribution is welcome.

property metric_fn

Docs. Contribution is welcome.

Type

@TODO

class catalyst.core.callbacks.metrics.MultiMetricCallback(prefix: str, metric_fn: Callable, list_args: List, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metrics_kwargs)

Bases: catalyst.core.callbacks.metrics.MetricCallback

A callback that returns multiple metrics on runner.on_batch_end.

__init__(prefix: str, metric_fn: Callable, list_args: List, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metrics_kwargs)

@TODO: Docs. Contribution is welcome.

on_batch_end(runner: catalyst.core.runner._Runner) → None

Batch end hook.

Parameters

runner (_Runner) – current runner

class catalyst.core.callbacks.metrics.MetricAggregationCallback(prefix: str, metrics: Union[str, List[str], Dict[str, float]] = None, mode: str = 'mean', multiplier: float = 1.0)

Bases: catalyst.core.callback.Callback

A callback to aggregate several metrics in one value.

__init__(prefix: str, metrics: Union[str, List[str], Dict[str, float]] = None, mode: str = 'mean', multiplier: float = 1.0) → None

Parameters

• prefix (str) – new key for aggregated metric.

• metrics (Union[str, List[str], Dict[str, float]]) – If not None, it aggregates only the values from the metric by these keys. for weighted_sum aggregation it must be a Dict[str, float].

• mode (str) – function for aggregation. Must be either sum, mean or weighted_sum.

• multiplier (float) – scale factor for the aggregated metric.

on_batch_end(runner: catalyst.core.runner._Runner) → None

Computes the metric and add it to the metrics.

Parameters

runner (_Runner) – current runner

class catalyst.core.callbacks.metrics.MetricManagerCallback

Bases: catalyst.core.callback.Callback

Prepares metrics for logging, transferring values from PyTorch to numpy.

__init__()

@TODO: Docs. Contribution is welcome.

on_batch_end(runner: catalyst.core.runner._Runner) → None

Batch end hook.

Parameters

runner (_Runner) – current runner

on_batch_start(runner: catalyst.core.runner._Runner) → None

Batch start hook.

Parameters

runner (_Runner) – current runner

on_epoch_start(runner: catalyst.core.runner._Runner) → None

Epoch start hook.

Parameters

runner (_Runner) – current runner

on_loader_end(runner: catalyst.core.runner._Runner) → None

Loader end hook.

Parameters

runner (_Runner) – current runner

on_loader_start(runner: catalyst.core.runner._Runner) → None

Loader start hook.

Parameters

runner (_Runner) – current runner

Optimizer

class catalyst.core.callbacks.optimizer.OptimizerCallback(metric_key: str = None, optimizer_key: str = None, accumulation_steps: int = 1, grad_clip_params: Dict = None, decouple_weight_decay: bool = True, loss_key: str = None)

Bases: catalyst.core.callback.Callback

Optimizer callback, abstraction over optimizer step.

__init__(metric_key: str = None, optimizer_key: str = None, accumulation_steps: int = 1, grad_clip_params: Dict = None, decouple_weight_decay: bool = True, loss_key: str = None)

Parameters

• loss_key (str) – key to get loss from runner.batch_metrics

• optimizer_key (str) – A key to take a optimizer in case there are several of them and they are in a dictionary format.

• accumulation_steps (int) – number of steps before model.zero_grad()

• grad_clip_params (dict) – params for gradient clipping

• decouple_weight_decay (bool) – If True - decouple weight decay regularization.

static grad_step(*, optimizer: torch.optim.optimizer.Optimizer, optimizer_wds: List[float] = 0, grad_clip_fn: Callable = None) → None

Makes a gradient step for a given optimizer.

Parameters

• optimizer (Optimizer) – the optimizer

• optimizer_wds (List[float]) – list of weight decay parameters for each param group

• grad_clip_fn (Callable) – function for gradient clipping

on_batch_end(runner: catalyst.core.runner._Runner) → None

On batch end event

Parameters

runner (_Runner) – current runner

on_epoch_end(runner: catalyst.core.runner._Runner) → None

On epoch end event.

Parameters

runner (_Runner) – current runner

on_epoch_start(runner: catalyst.core.runner._Runner) → None

On epoch start event.

Parameters

runner (_Runner) – current runner

on_stage_start(runner: catalyst.core.runner._Runner) → None

Checks that the current stage has correct optimizer.

Scheduler

class catalyst.core.callbacks.scheduler.SchedulerCallback(scheduler_key: str = None, mode: str = None, reduced_metric: str = None)

Bases: catalyst.core.callback.Callback

@TODO: Docs. Contribution is welcome.

__init__(scheduler_key: str = None, mode: str = None, reduced_metric: str = None)

@TODO: Docs. Contribution is welcome.

on_batch_end(runner: catalyst.core.runner._Runner) → None

Batch end hook.

Parameters

runner (_Runner) – current runner

on_epoch_end(runner: catalyst.core.runner._Runner) → None

Epoch end hook.

Parameters

runner (_Runner) – current runner

on_loader_start(runner: catalyst.core.runner._Runner) → None

Loader start hook.

Parameters

runner (_Runner) – current runner

on_stage_start(runner: catalyst.core.runner._Runner) → None

Stage start hook.

Parameters

runner (_Runner) – current runner

step_batch(runner: catalyst.core.runner._Runner) → None

@TODO: Docs. Contribution is welcome.

Parameters

runner (_Runner) – current runner

step_epoch(runner: catalyst.core.runner._Runner) → None

@TODO: Docs. Contribution is welcome.

Parameters

runner (_Runner) – current runner

class catalyst.core.callbacks.scheduler.LRUpdater(optimizer_key: str = None)

Bases: catalyst.core.callback.Callback

Basic class that all Lr updaters inherit from.

__init__(optimizer_key: str = None)

Parameters

optimizer_key (str) – which optimizer key to use for learning rate scheduling

calc_lr()

@TODO: Docs. Contribution is welcome.

calc_momentum()

@TODO: Docs. Contribution is welcome.

on_batch_end(runner: catalyst.core.runner._Runner) → None

Batch end hook.

Parameters

runner (_Runner) – current runner

on_loader_start(runner: catalyst.core.runner._Runner) → None

Loader start hook.

Parameters

runner (_Runner) – current runner

on_stage_start(runner: catalyst.core.runner._Runner) → None

Stage start hook.

Parameters

runner (_Runner) – current runner

update_optimizer(runner: catalyst.core.runner._Runner) → None

@TODO: Docs. Contribution is welcome.

Parameters

runner (_Runner) – current runner

Timer

class catalyst.core.callbacks.timer.TimerCallback

Bases: catalyst.core.callback.Callback

Logs pipeline execution time.

__init__()

@TODO: Docs. Contribution is welcome.

on_batch_end(runner: catalyst.core.runner._Runner) → None

Batch end hook.

Parameters

runner (_Runner) – current runner

on_batch_start(runner: catalyst.core.runner._Runner) → None

Batch start hook.

Parameters

runner (_Runner) – current runner

on_loader_end(runner: catalyst.core.runner._Runner) → None

Loader end hook.

Parameters

runner (_Runner) – current runner

on_loader_start(runner: catalyst.core.runner._Runner) → None

Loader start hook.

Parameters

runner (_Runner) – current runner

Validation

class catalyst.core.callbacks.validation.ValidationManagerCallback

Bases: catalyst.core.callback.Callback

A callback to aggregate runner.valid_metrics from runner.epoch_metrics.

__init__()

@TODO: Docs. Contribution is welcome.

on_epoch_end(runner: catalyst.core.runner._Runner) → None

Epoch end hook.

Parameters

runner (_Runner) – current runner

on_epoch_start(runner: catalyst.core.runner._Runner) → None

Epoch start hook.

Parameters

runner (_Runner) – current runner

Registry

catalyst.core.registry.Callback(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

catalyst.core.registry.Criterion(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

catalyst.core.registry.Optimizer(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

catalyst.core.registry.Scheduler(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

catalyst.core.registry.Module(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

catalyst.core.registry.Model(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

catalyst.core.registry.Sampler(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

catalyst.core.registry.Transform(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

Utils

catalyst.core.utils.callbacks.sort_callbacks_by_order(callbacks: Union[List, Dict, collections.OrderedDict]) → collections.OrderedDict

Creates an sequence of callbacks and sort them.

Parameters

callbacks – either list of callbacks or ordered dict

Returns

sequence of callbacks sorted by callback order

catalyst.core.utils.callbacks.filter_callbacks_by_node(callbacks: Union[Dict, collections.OrderedDict]) → Union[Dict, collections.OrderedDict]

Filters callbacks based on running node. Deletes worker-only callbacks from CallbackNode.Master and master-only callbacks from CallbackNode.Worker.

Parameters

callbacks (Union[Dict, OrderedDict]) – callbacks

Returns

filtered callbacks dictionary.

Return type

Union[Dict, OrderedDict]

Legacy

Runner

class catalyst.core.legacy._RunnerLegacy

Bases: object

Special class to encapsulate all catalyst.core.runner._Runner and catalyst.core.runner.State legacy into one place. Used to make catalyst.core.runner._Runner cleaner and easier to understand.

Saved for backward compatibility. Should be removed someday.

property batch_in

Alias for runner.input.

Warning

Deprecated, saved for backward compatibility. Please use runner.input instead.

property batch_out

Alias for runner.output.

Warning

Deprecated, saved for backward compatibility. Please use runner.output instead.

property loader_step

Alias for runner.loader_batch_step.

Warning

Deprecated, saved for backward compatibility. Please use runner.loader_batch_step instead.

property need_backward_pass

Alias for runner.is_train_loader.

Warning

Deprecated, saved for backward compatibility. Please use runner.is_train_loader instead.

property state

Alias for runner.

Warning

Deprecated, saved for backward compatibility. Please use runner instead.