DL

Core

class catalyst.dl.core.callback.CallbackOrder

Bases: enum.IntFlag

An enumeration.

Criterion = 20

External = 100

Internal = 0

Metric = 80

Optimizer = 40

Other = 200

Scheduler = 60

Unknown = -100

class catalyst.dl.core.callback.Callback(order: int)

Bases: object

Abstract class that all callback (e.g., Logger) classes extends from. Must be extended before usage.

usage example:

-- stage start
---- epoch start (one epoch - one run of every loader)
------ loader start
-------- batch start
-------- batch handler
-------- batch end
------ loader end
---- epoch end
-- stage end

exception – if an Exception was raised

All callbacks has order value from CallbackOrder

__init__(order: int)

For order see CallbackOrder class

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_batch_start(state: catalyst.dl.core.state.RunnerState)

on_epoch_end(state: catalyst.dl.core.state.RunnerState)

on_epoch_start(state: catalyst.dl.core.state.RunnerState)

on_exception(state: catalyst.dl.core.state.RunnerState)

on_loader_end(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state: catalyst.dl.core.state.RunnerState)

on_stage_end(state: catalyst.dl.core.state.RunnerState)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.core.callback.MetricCallback(prefix: str, metric_fn: Callable, input_key: str = 'targets', output_key: str = 'logits', **metric_params)

Bases: catalyst.dl.core.callback.Callback

A callback that returns single metric on state.on_batch_end

on_batch_end(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.core.callback.MultiMetricCallback(prefix: str, metric_fn: Callable, list_args: List, input_key: str = 'targets', output_key: str = 'logits', **metric_params)

Bases: catalyst.dl.core.callback.Callback

A callback that returns multiple metrics on state.on_batch_end

on_batch_end(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.core.callback.LoggerCallback(order: int = None)

Bases: catalyst.dl.core.callback.Callback

Loggers are executed on start before all callbacks, and on end after all callbacks.

class catalyst.dl.core.callback.MeterMetricsCallback(metric_names: List[str], meter_list: List, input_key: str = 'targets', output_key: str = 'logits', class_names: List[str] = None, num_classes: int = 2, activation: str = 'Sigmoid')

Bases: catalyst.dl.core.callback.Callback

A callback that tracks metrics through meters and prints metrics for each class on state.on_loader_end. This callback works for both single metric and multi-metric meters.

__init__(metric_names: List[str], meter_list: List, input_key: str = 'targets', output_key: str = 'logits', class_names: List[str] = None, num_classes: int = 2, activation: str = 'Sigmoid')

Parameters

• metric_names (List[str]) – of metrics to print Make sure that they are in the same order that metrics are outputted by the meters in meter_list

• meter_list (list-like) – List of meters.meter.Meter instances len(meter_list) == n_classes

• input_key (str) – input key to use for metric calculation specifies our y_true.

• output_key (str) – output key to use for metric calculation; specifies our y_pred

• class_names (List[str]) – class names to display in the logs. If None, defaults to indices for each class, starting from 0.

• num_classes (int) – Number of classes; must be > 1

• activation (str) – An torch.nn activation applied to the logits. Must be one of [‘none’, ‘Sigmoid’, ‘Softmax2d’]

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_loader_end(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state)

class catalyst.dl.core.experiment.Experiment

Bases: abc.ABC

Object containing all information required to run the experiment

Abstract, look for implementations

abstract property distributed_params

abstract get_callbacks(stage: str) → OrderedDict[str, Callback]

abstract get_criterion(stage: str) → torch.nn.modules.module.Module

get_datasets(stage: str, **kwargs) → OrderedDict[str, Dataset]

get_experiment_components(model: torch.nn.modules.module.Module, stage: str) → Tuple[torch.nn.modules.module.Module, torch.optim.optimizer.Optimizer, torch.optim.lr_scheduler._LRScheduler]

abstract get_loaders(stage: str) → OrderedDict[str, DataLoader]

abstract get_model(stage: str) → torch.nn.modules.module.Module

abstract get_optimizer(stage: str, model: torch.nn.modules.module.Module) → torch.optim.optimizer.Optimizer

abstract get_scheduler(stage: str, optimizer) → torch.optim.lr_scheduler._LRScheduler

abstract get_state_params(stage: str) → Mapping[str, Any]

static get_transforms(stage: str = None, mode: str = None)

abstract property initial_seed

abstract property logdir

abstract property monitoring_params

abstract property stages

class catalyst.dl.core.metric_manager.TimerManager

Bases: object

reset() → None

Reset all previous timers

start(name: str) → None

Starts timer name :param name: name of a timer :type name: str

stop(name: str) → None

Stops timer name :param name: name of a timer :type name: str

class catalyst.dl.core.metric_manager.MetricManager(valid_loader: str = 'valid', main_metric: str = 'loss', minimize: bool = True, batch_consistant_metrics: bool = True)

Bases: object

add_batch_value(name: str = None, value: Any = None, metrics_dict: Dict[str, Any] = None)

property batch_values

begin_batch()

begin_epoch()

begin_loader(name: str)

end_batch()

end_epoch_train()

end_loader()

property main_metric_value

class catalyst.dl.core.runner.Runner(model: torch.nn.modules.module.Module = None, device=None)

Bases: abc.ABC

__init__(model: torch.nn.modules.module.Module = None, device=None)

@TODO: write docs

abstract forward(batch: Mapping[str, Any]) → Mapping[str, Any]

predict_batch(batch: Mapping[str, Any]) → Mapping[str, Any]

Run model for a batch of elements WARN: You should not override this method. If you need specific model call, override forward() method :param batch: Key-value batch items

Returns: model output key-value

run_experiment(experiment: catalyst.dl.core.experiment.Experiment, check: bool = False)

class catalyst.dl.core.state.RunnerState(*, device=None, model=None, criterion=None, optimizer: torch.optim.optimizer.Optimizer = None, scheduler=None, logdir: str = None, stage: str = 'infer', num_epochs: int = 1, main_metric: str = 'loss', minimize_metric: bool = True, valid_loader: str = 'valid', verbose: bool = False, checkpoint_data: Dict = None, batch_consistant_metrics: bool = True, **kwargs)

Bases: catalyst.utils.frozen.FrozenClass

An object that is used to pass internal state during train/valid/infer.

property epoch_log

get_key(key, inner_key=None)

on_batch_end_post()

on_batch_end_pre()

on_batch_start_post()

on_batch_start_pre()

on_epoch_end_post()

on_epoch_end_pre()

on_epoch_start_post()

on_epoch_start_pre()

on_exception_post()

on_exception_pre()

on_loader_end_post()

on_loader_end_pre()

on_loader_start_post()

on_loader_start_pre()

on_stage_end_post()

on_stage_end_pre()

on_stage_start_post()

on_stage_start_pre()

set_key(value, key, inner_key=None)

property stage_epoch_log

Callbacks

class catalyst.dl.callbacks.checkpoint.CheckpointCallback(save_n_best: int = 1, resume: str = None, resume_dir: str = None, metric_filename: str = '_metrics.json')

Bases: catalyst.dl.callbacks.checkpoint.BaseCheckpointCallback

Checkpoint callback to save/restore your model/criterion/optimizer/metrics.

__init__(save_n_best: int = 1, resume: str = None, resume_dir: str = None, metric_filename: str = '_metrics.json')

Parameters

• save_n_best (int) – number of best checkpoint to keep

• resume (str) – path to checkpoint to load and initialize runner state

• metric_filename (str) – filename to save metrics in checkpoint folder. Must ends on .json or .yml

get_checkpoint_suffix(checkpoint: dict) → str

get_metric(last_valid_metrics) → Dict

static load_checkpoint(*, filename, state: catalyst.dl.core.state.RunnerState)

on_epoch_end(state: catalyst.dl.core.state.RunnerState)

on_stage_end(state: catalyst.dl.core.state.RunnerState)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

process_checkpoint(logdir: str, checkpoint: Dict, is_best: bool, main_metric: str = 'loss', minimize_metric: bool = True)

truncate_checkpoints(minimize_metric: bool) → None

class catalyst.dl.callbacks.checkpoint.IterationCheckpointCallback(save_n_last: int = 3, num_iters: int = 100, stage_restart: bool = True, metric_filename: str = '_metrics_iter.json')

Bases: catalyst.dl.callbacks.checkpoint.BaseCheckpointCallback

Iteration checkpoint callback to save your model/criterion/optimizer

__init__(save_n_last: int = 3, num_iters: int = 100, stage_restart: bool = True, metric_filename: str = '_metrics_iter.json')

Parameters

• save_n_last (int) – number of last checkpoint to keep

• num_iters (int) – save the checkpoint every num_iters

• stage_restart (bool) – restart counter every stage or not

• metric_filename (str) – filename to save metrics in checkpoint folder. Must ends on .json or .yml

get_checkpoint_suffix(checkpoint: dict) → str

get_metric(**kwargs) → Dict

on_batch_end(state)

on_stage_start(state)

process_checkpoint(logdir: str, checkpoint: Dict, batch_values: Dict[str, float])

truncate_checkpoints(**kwargs) → None

class catalyst.dl.callbacks.criterion.CriterionCallback(input_key: Union[str, List[str]] = 'targets', output_key: Union[str, List[str]] = 'logits', prefix: str = 'loss', criterion_key: str = None, multiplier: float = 1.0)

Bases: catalyst.dl.core.callback.Callback

Callback for that measures loss with specified criterion.

__init__(input_key: Union[str, List[str]] = 'targets', output_key: Union[str, List[str]] = 'logits', prefix: str = 'loss', criterion_key: str = None, multiplier: float = 1.0)

Parameters

• input_key (Union[str, List[str]]) – key or list of keys that takes values from the input dictionary If None, the whole input will be passed to the criterion.

• output_key (Union[str, List[str]]) – key or list of keys that takes values from the output dictionary If None, the whole output will be passed to the criterion.

• prefix (str) – prefix for metrics and output key for loss in state.loss dictionary

• criterion_key (str) – A key to take a criterion in case there are several of them and they are in a dictionary format.

• multiplier (float) – scale factor for the output loss.

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.criterion.CriterionAggregatorCallback(prefix: str, loss_keys: Union[str, List[str]] = None, loss_aggregate_fn: str = 'sum', multiplier: float = 1.0)

Bases: catalyst.dl.core.callback.Callback

This callback allows you to aggregate the values of the loss (by sum or mean) and put the value back into state.loss.

__init__(prefix: str, loss_keys: Union[str, List[str]] = None, loss_aggregate_fn: str = 'sum', multiplier: float = 1.0) → None

Parameters

• prefix (str) – new key for aggregated loss.

• loss_keys (List[str]) – If not empty, it aggregates only the values from the loss by these keys.

• loss_aggregate_fn (str) – function for aggregation. Must be either sum or mean.

• multiplier (float) – scale factor for the aggregated loss.

on_batch_end(state: catalyst.dl.core.state.RunnerState) → None

class catalyst.dl.callbacks.inference.InferCallback(out_dir=None, out_prefix=None)

Bases: catalyst.dl.core.callback.Callback

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_loader_end(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state: catalyst.dl.core.state.RunnerState)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.inference.InferMaskCallback(out_dir=None, out_prefix=None, input_key=None, output_key=None, name_key=None, mean=None, std=None, threshold: float = 0.5, mask_strength: float = 0.5, mask_type: str = 'soft')

Bases: catalyst.dl.core.callback.Callback

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state: catalyst.dl.core.state.RunnerState)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.logging.VerboseLogger(always_show: List[str] = None, never_show: List[str] = None)

Bases: catalyst.dl.core.callback.LoggerCallback

__init__(always_show: List[str] = None, never_show: List[str] = None)

Logs the params into console

Parameters

• always_show (List[str]) – list of metrics to always show if None default is ["_timers/_fps"] to remove always_show metrics set it to an empty list []

• never_show (List[str]) – list of metrics which will not be shown

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_exception(state: catalyst.dl.core.state.RunnerState)

on_loader_end(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.logging.ConsoleLogger

Bases: catalyst.dl.core.callback.LoggerCallback

Logger callback, translates state.metrics to console and text file

on_epoch_end(state)

on_stage_end(state)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.logging.TensorboardLogger(metric_names: List[str] = None, log_on_batch_end: bool = True, log_on_epoch_end: bool = True)

Bases: catalyst.dl.core.callback.LoggerCallback

Logger callback, translates state.metrics to tensorboard

__init__(metric_names: List[str] = None, log_on_batch_end: bool = True, log_on_epoch_end: bool = True)

Parameters

• metric_names – List of metric names to log. If none - logs everything.

• log_on_batch_end – Logs per-batch metrics if set True.

• log_on_epoch_end – Logs per-epoch metrics if set True.

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_loader_end(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state)

on_stage_end(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.misc.EarlyStoppingCallback(patience: int, metric: str = 'loss', minimize: bool = True, min_delta: float = 1e-06)

Bases: catalyst.dl.core.callback.Callback

on_epoch_end(state: catalyst.dl.core.state.RunnerState) → None

class catalyst.dl.callbacks.misc.ConfusionMatrixCallback(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'confusion_matrix', version: str = 'tnt', class_names: List[str] = None, num_classes: int = None, plot_params: Dict = None)

Bases: catalyst.dl.core.callback.Callback

on_batch_end(state: catalyst.dl.core.state.RunnerState)

on_loader_end(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.misc.RaiseExceptionCallback

Bases: catalyst.dl.core.callback.LoggerCallback

on_exception(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.mixup.MixupCallback(fields: List[str] = ('features', ), alpha=1.0, on_train_only=True, **kwargs)

Bases: catalyst.dl.callbacks.criterion.CriterionCallback

Callback to do mixup augmentation.

Paper: https://arxiv.org/abs/1710.09412

Note

MixupCallback is inherited from CriterionCallback and does its work.

You may not use them together.

__init__(fields: List[str] = ('features', ), alpha=1.0, on_train_only=True, **kwargs)

Parameters

• fields (List[str]) – list of features which must be affected.

• alpha (float) – beta distribution a=b parameters. Must be >=0. The more alpha closer to zero the less effect of the mixup.

• on_train_only (bool) – Apply to train only. As the mixup use the proxy inputs, the targets are also proxy. We are not interested in them, are we? So, if on_train_only is True, use a standard output/metric for validation.

on_batch_start(state: catalyst.dl.core.state.RunnerState)

on_loader_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.optimizer.OptimizerCallback(grad_clip_params: Dict = None, accumulation_steps: int = 1, optimizer_key: str = None, loss_key: str = 'loss', decouple_weight_decay: bool = True)

Bases: catalyst.dl.core.callback.Callback

Optimizer callback, abstraction over optimizer step.

__init__(grad_clip_params: Dict = None, accumulation_steps: int = 1, optimizer_key: str = None, loss_key: str = 'loss', decouple_weight_decay: bool = True)

Parameters

• grad_clip_params (dict) – params for gradient clipping

• accumulation_steps (int) – number of steps before model.zero_grad()

• optimizer_key (str) – A key to take a optimizer in case there are several of them and they are in a dictionary format.

• loss_key (str) – key to get loss from state.loss

• decouple_weight_decay (bool) – If True - decouple weight decay regularization.

static grad_step(*, optimizer: torch.optim.optimizer.Optimizer, optimizer_wds: List[float] = 0, grad_clip_fn: Callable = None)

on_batch_end(state)

on_batch_start(state)

on_epoch_end(state)

on_epoch_start(state)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.scheduler.SchedulerCallback(scheduler_key: str = None, mode: str = None, reduce_metric: str = 'loss')

Bases: catalyst.dl.core.callback.Callback

on_batch_end(state)

on_epoch_end(state)

on_loader_start(state: catalyst.dl.core.state.RunnerState)

on_stage_start(state: catalyst.dl.core.state.RunnerState)

step(state: catalyst.dl.core.state.RunnerState)

class catalyst.dl.callbacks.scheduler.LRUpdater(optimizer_key: str = None)

Bases: catalyst.dl.core.callback.Callback

Basic class that all Lr updaters inherit from

__init__(optimizer_key: str = None)

Parameters

optimizer_key – which optimizer key to use for learning rate scheduling

calc_lr()

calc_momentum()

on_batch_end(state)

on_loader_start(state)

on_stage_start(state)

update_optimizer(state)

class catalyst.dl.callbacks.scheduler.LRFinder(final_lr, scale='log', num_steps=None, optimizer_key=None)

Bases: catalyst.dl.callbacks.scheduler.LRUpdater

Helps you find an optimal learning rate for a model, as per suggestion of 2015 CLR paper. Learning rate is increased in linear or log scale, depending on user input.

https://sgugger.github.io/how-do-you-find-a-good-learning-rate.html

__init__(final_lr, scale='log', num_steps=None, optimizer_key=None)

Parameters

• final_lr – final learning rate to try with

• scale – learning rate increasing scale (“log” or “linear”)

• num_steps – number of batches to try; if None - whole loader would be used.

• optimizer_key – which optimizer key to use for learning rate scheduling

calc_lr()

on_batch_end(state)

on_loader_start(state)

Metrics

class catalyst.dl.callbacks.metrics.accuracy.AccuracyCallback(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'accuracy', accuracy_args: List[int] = None, num_classes: int = None, threshold: float = None, activation: str = None)

Bases: catalyst.dl.core.callback.MultiMetricCallback

Accuracy metric callback.

It can be used either for

• multi-class task:

  -you can use accuracy_args. -threshold and activation are not required. -input_key point on tensor: batch_size. -output_key point on tensor: batch_size x num_classes.

• OR multi-label task, in this case:

  -you must specify threshold and activation. -accuracy_args and num_classes will not be used (because of there is no method to apply top-k in multi-label classification). -input_key, output_key point on tensor: batch_size x num_classes. -output_key point on a tensor with binary vectors.

There is no need to choose a type (multi-class/multi label). An appropriate type will be chosen automatically via shape of tensors.

__init__(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'accuracy', accuracy_args: List[int] = None, num_classes: int = None, threshold: float = None, activation: str = None)

Parameters

• input_key (str) – input key to use for accuracy calculation; specifies our y_true.

• output_key (str) – output key to use for accuracy calculation; specifies our y_pred.

• prefix (str) – key for the metric’s name

• accuracy_args (List[int]) – specifies which accuracy@K to log. [1] - accuracy [1, 3] - accuracy at 1 and 3 [1, 3, 5] - accuracy at 1, 3 and 5

• num_classes (int) – number of classes to calculate accuracy_args if accuracy_args is None

• threshold (float) – threshold for outputs binarization.

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax”].

class catalyst.dl.callbacks.metrics.accuracy.MapKCallback(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'map', map_args: List[int] = None, num_classes: int = None)

Bases: catalyst.dl.core.callback.MultiMetricCallback

mAP@k metric callback.

__init__(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'map', map_args: List[int] = None, num_classes: int = None)

Parameters

• input_key (str) – input key to use for calculation mean average accuracy at k; specifies our y_true.

• output_key (str) – output key to use for calculation mean average accuracy at k; specifies our y_pred.

• prefix (str) – key for the metric’s name

• map_args (List[int]) – specifies which map@K to log. [1] - map@1 [1, 3] - map@1 and map@3 [1, 3, 5] - map@1, map@3 and map@5

• num_classes (int) – number of classes to calculate map_args if map_args is None

class catalyst.dl.callbacks.metrics.auc.AUCCallback(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'auc', class_names: List[str] = None, num_classes: int = 2, activation: str = 'Sigmoid')

Bases: catalyst.dl.core.callback.MeterMetricsCallback

Calculates the AUC per class for each loader. Currently, supports binary and multi-label cases.

__init__(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'auc', class_names: List[str] = None, num_classes: int = 2, activation: str = 'Sigmoid')

Parameters

• input_key (str) – input key to use for auc calculation specifies our y_true.

• output_key (str) – output key to use for auc calculation; specifies our y_pred

• prefix (str) – name to display for auc when printing

• class_names (List[str]) – class names to display in the logs. If None, defaults to indices for each class, starting from 0.

• num_classes (int) – Number of classes; must be > 1

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [‘none’, ‘Sigmoid’, ‘Softmax2d’]

class catalyst.dl.callbacks.metrics.dice.DiceCallback(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'dice', eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Bases: catalyst.dl.core.callback.MetricCallback

Dice metric callback.

__init__(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'dice', eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Parameters

• input_key – input key to use for dice calculation; specifies our y_true.

• output_key – output key to use for dice calculation; specifies our y_pred.

class catalyst.dl.callbacks.metrics.f1_score.F1ScoreCallback(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'f1_score', beta: float = 1.0, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Bases: catalyst.dl.core.callback.MetricCallback

F1 score metric callback.

__init__(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'f1_score', beta: float = 1.0, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Parameters

• input_key (str) – input key to use for iou calculation specifies our y_true.

• output_key (str) – output key to use for iou calculation; specifies our y_pred

• prefix (str) – key to store in logs

• beta (float) – beta param for f_score

• eps (float) – epsilon to avoid zero division

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [‘none’, ‘Sigmoid’, ‘Softmax2d’]

class catalyst.dl.callbacks.metrics.iou.IouCallback(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'iou', eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Bases: catalyst.dl.core.callback.MetricCallback

IoU (Jaccard) metric callback.

__init__(input_key: str = 'targets', output_key: str = 'logits', prefix: str = 'iou', eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Parameters

• input_key (str) – input key to use for iou calculation specifies our y_true.

• output_key (str) – output key to use for iou calculation; specifies our y_pred

• prefix (str) – key to store in logs

• eps (float) – epsilon to avoid zero division

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [‘none’, ‘Sigmoid’, ‘Softmax2d’]

catalyst.dl.callbacks.metrics.iou.JaccardCallback

alias of catalyst.dl.callbacks.metrics.iou.IouCallback

Experiment

class catalyst.dl.experiment.base.BaseExperiment(model: torch.nn.modules.module.Module, loaders: OrderedDict[str, DataLoader], callbacks: Union[OrderedDict[str, Callback], List[Callback]] = None, logdir: str = None, stage: str = 'train', criterion: torch.nn.modules.module.Module = None, optimizer: torch.optim.optimizer.Optimizer = None, scheduler: torch.optim.lr_scheduler._LRScheduler = None, num_epochs: int = 1, valid_loader: str = 'valid', main_metric: str = 'loss', minimize_metric: bool = True, verbose: bool = False, state_kwargs: Dict = None, checkpoint_data: Dict = None, distributed_params: Dict = None, monitoring_params: Dict = None, initial_seed: int = 42)

Bases: catalyst.dl.core.experiment.Experiment

Super-simple one-staged experiment

you can use to declare experiment in code

property distributed_params

get_callbacks(stage: str) → OrderedDict[str, Callback]

get_criterion(stage: str) → torch.nn.modules.module.Module

get_loaders(stage: str) → OrderedDict[str, DataLoader]

get_model(stage: str) → torch.nn.modules.module.Module

get_optimizer(stage: str, model: torch.nn.modules.module.Module) → torch.optim.optimizer.Optimizer

get_scheduler(stage: str, optimizer=None) → torch.optim.lr_scheduler._LRScheduler

get_state_params(stage: str) → Mapping[str, Any]

property initial_seed

property logdir

property monitoring_params

property stages

class catalyst.dl.experiment.config.ConfigExperiment(config: Dict)

Bases: catalyst.dl.core.experiment.Experiment

STAGE_KEYWORDS = ['criterion_params', 'optimizer_params', 'scheduler_params', 'data_params', 'state_params', 'callbacks_params']

property distributed_params

get_callbacks(stage: str) → OrderedDict[Callback]

get_criterion(stage: str) → torch.nn.modules.module.Module

get_loaders(stage: str) → OrderedDict[str, DataLoader]

get_model(stage: str)

get_optimizer(stage: str, model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]]) → Union[torch.optim.optimizer.Optimizer, Dict[str, torch.optim.optimizer.Optimizer]]

get_scheduler(stage: str, optimizer) → torch.optim.lr_scheduler._LRScheduler

get_state_params(stage: str) → Mapping[str, Any]

property initial_seed

property logdir

property monitoring_params

property stages

class catalyst.dl.experiment.supervised.SupervisedExperiment(model: torch.nn.modules.module.Module, loaders: OrderedDict[str, DataLoader], callbacks: Union[OrderedDict[str, Callback], List[Callback]] = None, logdir: str = None, stage: str = 'train', criterion: torch.nn.modules.module.Module = None, optimizer: torch.optim.optimizer.Optimizer = None, scheduler: torch.optim.lr_scheduler._LRScheduler = None, num_epochs: int = 1, valid_loader: str = 'valid', main_metric: str = 'loss', minimize_metric: bool = True, verbose: bool = False, state_kwargs: Dict = None, checkpoint_data: Dict = None, distributed_params: Dict = None, monitoring_params: Dict = None, initial_seed: int = 42)

Bases: catalyst.dl.experiment.base.BaseExperiment

get_callbacks(stage: str) → OrderedDict[str, Callback]

Runner

class catalyst.dl.runner.supervised.SupervisedRunner(model: torch.nn.modules.module.Module = None, device=None, input_key: str = 'features', output_key: str = 'logits', input_target_key: str = 'targets')

Bases: catalyst.dl.core.runner.Runner

Runner for experiments with supervised model

__init__(model: torch.nn.modules.module.Module = None, device=None, input_key: str = 'features', output_key: str = 'logits', input_target_key: str = 'targets')

@TODO update docs :param input_key: Key in batch dict mapping to model input :param output_key: Key in output dict model output will be stored under

forward(batch)

Should not be called directly outside of runner. If your model has specific interface, override this method to use it

infer(model: torch.nn.modules.module.Module, loaders: OrderedDict[str, DataLoader], callbacks: Union[List[Callback], OrderedDict[str, Callback]] = None, verbose: bool = False, state_kwargs: Dict = None, fp16: Union[Dict, bool] = None, check: bool = False) → None

Makes the inference on the model.

Parameters

• model (torch.nn.Module) – model to infer

• loaders (dict) – dictionary containing one or several torch.utils.data.DataLoader for inference

• callbacks (List[catalyst.dl.Callback]) – list of inference callbacks

• verbose (bool) – ff true, it displays the status of the inference to the console.

• state_kwargs (dict) – additional state params to RunnerState

• fp16 (Union[Dict, bool]) – If not None, then sets inference to FP16. See https://nvidia.github.io/apex/amp.html#properties if fp16=True, params by default will be {"opt_level": "O1"}

• check (bool) – if True, then only checks that pipeline is working (3 epochs only)

predict_loader(model: torch.nn.modules.module.Module, loader: torch.utils.data.dataloader.DataLoader, resume: str = None, verbose: bool = False, state_kwargs: Dict = None, fp16: Union[Dict, bool] = None, check: bool = False) → Any

Makes a prediction on the whole loader with the specified model.

Parameters

• model (torch.nn.Module) – model to infer

• loader (DataLoader) – dictionary containing only one torch.utils.data.DataLoader for inference

• resume (str) – path to checkpoint for model

• verbose (bool) – ff true, it displays the status of the inference to the console.

• state_kwargs (dict) – additional state params to RunnerState

• fp16 (Union[Dict, bool]) – If not None, then sets inference to FP16. See https://nvidia.github.io/apex/amp.html#properties if fp16=True, params by default will be {"opt_level": "O1"}

• check (bool) – if True, then only checks that pipeline is working (3 epochs only)

train(model: torch.nn.modules.module.Module, criterion: torch.nn.modules.module.Module, optimizer: torch.optim.optimizer.Optimizer, loaders: OrderedDict[str, DataLoader], logdir: str, callbacks: Union[List[Callback], OrderedDict[str, Callback]] = None, scheduler: torch.optim.lr_scheduler._LRScheduler = None, resume: str = None, num_epochs: int = 1, valid_loader: str = 'valid', main_metric: str = 'loss', minimize_metric: bool = True, verbose: bool = False, state_kwargs: Dict = None, checkpoint_data: Dict = None, fp16: Union[Dict, bool] = None, monitoring_params: Dict = None, check: bool = False) → None

Starts the training process of the model.

Parameters

• model (torch.nn.Module) – model to train

• criterion (nn.Module) – criterion function for training

• optimizer (optim.Optimizer) – optimizer for training

• loaders (dict) – dictionary containing one or several torch.utils.data.DataLoader for training and validation

• logdir (str) – path to output directory

• callbacks (List[catalyst.dl.Callback]) – list of callbacks

• scheduler (optim.lr_scheduler._LRScheduler) – scheduler for training

• resume (str) – path to checkpoint for model

• num_epochs (int) – number of training epochs

• valid_loader (str) – loader name used to calculate the metrics and save the checkpoints. For example, you can pass train and then the metrics will be taken from train loader.

• main_metric (str) – the key to the name of the metric by which the checkpoints will be selected.

• minimize_metric (bool) – flag to indicate whether the main_metric should be minimized.

• verbose (bool) – ff true, it displays the status of the training to the console.

• state_kwargs (dict) – additional state params to RunnerState

• checkpoint_data (dict) – additional data to save in checkpoint, for example: class_names, date_of_training, etc

• fp16 (Union[Dict, bool]) – If not None, then sets training to FP16. See https://nvidia.github.io/apex/amp.html#properties if fp16=True, params by default will be {"opt_level": "O1"}

• monitoring_params (dict) – If not None, then create monitoring through Weights&Biases. This params is used for wandb.init see https://docs.wandb.com/wandb/init

• check (bool) – if True, then only checks that pipeline is working (3 epochs only)

Utils

class catalyst.dl.utils.formatters.MetricsFormatter(message_prefix)

Bases: abc.ABC, logging.Formatter

__init__(message_prefix)

Parameters

message_prefix – logging format string that will be prepended to message

format(record: logging.LogRecord)

class catalyst.dl.utils.formatters.TxtMetricsFormatter

Bases: catalyst.dl.utils.formatters.MetricsFormatter

Translate batch metrics in human-readable format.

This class is used by logging.Logger to make a string from record. For details refer to official docs for ‘logging’ module.

Note:

This is inner class used by Logger callback, no need to use it directly!

class catalyst.dl.utils.formatters.JsonMetricsFormatter

Bases: catalyst.dl.utils.formatters.MetricsFormatter

Translate batch metrics in json format.

This class is used by logging.Logger to make a string from record. For details refer to official docs for ‘logging’ module.

Note

This is inner class used by Logger callback, no need to use it directly!

catalyst.dl.utils.scripts.import_experiment_and_runner(expdir: pathlib.Path)

catalyst.dl.utils.scripts.dump_base_experiment_code(src: pathlib.Path, dst: pathlib.Path)

catalyst.dl.utils.torch.process_components(model: torch.nn.modules.module.Module, criterion: torch.nn.modules.module.Module = None, optimizer: torch.optim.optimizer.Optimizer = None, scheduler: torch.optim.lr_scheduler._LRScheduler = None, distributed_params: Dict = None, device: Union[str, torch.device] = None) → Tuple[torch.nn.modules.module.Module, torch.nn.modules.module.Module, torch.optim.optimizer.Optimizer, torch.optim.lr_scheduler._LRScheduler, torch.device]

catalyst.dl.utils.torch.get_loader(data_source, open_fn, dict_transform=None, dataset_cache_prob=-1, sampler=None, collate_fn=<function default_collate>, batch_size=32, num_workers=4, shuffle=False, drop_last=False)

catalyst.dl.utils.trace.trace_model(model: torch.nn.modules.module.Module, experiment: catalyst.dl.core.experiment.Experiment, runner_type: Type[catalyst.dl.core.runner.Runner], method_name: str = 'forward', mode: str = 'eval', requires_grad: bool = False) → torch.jit.ScriptModule

Traces model using it’s native experiment and runner.

Parameters

• model – Model to trace

• experiment – Native experiment that was used to train model

• runner_type – Model’s native runner that was used to train model

• method_name (str) – Model’s method name that will be used as entrypoint during tracing

• mode (str) – Mode for model to trace (train or eval)

• requires_grad (bool) – Flag to use grads

Returns

Traced model ScriptModule

catalyst.dl.utils.visualization.plot_metrics(logdir: Union[str, pathlib.Path], step: Optional[str] = 'epoch', metrics: Optional[List[str]] = None, height: Optional[int] = None, width: Optional[int] = None) → None

Plots your learning results.

Parameters

• logdir – the logdir that was specified during training.

• step – ‘batch’ or ‘epoch’ - what logs to show: for batches or for epochs

• metrics – list of metrics to plot. The loss should be specified as ‘loss’, learning rate = ‘_base/lr’ and other metrics should be specified as names in metrics dict that was specified during training

• height – the height of the whole resulting plot

• width – the width of the whole resulting plot

Criterion

catalyst.dl.utils.criterion.accuracy.accuracy(outputs, targets, topk=(1, ), threshold: float = None, activation: str = None)

Computes the accuracy.

It can be used either for:

• multi-class task:

  -you can use topk. -threshold and activation are not required. -targets is a tensor: batch_size -outputs is a tensor: batch_size x num_classes -computes the accuracy@k for the specified values of k.

• OR multi-label task, in this case:

  -you must specify threshold and activation -topk will not be used (because of there is no method to apply top-k in multi-label classification). -outputs, targets are tensors with shape: batch_size x num_classes -targets is a tensor with binary vectors

catalyst.dl.utils.criterion.accuracy.average_accuracy(outputs, targets, k=10)

Computes the average accuracy at k. This function computes the average accuracy at k between two lists of items.

Parameters

• outputs (list) – A list of predicted elements

• targets (list) – A list of elements that are to be predicted

• k (int, optional) – The maximum number of predicted elements

Returns

The average accuracy at k over the input lists

Return type

double

catalyst.dl.utils.criterion.accuracy.mean_average_accuracy(outputs, targets, topk=(1, ))

Computes the mean average accuracy at k. This function computes the mean average accuracy at k between two lists of lists of items.

Parameters

• outputs (list) – A list of lists of predicted elements

• targets (list) – A list of lists of elements that are to be predicted

• topk (int, optional) – The maximum number of predicted elements

Returns

The mean average accuracy at k over the input lists

Return type

double

catalyst.dl.utils.criterion.dice.dice(outputs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Computes the dice metric

Parameters

• outputs (list) – A list of predicted elements

• targets (list) – A list of elements that are to be predicted

• eps (float) – epsilon

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

Dice score

Return type

double

catalyst.dl.utils.criterion.f1_score.f1_score(outputs: torch.Tensor, targets: torch.Tensor, beta: float = 1.0, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Source https://github.com/qubvel/segmentation_models.pytorch

Parameters

• outputs (torch.Tensor) – A list of predicted elements

• targets (torch.Tensor) – A list of elements that are to be predicted

• eps (float) – epsilon to avoid zero division

• beta (float) – beta param for f_score

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

F_1 score

Return type

float

catalyst.dl.utils.criterion.focal.sigmoid_focal_loss(outputs: torch.Tensor, targets: torch.Tensor, gamma: float = 2.0, alpha: float = 0.25, reduction: str = 'mean')

Compute binary focal loss between target and output logits.

Source https://github.com/BloodAxe/pytorch-toolbelt See losses for details.

Parameters

• outputs – Tensor of arbitrary shape

• targets – Tensor of the same shape as input

• reduction (string, optional) – Specifies the reduction to apply to the output: “none” | “mean” | “sum” | “batchwise_mean”. “none”: no reduction will be applied, “mean”: the sum of the output will be divided by the number of elements in the output, “sum”: the output will be summed.

See https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/loss/losses.py # noqa: E501

catalyst.dl.utils.criterion.focal.reduced_focal_loss(outputs: torch.Tensor, targets: torch.Tensor, threshold: float = 0.5, gamma: float = 2.0, reduction='mean')

Compute reduced focal loss between target and output logits.

Source https://github.com/BloodAxe/pytorch-toolbelt See losses for details.

Parameters

• outputs – Tensor of arbitrary shape

• targets – Tensor of the same shape as input

• reduction (string, optional) –

  Specifies the reduction to apply to the output: “none” | “mean” | “sum” | “batchwise_mean”.

  ”none”: no reduction will be applied, “mean”: the sum of the output will be divided by the number of elements in the output, “sum”: the output will be summed.

  Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction.

  ”batchwise_mean” computes mean loss per sample in batch. Default: “mean”

See https://arxiv.org/abs/1903.01347

catalyst.dl.utils.criterion.iou.iou(outputs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Parameters

• outputs (torch.Tensor) – A list of predicted elements

• targets (torch.Tensor) – A list of elements that are to be predicted

• eps (float) – epsilon to avoid zero division

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

IoU (Jaccard) score

Return type

float

catalyst.dl.utils.criterion.iou.jaccard(outputs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Parameters

• outputs (torch.Tensor) – A list of predicted elements

• targets (torch.Tensor) – A list of elements that are to be predicted

• eps (float) – epsilon to avoid zero division

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

IoU (Jaccard) score

Return type

float

Meters

The meters from torchnet.meters

class catalyst.dl.meters.meter.Meter

Bases: object

Meters provide a way to keep track of important statistics in an online manner.

This class is abstract, but provides a standard interface for all meters to follow.

add(value)

Log a new value to the meter

Parameters

value – Next restult to include.

reset()

Resets the meter to default settings.

value()

Get the value of the meter in the current state.

class catalyst.dl.meters.apmeter.APMeter

Bases: catalyst.dl.meters.meter.Meter

The APMeter measures the average precision per class.

The APMeter is designed to operate on NxK Tensors output and target, and optionally a Nx1 Tensor weight where (1) the output contains model output scores for N examples and K classes that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a sigmoid function); (2) the target contains only values 0 (for negative examples) and 1 (for positive examples); and (3) the weight ( > 0) represents weight for each sample.

add(output, target, weight=None)

Add a new observation

Parameters

• output (Tensor) – NxK tensor that for each of the N examples indicates the probability of the example belonging to each of the K classes, according to the model. The probabilities should sum to one over all classes

• target (Tensor) – binary NxK tensort that encodes which of the K classes are associated with the N-th input (eg: a row [0, 1, 0, 1] indicates that the example is associated with classes 2 and 4)

• weight (optional, Tensor) – Nx1 tensor representing the weight for each example (each weight > 0)

reset()

Resets the meter with empty member variables

value()

Returns the model”s average precision for each class

Returns

1xK tensor, with avg precision for each class k

Return type

ap (FloatTensor)

class catalyst.dl.meters.aucmeter.AUCMeter

Bases: catalyst.dl.meters.meter.Meter

The AUCMeter measures the area under the receiver-operating characteristic (ROC) curve for binary classification problems. The area under the curve (AUC) can be interpreted as the probability that, given a randomly selected positive example and a randomly selected negative example, the positive example is assigned a higher score by the classification model than the negative example.

The AUCMeter is designed to operate on one-dimensional Tensors output and target, where (1) the output contains model output scores that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a signoid function); and (2) the target contains only values 0 (for negative examples) and 1 (for positive examples).

add(output, target)

reset()

value()

class catalyst.dl.meters.averagevaluemeter.AverageValueMeter

Bases: catalyst.dl.meters.meter.Meter

add(value, n=1)

reset()

value()

class catalyst.dl.meters.classerrormeter.ClassErrorMeter(topk=[1], accuracy=False)

Bases: catalyst.dl.meters.meter.Meter

add(output, target)

reset()

value(k=-1)

class catalyst.dl.meters.confusionmeter.ConfusionMeter(k, normalized=False)

Bases: catalyst.dl.meters.meter.Meter

Maintains a confusion matrix for a given calssification problem.

The ConfusionMeter constructs a confusion matrix for a multi-class classification problems. It does not support multi-label, multi-class problems: for such problems, please use MultiLabelConfusionMeter.

Parameters

• k (int) – number of classes in the classification problem

• normalized (boolean) – Determines whether or not the confusion matrix is normalized or not

add(predicted, target)

Computes the confusion matrix of K x K size where K is no of classes

Args: predicted (tensor): Can be an N x K tensor of predicted scores obtained from the model for N examples and K classes or an N-tensor of integer values between 0 and K-1. target (tensor): Can be a N-tensor of integer values assumed to be integer values between 0 and K-1 or N x K tensor, where targets are assumed to be provided as one-hot vectors

reset()

value()

Returns

Confustion matrix of K rows and K columns, where rows corresponds to ground-truth targets and columns corresponds to predicted targets.

class catalyst.dl.meters.mapmeter.mAPMeter

Bases: catalyst.dl.meters.meter.Meter

The mAPMeter measures the mean average precision over all classes.

The mAPMeter is designed to operate on NxK Tensors output and target, and optionally a Nx1 Tensor weight where (1) the output contains model output scores for N examples and K classes that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a sigmoid function); (2) the target contains only values 0 (for negative examples) and 1 (for positive examples); and (3) the weight ( > 0) represents weight for each sample.

add(output, target, weight=None)

reset()

value()

class catalyst.dl.meters.movingaveragevaluemeter.MovingAverageValueMeter(windowsize)

Bases: catalyst.dl.meters.meter.Meter

add(value)

reset()

value()

class catalyst.dl.meters.msemeter.MSEMeter(root=False)

Bases: catalyst.dl.meters.meter.Meter

add(output, target)

reset()

value()