Core¶
-
class
catalyst.core.experiment._Experiment[source]¶ Bases:
abc.ABCObject containing all information required to run the experiment
Abstract, look for implementations
-
abstract property
distributed_params¶ Dict with the parameters for distributed and FP16 methond
-
abstract
get_callbacks(stage: str) → OrderedDict[str, Callback][source]¶ Returns the callbacks for a given stage
-
abstract
get_criterion(stage: str) → torch.nn.modules.module.Module[source]¶ Returns the criterion for a given stage
-
get_datasets(stage: str, epoch: int = None, **kwargs) → OrderedDict[str, Dataset][source]¶ Returns the datasets for a given stage and kwargs
-
get_experiment_components(model: torch.nn.modules.module.Module, stage: str) → Tuple[torch.nn.modules.module.Module, torch.optim.optimizer.Optimizer, torch.optim.lr_scheduler._LRScheduler][source]¶ Returns the tuple containing criterion, optimizer and scheduler by giving model and stage.
-
abstract
get_loaders(stage: str, epoch: int = None) → OrderedDict[str, DataLoader][source]¶ Returns the loaders for a given stage
-
abstract
get_model(stage: str) → torch.nn.modules.module.Module[source]¶ Returns the model for a given stage
-
abstract
get_optimizer(stage: str, model: torch.nn.modules.module.Module) → torch.optim.optimizer.Optimizer[source]¶ Returns the optimizer for a given stage
-
abstract
get_scheduler(stage: str, optimizer: torch.optim.optimizer.Optimizer) → torch.optim.lr_scheduler._LRScheduler[source]¶ Returns the scheduler for a given stage
-
abstract
get_state_params(stage: str) → Mapping[str, Any][source]¶ Returns the state parameters for a given stage
-
get_transforms(stage: str = None, dataset: str = None)[source]¶ Returns the data transforms for a given stage and mode
-
abstract property
initial_seed¶ Experiment’s initial seed value
-
abstract property
logdir¶ Path to the directory where the experiment logs
-
abstract property
monitoring_params¶ Dict with the parameters for monitoring services
-
abstract property
stages¶ Experiment’s stage names
-
abstract property
-
class
catalyst.core.runner._Runner(model: torch.nn.modules.module.Module = None, device: Union[str, torch.device] = None)[source]¶ Bases:
abc.ABCAbstract class for all runners inherited from
-
__init__(model: torch.nn.modules.module.Module = None, device: Union[str, torch.device] = None)[source]¶ - Parameters
model (Model) – Torch model object
device (Device) – Torch device
-
property
device¶ Returns the runner’s device instance
-
abstract
forward(batch: Mapping[str, Any], **kwargs) → Mapping[str, Any][source]¶ Forward method for your Runner
- Parameters
batch – Key-value batch items
**kwargs – kwargs to pass to the model
-
property
model¶ Returns the runner’s model instance
-
predict_batch(batch: Mapping[str, Any], **kwargs) → Mapping[str, Any][source]¶ Run model for a batch of elements WARN: You should not override this method. If you need specific model call, override forward() method :param batch: Key-value batch items :param **kwargs: kwargs to pass to the model
- Returns
model output key-value
-
-
class
catalyst.core.state.State(*, device: Union[str, torch.device] = None, model: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]] = None, criterion: Union[torch.nn.modules.module.Module, Dict[str, torch.nn.modules.module.Module]] = None, optimizer: Union[torch.optim.optimizer.Optimizer, Dict[str, torch.optim.optimizer.Optimizer]] = None, scheduler: Union[torch.optim.lr_scheduler._LRScheduler, Dict[str, torch.optim.lr_scheduler._LRScheduler]] = None, callbacks: Dict[str, Callback] = None, logdir: str = None, stage: str = 'infer', num_epochs: int = None, main_metric: str = 'loss', minimize_metric: bool = True, valid_loader: str = 'valid', checkpoint_data: Dict = None, is_check_run: bool = False, **kwargs)[source]¶ Bases:
catalyst.utils.tools.frozen_class.FrozenClassObject containing all information about current state of the experiment.
- state.loaders - ordered dictionary with torch.DataLoaders
“train” prefix is used for training loaders (metrics computations, backward pass, optimization)
“valid” prefix is used for validation loaders - metrics only
“infer” prefix is used for inference loaders - dataset prediction
state.loaders = { "train": MnistTrainLoader(), "valid": MnistValidLoader() }
- state.model - an instance of torch.nn.Module class
should implement
forwardmethodstate.model = torch.nn.Linear(10, 10)
- state.criterion - an instance of torch.nn.Module class or torch.nn.modules.loss._Loss
should implement
forwardmethodstate.criterion = torch.nn.CrossEntropyLoss()
- state.optimizer - an instance of torch.optim.optimizer.Optimizer
should implement
stepmethodstate.optimizer = torch.optim.Adam()
- state.scheduler - an instance of torch.optim.lr_scheduler._LRScheduler
should implement
stepmethodstate.scheduler = htorch.optim.lr_scheduler.ReduceLROnPlateau()
- state.device - an instance of torch.device (CPU, GPU, TPU)
state.device = torch.device("cpu")
- state.callbacks - ordered dictionary with Catalyst.Callback instances
state.callbacks = { "accuracy": AccuracyCallback(), "criterion": CriterionCallback(), "optim": OptimizerCallback(), "saver": CheckpointCallback() }
- state.batch_in - dictionary, containing current batch of data from DataLoader
state.batch_in = { "images": np.ndarray(batch_size, c, h, w), "targets": np.ndarray(batch_size, 1), }
- state.batch_out - dictionary, containing model output based on current batch
state.batch_out = {"logits": torch.Tensor(batch_size, num_classes)}
- state.batch_metrics - dictionary, flatten storage for batch metrics
state.batch_metrics = {"loss": ..., "accuracy": ..., "iou": ...}
- state.loader_metrics - dictionary with aggregated batch statistics for loader (mean over all batches) and global loader metrics, like AUC
state.loader_metrics = {"loss": ..., "accuracy": ..., "auc": ...}
- state.epoch_metrics - dictionary with summarized metrics for different loaders and global epoch metrics, like lr, momentum
state.epoch_metrics = { "train_loss": ..., "train_auc": ..., "valid_loss": ..., "lr": ..., "momentum": ..., }
- state.is_best_valid - bool, indicator flag
Trueif this training epoch is best over all epochsFalseif not
- state.valid_metrics - dictionary with validation metrics for currect epoch
just a subdictionary of epoch_metrics
state.valid_metrics = {"loss": ..., "accuracy": ..., "auc": ...}
state.best_valid_metrics - dictionary with best validation metrics during whole training process
state.distributed_rank
state.is_distributed_worker
state.stage_name
state.epoch
state.num_epochs
state.loader_name
state.loader_step
state.loader_len
state.batch_size
state.global_step
state.global_epoch
state.main_metric
state.minimize_metric
state.valid_loader
- state.logdir - path to logging directory to save
all logs, metrics, checkpoints and artifacts
- state.checkpoint_data - dictionary
with all extra data for experiment tracking
- state.is_check_run - bool, indicator flag
Trueif you want to check you pipeline and run only 2 batches per loader and 2 epochs per stageFalse(default) if you want to just the pipeline
- state.need_backward_pass - bool, indicator flag
Truefor training loadersFalseotherwise
- state.need_early_stop - bool, indicator flag
used for EarlyStopping and CheckRun Callbacks
Trueif we need to stop the trainingFalse(default) otherwise
- state.need_exception_reraise - bool, indicator flag
True(default) if you want to show exception during pipeline and stop the training processFalseotherwise
- state.exception - python Exception instance to raise
(or not ;) )
-
property
input¶
-
property
need_backward_pass¶
-
property
output¶
-
class
catalyst.core.callback.Callback(order: int, node: int = <CallbackNode.All: 0>, scope: int = <CallbackScope.Stage: 0>)[source]¶ Bases:
objectAbstract class that all callback (e.g., Logger) classes extends from. Must be extended before usage.
usage example:
-- stage start ---- epoch start (one epoch - one run of every loader) ------ loader start -------- batch start -------- batch handler -------- batch end ------ loader end ---- epoch end -- stage end exception – if an Exception was raised
All callbacks has
ordervalue fromCallbackOrderandnodevalue fromCallbackNode
-
class
catalyst.core.callback.CallbackNode[source]¶ Bases:
enum.IntFlagAn enumeration.
-
All= 0¶
-
Master= 1¶
-
Worker= 2¶
-
-
class
catalyst.core.callback.CallbackOrder[source]¶ Bases:
enum.IntFlagAn enumeration.
-
External= 200¶
-
Internal= 0¶
-
Logging= 120¶
-
Metric= 20¶
-
MetricAggregation= 40¶
-
Optimizer= 60¶
-
Scheduler= 100¶
-
Validation= 80¶
-
-
class
catalyst.core.callback.CallbackScope[source]¶ Bases:
enum.IntFlagAn enumeration.
-
Experiment= 1¶
-
Stage= 0¶
-
Callbacks¶
-
class
catalyst.core.callbacks.checkpoint.CheckpointCallback(save_n_best: int = 1, resume: str = None, resume_dir: str = None, metrics_filename: str = '_metrics.json')[source]¶ Bases:
catalyst.core.callbacks.checkpoint.BaseCheckpointCallbackCheckpoint callback to save/restore your model/criterion/optimizer/metrics.
-
__init__(save_n_best: int = 1, resume: str = None, resume_dir: str = None, metrics_filename: str = '_metrics.json')[source]¶ - Parameters
save_n_best (int) – number of best checkpoint to keep
resume (str) – path to checkpoint to load and initialize runner state
metrics_filename (str) – filename to save metrics in checkpoint folder. Must ends on
.jsonor.yml
-
-
class
catalyst.core.callbacks.checkpoint.IterationCheckpointCallback(save_n_last: int = 1, period: int = 100, stage_restart: bool = True, metrics_filename: str = '_metrics_iter.json')[source]¶ Bases:
catalyst.core.callbacks.checkpoint.BaseCheckpointCallbackIteration checkpoint callback to save your model/criterion/optimizer
-
__init__(save_n_last: int = 1, period: int = 100, stage_restart: bool = True, metrics_filename: str = '_metrics_iter.json')[source]¶ - Parameters
save_n_last (int) – number of last checkpoint to keep
period (int) – save the checkpoint every period
stage_restart (bool) – restart counter every stage or not
metrics_filename (str) – filename to save metrics in checkpoint folder. Must ends on
.jsonor.yml
-
-
class
catalyst.core.callbacks.criterion.CriterionCallback(input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', prefix: str = 'loss', criterion_key: str = None, multiplier: float = 1.0, **metric_kwargs)[source]¶ Bases:
catalyst.core.callbacks.metrics._MetricCallbackCallback for that measures loss with specified criterion.
-
__init__(input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', prefix: str = 'loss', criterion_key: str = None, multiplier: float = 1.0, **metric_kwargs)[source]¶ - Parameters
input_key (Union[str, List[str], Dict[str, str]]) – key/list/dict of keys that takes values from the input dictionary If ‘__all__’, the whole input will be passed to the criterion If None, empty dict will be passed to the criterion.
output_key (Union[str, List[str], Dict[str, str]]) – key/list/dict of keys that takes values from the input dictionary If ‘__all__’, the whole output will be passed to the criterion If None, empty dict will be passed to the criterion.
prefix (str) – prefix for metrics and output key for loss in
state.batch_metricsdictionarycriterion_key (str) – A key to take a criterion in case there are several of them and they are in a dictionary format.
multiplier (float) – scale factor for the output loss.
-
property
metric_fn¶
-
-
class
catalyst.core.callbacks.early_stop.CheckRunCallback(num_batch_steps: int = 2, num_epoch_steps: int = 2)[source]¶
-
class
catalyst.core.callbacks.early_stop.EarlyStoppingCallback(patience: int, metric: str = 'loss', minimize: bool = True, min_delta: float = 1e-06)[source]¶
-
class
catalyst.core.callbacks.logging.ConsoleLogger[source]¶ Bases:
catalyst.core.callback.CallbackLogger callback, translates
state.*_metricsto console and text file
-
class
catalyst.core.callbacks.logging.TensorboardLogger(metric_names: List[str] = None, log_on_batch_end: bool = True, log_on_epoch_end: bool = True)[source]¶ Bases:
catalyst.core.callback.CallbackLogger callback, translates
state.metric_managerto tensorboard-
__init__(metric_names: List[str] = None, log_on_batch_end: bool = True, log_on_epoch_end: bool = True)[source]¶ - Parameters
metric_names (List[str]) – list of metric names to log, if none - logs everything
log_on_batch_end (bool) – logs per-batch metrics if set True
log_on_epoch_end (bool) – logs per-epoch metrics if set True
-
-
class
catalyst.core.callbacks.logging.VerboseLogger(always_show: List[str] = None, never_show: List[str] = None)[source]¶ Bases:
catalyst.core.callback.CallbackLogs the params into console
-
__init__(always_show: List[str] = None, never_show: List[str] = None)[source]¶ - Parameters
always_show (List[str]) – list of metrics to always show if None default is
["_timer/_fps"]to remove always_show metrics set it to an empty list[]never_show (List[str]) – list of metrics which will not be shown
-
-
class
catalyst.core.callbacks.metrics._MetricCallback(prefix: str, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metrics_kwargs)[source]¶ Bases:
abc.ABC,catalyst.core.callback.Callback-
abstract property
metric_fn¶
-
abstract property
-
class
catalyst.core.callbacks.metrics.MetricCallback(prefix: str, metric_fn: Callable, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metric_kwargs)[source]¶ Bases:
catalyst.core.callbacks.metrics._MetricCallbackA callback that returns single metric on state.on_batch_end
-
property
metric_fn¶
-
property
-
class
catalyst.core.callbacks.metrics.MultiMetricCallback(prefix: str, metric_fn: Callable, list_args: List, input_key: Union[str, List[str], Dict[str, str]] = 'targets', output_key: Union[str, List[str], Dict[str, str]] = 'logits', multiplier: float = 1.0, **metrics_kwargs)[source]¶ Bases:
catalyst.core.callbacks.metrics.MetricCallbackA callback that returns multiple metrics on state.on_batch_end
-
class
catalyst.core.callbacks.metrics.MetricAggregationCallback(prefix: str, metrics: Union[str, List[str], Dict[str, float]] = None, mode: str = 'mean', multiplier: float = 1.0)[source]¶ Bases:
catalyst.core.callback.CallbackA callback to aggregate several metrics in one value.
-
__init__(prefix: str, metrics: Union[str, List[str], Dict[str, float]] = None, mode: str = 'mean', multiplier: float = 1.0) → None[source]¶ - Parameters
prefix (str) – new key for aggregated metric.
metrics (Union[str, List[str], Dict[str, float]]) – If not None, it aggregates only the values from the metric by these keys. for
weighted_sumaggregation it must be a Dict[str, float].mode (str) – function for aggregation. Must be either
sum,meanorweighted_sum.multiplier (float) – scale factor for the aggregated metric.
-
-
class
catalyst.core.callbacks.metrics.MetricManagerCallback[source]¶ Bases:
catalyst.core.callback.CallbackPrepares metrics for logging, transferring values from PyTorch to numpy
-
class
catalyst.core.callbacks.optimizer.OptimizerCallback(loss_key: str = 'loss', optimizer_key: str = None, accumulation_steps: int = 1, grad_clip_params: Dict = None, decouple_weight_decay: bool = True)[source]¶ Bases:
catalyst.core.callback.CallbackOptimizer callback, abstraction over optimizer step.
-
__init__(loss_key: str = 'loss', optimizer_key: str = None, accumulation_steps: int = 1, grad_clip_params: Dict = None, decouple_weight_decay: bool = True)[source]¶ - Parameters
grad_clip_params (dict) – params for gradient clipping
accumulation_steps (int) – number of steps before
model.zero_grad()optimizer_key (str) – A key to take a optimizer in case there are several of them and they are in a dictionary format.
loss_key (str) – key to get loss from
state.lossdecouple_weight_decay (bool) – If True - decouple weight decay regularization.
save_model_grads (#) – If True - State.model_grads will
contain gradients calculated (#) –
on backward propagation on current (#) –
batch (#) –
-
static
grad_step(*, optimizer: torch.optim.optimizer.Optimizer, optimizer_wds: List[float] = 0, grad_clip_fn: Callable = None)[source]¶ Makes a gradient step for a given optimizer
- Parameters
optimizer (Optimizer) – the optimizer
optimizer_wds (List[float]) – list of weight decay parameters for each param group
grad_clip_fn (Callable) – function for gradient clipping
-
-
class
catalyst.core.callbacks.scheduler.SchedulerCallback(scheduler_key: str = None, mode: str = None, reduced_metric: str = None)[source]¶
-
class
catalyst.core.callbacks.scheduler.LRUpdater(optimizer_key: str = None)[source]¶ Bases:
catalyst.core.callback.CallbackBasic class that all Lr updaters inherit from
-
class
catalyst.core.callbacks.timer.TimerCallback[source]¶ Bases:
catalyst.core.callback.CallbackLogs pipeline execution time
Registry¶
-
catalyst.core.registry.Callback(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
catalyst.core.registry.Criterion(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
catalyst.core.registry.Optimizer(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
catalyst.core.registry.Scheduler(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
catalyst.core.registry.Module(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
catalyst.core.registry.Model(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
catalyst.core.registry.Sampler(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
catalyst.core.registry.Transform(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]¶ Adds factory to registry with it’s
__name__attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)