Utils

catalyst.utils.checkpoint.pack_checkpoint(model=None, criterion=None, optimizer=None, scheduler=None, **kwargs)

catalyst.utils.checkpoint.unpack_checkpoint(checkpoint, model=None, criterion=None, optimizer=None, scheduler=None)

catalyst.utils.checkpoint.save_checkpoint(checkpoint: Dict, logdir: Union[pathlib.Path, str], suffix: str, is_best: bool = False, is_last: bool = False, special_suffix: str = '')

catalyst.utils.checkpoint.load_checkpoint(filepath)

catalyst.utils.config.load_config(path: Union[str, pathlib.Path], ordered: bool = False, data_format: str = None, encoding: str = 'utf-8') → Union[Dict, List]

Loads config by giving path. Supports YAML and JSON files. :param path: path to config file (YAML or JSON) :type path: str :param ordered: if true the config will be loaded as OrderedDict :type ordered: bool :param data_format: yaml, yml or json. :type data_format: str :param encoding: encoding to read the config :type encoding: str

Returns

Config

Return type

(Union[Dict, List])

Raises

Exception – if path path doesn’t exists or file format is not YAML or JSON

Examples

>>> load(path="./config.yml", ordered=True)

catalyst.utils.config.save_config(config: Union[Dict, List], path: Union[str, pathlib.Path], data_format: str = None, encoding: str = 'utf-8', ensure_ascii: bool = False, indent: int = 2) → None

Saves config to file. Path must be either YAML or JSON :param config: config to save :type config: Union[Dict, List] :param path: path to save :type path: Union[str, Path] :param data_format: yaml, yml or json. :type data_format: str :param encoding: Encoding to write file. Default is utf-8 :type encoding: str :param ensure_ascii: Used for JSON, if True non-ASCII :type ensure_ascii: bool :param characters are escaped in JSON strings.: :param indent: Used for JSON :type indent: int

catalyst.utils.distributed.get_rank() → int

Returns the rank of the current worker.

Returns

rank if torch.distributed is initialized,

otherwise -1

Return type

int

catalyst.utils.distributed.process_components(model: torch.nn.modules.module.Module, criterion: torch.nn.modules.module.Module = None, optimizer: torch.optim.optimizer.Optimizer = None, scheduler: torch.optim.lr_scheduler._LRScheduler = None, distributed_params: Dict = None, device: Union[str, torch.device] = None) → Tuple[torch.nn.modules.module.Module, torch.nn.modules.module.Module, torch.optim.optimizer.Optimizer, torch.optim.lr_scheduler._LRScheduler, Union[str, torch.device]]

Returns the processed model, criterion, optimizer, scheduler and device

Parameters

• model (Model) – torch model

• criterion (Criterion) – criterion function

• optimizer (Optimizer) – optimizer

• scheduler (Scheduler) – scheduler

• distributed_params (dict, optional) – dict with the parameters for distributed and FP16 methond

• device (Device, optional) – device

catalyst.utils.distributed.get_distributed_mean(value: float)

Computes distributed mean among all nodes

catalyst.utils.distributed.is_apex_available() → bool

Checks if apex is available

catalyst.utils.distributed.assert_fp16_available() → None

Asserts for installed and available Apex FP16

catalyst.utils.distributed.distributed_run(distributed, worker_fn, *args, **kwargs)

Distributed run :param distributed: :param worker_fn: :param args: :param kwargs:

catalyst.utils.distributed.is_slurm_available()

catalyst.utils.distributed.is_torch_distributed_initialized() → bool

Checks if torch.distributed is available and initialized

catalyst.utils.distributed.initialize_apex(model, optimizer=None, **distributed_params)

catalyst.utils.distributed.is_wrapped_with_ddp(model: torch.nn.modules.module.Module) → bool

Checks whether model is wrapped with DataParallel/DistributedDataParallel.

catalyst.utils.distributed.get_distributed_env(local_rank, rank, world_size, use_cuda_visible_devices=True)

catalyst.utils.distributed.get_distributed_params()

catalyst.utils.distributed.get_nn_from_ddp_module(model: torch.nn.modules.module.Module) → torch.nn.modules.module.Module

Return a real model from a torch.nn.DataParallel, torch.nn.parallel.DistributedDataParallel, or apex.parallel.DistributedDataParallel.

Parameters

model – A model, or DataParallel wrapper.

Returns

A model

catalyst.utils.distributed.get_slurm_params()

catalyst.utils.hash.get_hash(obj: Any) → str

Creates unique hash from object following way: - Represent obj as sting recursively - Hash this string with sha256 hash function - encode hash with url-safe base64 encoding

Parameters

obj – object to hash

Returns

base64-encoded string

catalyst.utils.hash.get_short_hash(o) → str

catalyst.utils.initialization.get_optimal_inner_init(nonlinearity: torch.nn.modules.module.Module, **kwargs) → Callable[[torch.nn.modules.module.Module], None]

Create initializer for inner layers based on their activation function (nonlinearity).

Parameters

nonlinearity – non-linear activation

catalyst.utils.initialization.outer_init(layer: torch.nn.modules.module.Module) → None

Initialization for output layers of policy and value networks typically used in deep reinforcement learning literature.

catalyst.utils.misc.copy_directory(input_dir: pathlib.Path, output_dir: pathlib.Path) → None

Recursively copies the input directory

Parameters

• input_dir (Path) – input directory

• output_dir (Path) – output directory

catalyst.utils.misc.fn_ends_with_pass(fn: Callable[[...], Any])

Check that function end with pass statement (probably does nothing in any way). Mainly used to filter callbacks with empty on_{event} methods. :param fn: target Callable :type fn: Callable[…, Any]

Returns

True if there is pass in the first indentation level of fn

and nothing happens before it, False in any other case.

Return type

bool

catalyst.utils.misc.format_metric(name: str, value: float) → str

Format metric. Metric will be returned in the scientific format if 4 decimal chars are not enough (metric value lower than 1e-4)

Parameters

• name (str) – metric name

• value (float) – value of metric

catalyst.utils.misc.get_fn_argsnames(fn: Callable[[...], Any], exclude: List[str] = None)

Return parameter names of Callable. :param fn: target Callable :type fn: Callable[…, Any] :param exclude: exclude list of parameters :type exclude: List[str]

Returns

contains parameter names of fn

Return type

list

catalyst.utils.misc.get_fn_default_params(fn: Callable[[...], Any], exclude: List[str] = None)

Return default parameters of Callable. :param fn: target Callable :type fn: Callable[…, Any] :param exclude: exclude list of parameters :type exclude: List[str]

Returns

contains default parameters of fn

Return type

dict

catalyst.utils.misc.get_utcnow_time(format: str = None) → str

Return string with current utc time in chosen format

Parameters

format (str) – format string. if None “%y%m%d.%H%M%S” will be used.

Returns

formatted utc time string

Return type

str

catalyst.utils.misc.is_exception(ex: Any) → bool

Check if the argument is of Exception type

catalyst.utils.misc.maybe_recursive_call(object_or_dict, method: str, recursive_args=None, recursive_kwargs=None, **kwargs)

Calls the method recursively for the object_or_dict

Parameters

• object_or_dict (Any) – some object or a dictionary of objects

• method (str) – method name to call

• recursive_args – list of arguments to pass to the method

• recursive_kwargs – list of key-arguments to pass to the method

• **kwargs – Arbitrary keyword arguments

catalyst.utils.numpy.get_one_hot(label: int, num_classes: int, smoothing: float = None) → numpy.ndarray

Applies OneHot vectorization to a giving scalar, optional with label smoothing from https://arxiv.org/abs/1812.01187

Parameters

• label (int) – scalar value to be vectorized

• num_classes (int) – total number of classes

• smoothing (float, optional) – if specified applies label smoothing from Bag of Tricks for Image Classification with Convolutional Neural Networks paper

Returns

a one-hot vector with shape (num_classes,)

Return type

np.ndarray

catalyst.utils.parser.parse_config_args(*, config, args, unknown_args)

catalyst.utils.parser.parse_args_uargs(args, unknown_args)

Function for parsing configuration files

Parameters

• args – recognized arguments

• unknown_args – unrecognized arguments

Returns

updated arguments, dict with config

Return type

tuple

catalyst.utils.scripts.import_module(expdir: pathlib.Path)

catalyst.utils.scripts.dump_code(expdir, logdir)

catalyst.utils.scripts.dump_python_files(src, dst)

catalyst.utils.scripts.import_experiment_and_runner(expdir: pathlib.Path)

catalyst.utils.scripts.dump_base_experiment_code(src: pathlib.Path, dst: pathlib.Path)

catalyst.utils.seed.set_global_seed(seed: int) → None

Sets random seed into PyTorch, TensorFlow, Numpy and Random.

Parameters

seed – random seed

catalyst.utils.sys.get_environment_vars() → Dict[str, Any]

Creates a dictionary with environment variables

Returns

environment variables

Return type

dict

catalyst.utils.sys.list_conda_packages() → str

catalyst.utils.sys.list_pip_packages() → str

catalyst.utils.sys.dump_environment(experiment_config: Dict, logdir: str, configs_path: List[str] = None) → None

Saves config, environment variables and package list in JSON into logdir

Parameters

• experiment_config (dict) – experiment config

• logdir (str) – path to logdir

• configs_path – path(s) to config

catalyst.utils.torch.get_optimizable_params(model_or_params)

Returns all the parameters that requires gradients

catalyst.utils.torch.get_optimizer_momentum(optimizer: torch.optim.optimizer.Optimizer) → float

Get momentum of current optimizer.

Parameters

optimizer – PyTorch optimizer

Returns

momentum at first param group

Return type

float

catalyst.utils.torch.set_optimizer_momentum(optimizer: torch.optim.optimizer.Optimizer, value: float, index: int = 0)

Set momentum of index ‘th param group of optimizer to value

Parameters

• optimizer – PyTorch optimizer

• value (float) – new value of momentum

• index (int, optional) – integer index of optimizer’s param groups, default is 0

catalyst.utils.torch.get_device() → torch.device

Simple returning the best available device (GPU or CPU)

catalyst.utils.torch.get_available_gpus()

Array of available GPU ids :returns: available GPU ids :rtype: iterable

Examples

>>> os.environ["CUDA_VISIBLE_DEVICES"] = "0,2"
>>> get_available_gpus()
>>> [0, 2]

>>> os.environ["CUDA_VISIBLE_DEVICES"] = "0,-1,1"
>>> get_available_gpus()
>>> [0]

>>> os.environ["CUDA_VISIBLE_DEVICES"] = ""
>>> get_available_gpus()
>>> []

>>> os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
>>> get_available_gpus()
>>> []

catalyst.utils.torch.get_activation_fn(activation: str = None)

Returns the activation function from torch.nn by its name

catalyst.utils.torch.any2device(value, device: Union[str, torch.device])

Move tensor, list of tensors, list of list of tensors, dict of tensors, tuple of tensors to target device.

Parameters

• value – Object to be moved

• device (Device) – target device ids

Returns

Same structure as value, but all tensors and np.arrays moved to device

catalyst.utils.torch.prepare_cudnn(deterministic: bool = None, benchmark: bool = None) → None

Prepares CuDNN benchmark and sets CuDNN to be deterministic/non-deterministic mode

Parameters

• deterministic (bool) – deterministic mode if running in CuDNN backend.

• benchmark (bool) – If True use CuDNN heuristics to figure out which algorithm will be most performant for your model architecture and input. Setting it to False may slow down your training.

catalyst.utils.torch.process_model_params(model: torch.nn.modules.module.Module, layerwise_params: Dict[str, dict] = None, no_bias_weight_decay: bool = True, lr_scaling: float = 1.0) → List[Union[torch.nn.parameter.Parameter, dict]]

Gains model parameters for torch.optim.Optimizer

Parameters

• model (torch.nn.Module) – Model to process

• layerwise_params (Dict) – Order-sensitive dict where each key is regex pattern and values are layer-wise options for layers matching with a pattern

• no_bias_weight_decay (bool) – If true, removes weight_decay for all bias parameters in the model

• lr_scaling (float) – layer-wise learning rate scaling, if 1.0, learning rates will not be scaled

Returns

parameters for an optimizer

Return type

iterable

Examples

>>> model = catalyst.contrib.models.segmentation.ResnetUnet()
>>> layerwise_params = collections.OrderedDict([
>>>     ("conv1.*", dict(lr=0.001, weight_decay=0.0003)),
>>>     ("conv.*", dict(lr=0.002))
>>> ])
>>> params = process_model_params(model, layerwise_params)
>>> optimizer = torch.optim.Adam(params, lr=0.0003)

catalyst.utils.torch.set_requires_grad(model: torch.nn.modules.module.Module, requires_grad: bool)

Sets the requires_grad value for all model parameters.

Parameters

• model (torch.nn.Module) – Model

• requires_grad (bool) – value

Examples

>>> model = SimpleModel()
>>> set_requires_grad(model, requires_grad=True)

Tools

class catalyst.utils.tools.frozen_class.FrozenClass

Bases: object

Class which prohibit __setattr__ on existing attributes

Examples

>>> class State(FrozenClass):

class catalyst.utils.tools.registry.Registry(default_name_key: str, default_meta_factory: Callable[[Union[Type, Callable[[...], Any]], Tuple, Mapping], Any] = <function _default_meta_factory>)

Bases: collections.abc.MutableMapping

Universal class allowing to add and access various factories by name

__init__(default_name_key: str, default_meta_factory: Callable[[Union[Type, Callable[[...], Any]], Tuple, Mapping], Any] = <function _default_meta_factory>)

Parameters

• default_name_key (str) – Default key containing factory name when creating from config

• default_meta_factory (MetaFactory) – default object that calls factory. Optional. Default just calls factory.

add(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]]

Adds factory to registry with it’s __name__ attribute or provided name. Signature is flexible.

Parameters

• factory – Factory instance

• factories – More instances

• name – Provided name for first instance. Use only when pass single instance.

• named_factories – Factory and their names as kwargs

Returns

First factory passed

Return type

(Factory)

add_from_module(module, prefix: Union[str, List[str]] = None) → None

Adds all factories present in module. If __all__ attribute is present, takes ony what mentioned in it

Parameters

• module – module to scan

• prefix (Union[str, List[str]]) – prefix string for all the module’s factories. If prefix is a list, all values will be treated as aliases.

all() → List[str]

Returns

list of names of registered items

get(name: str) → Union[Type, Callable[[...], Any], None]

Retrieves factory, without creating any objects with it or raises error

Parameters

name – factory name

Returns

factory by name

Return type

Factory

get_from_params(*, meta_factory=None, **kwargs) → Union[Any, Tuple[Any, Mapping[str, Any]]]

Creates instance based in configuration dict with instantiation_fn. If config[name_key] is None, None is returned.

Parameters

• meta_factory – Function that calls factory the right way. If not provided, default is used.

• **kwargs – additional kwargs for factory

Returns

result of calling instantiate_fn(factory, **config)

get_if_str(obj: Union[str, Type, Callable[[...], Any]])

Returns object from the registry if obj type is string

get_instance(name: str, *args, meta_factory=None, **kwargs)

Creates instance by calling specified factory with instantiate_fn

Parameters

• name – factory name

• meta_factory – Function that calls factory the right way. If not provided, default is used

• args – args to pass to the factory

• **kwargs – kwargs to pass to the factory

Returns

created instance

late_add(cb: Callable[[Registry], None])

Allows to prevent cycle imports by delaying some imports till next registry query

Parameters

cb – Callback receives registry and must call it’s methods to register factories

len() → int

Returns

length of registered items

exception catalyst.utils.tools.registry.RegistryException(message)

Bases: Exception

Exception class for all registry errors

__init__(message)

Init

exception catalyst.utils.tools.tensorboard.EventReadingException

Bases: Exception

An exception that correspond to an event file reading error

class catalyst.utils.tools.tensorboard.EventsFileReader(events_file: BinaryIO)

Bases: collections.abc.Iterable

An iterator over a Tensorboard events file

__init__(events_file: BinaryIO)

Initialize an iterator over an events file

Parameters

events_file – An opened file-like object.

class catalyst.utils.tools.tensorboard.SummaryItem(tag, step, wall_time, value, type)

Bases: tuple

property step

Alias for field number 1

property tag

Alias for field number 0

property type

Alias for field number 4

property value

Alias for field number 3

property wall_time

Alias for field number 2

class catalyst.utils.tools.tensorboard.SummaryReader(logdir: Union[str, pathlib.Path], tag_filter: Optional[collections.abc.Iterable] = None, types: collections.abc.Iterable = ('scalar',))

Bases: collections.abc.Iterable

Iterates over events in all the files in the current logdir. Only scalars and images are supported at the moment.

__init__(logdir: Union[str, pathlib.Path], tag_filter: Optional[collections.abc.Iterable] = None, types: collections.abc.Iterable = ('scalar',))

Initalize new summary reader

Parameters

• logdir – A directory with Tensorboard summary data

• tag_filter – A list of tags to leave (None for all)

• types – A list of types to get.

• "scalar" and "image" types are allowed at the moment. (Only) –

class catalyst.utils.tools.time_manager.TimeManager

Bases: object

reset() → None

Reset all previous timers

start(name: str) → None

Starts timer name :param name: name of a timer :type name: str

stop(name: str) → None

Stops timer name :param name: name of a timer :type name: str

catalyst.utils.tools.typing.Model

alias of torch.nn.modules.module.Module

catalyst.utils.tools.typing.Criterion

alias of torch.nn.modules.module.Module

class catalyst.utils.tools.typing.Optimizer(params, defaults)

Bases: object

Base class for all optimizers.

Warning

Parameters need to be specified as collections that have a deterministic ordering that is consistent between runs. Examples of objects that don’t satisfy those properties are sets and iterators over values of dictionaries.

Parameters

• params (iterable) – an iterable of torch.Tensor s or dict s. Specifies what Tensors should be optimized.

• defaults – (dict): a dict containing default values of optimization options (used when a parameter group doesn’t specify them).

add_param_group(param_group)

Add a param group to the Optimizer s param_groups.

This can be useful when fine tuning a pre-trained network as frozen layers can be made trainable and added to the Optimizer as training progresses.

Parameters

• param_group (dict) – Specifies what Tensors should be optimized along with group

• optimization options. (specific) –

load_state_dict(state_dict)

Loads the optimizer state.

Parameters

state_dict (dict) – optimizer state. Should be an object returned from a call to state_dict().

state_dict()

Returns the state of the optimizer as a dict.

It contains two entries:

• state - a dict holding current optimization state. Its content

  differs between optimizer classes.

• param_groups - a dict containing all parameter groups

step(closure)

Performs a single optimization step (parameter update).

Parameters

closure (callable) – A closure that reevaluates the model and returns the loss. Optional for most optimizers.

zero_grad()

Clears the gradients of all optimized torch.Tensor s.

catalyst.utils.tools.typing.Scheduler

alias of torch.optim.lr_scheduler._LRScheduler

class catalyst.utils.tools.typing.Dataset

Bases: object

An abstract class representing a Dataset.

All datasets that represent a map from keys to data samples should subclass it. All subclasses should overwrite __getitem__(), supporting fetching a data sample for a given key. Subclasses could also optionally overwrite __len__(), which is expected to return the size of the dataset by many Sampler implementations and the default options of DataLoader.

Note

DataLoader by default constructs a index sampler that yields integral indices. To make it work with a map-style dataset with non-integral indices/keys, a custom sampler must be provided.

Metrics

catalyst.utils.metrics.accuracy.accuracy(outputs, targets, topk=(1, ), threshold: float = None, activation: str = None)

Computes the accuracy.

It can be used either for:

• multi-class task:

  -you can use topk. -threshold and activation are not required. -targets is a tensor: batch_size -outputs is a tensor: batch_size x num_classes -computes the accuracy@k for the specified values of k.

• OR multi-label task, in this case:

  -you must specify threshold and activation -topk will not be used (because of there is no method to apply top-k in multi-label classification). -outputs, targets are tensors with shape: batch_size x num_classes -targets is a tensor with binary vectors

catalyst.utils.metrics.accuracy.average_accuracy(outputs, targets, k=10)

Computes the average accuracy at k. This function computes the average accuracy at k between two lists of items.

Parameters

• outputs (list) – A list of predicted elements

• targets (list) – A list of elements that are to be predicted

• k (int, optional) – The maximum number of predicted elements

Returns

The average accuracy at k over the input lists

Return type

double

catalyst.utils.metrics.accuracy.mean_average_accuracy(outputs, targets, topk=(1, ))

Computes the mean average accuracy at k. This function computes the mean average accuracy at k between two lists of lists of items.

Parameters

• outputs (list) – A list of lists of predicted elements

• targets (list) – A list of lists of elements that are to be predicted

• topk (int, optional) – The maximum number of predicted elements

Returns

The mean average accuracy at k over the input lists

Return type

double

catalyst.utils.metrics.dice.dice(outputs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Computes the dice metric

Parameters

• outputs (list) – A list of predicted elements

• targets (list) – A list of elements that are to be predicted

• eps (float) – epsilon

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

Dice score

Return type

double

catalyst.utils.metrics.f1_score.f1_score(outputs: torch.Tensor, targets: torch.Tensor, beta: float = 1.0, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid')

Source https://github.com/qubvel/segmentation_models.pytorch

Parameters

• outputs (torch.Tensor) – A list of predicted elements

• targets (torch.Tensor) – A list of elements that are to be predicted

• eps (float) – epsilon to avoid zero division

• beta (float) – beta param for f_score

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

F_1 score

Return type

float

catalyst.utils.metrics.focal.sigmoid_focal_loss(outputs: torch.Tensor, targets: torch.Tensor, gamma: float = 2.0, alpha: float = 0.25, reduction: str = 'mean')

Compute binary focal loss between target and output logits.

Source https://github.com/BloodAxe/pytorch-toolbelt See losses for details.

Parameters

• outputs – Tensor of arbitrary shape

• targets – Tensor of the same shape as input

• reduction (string, optional) – Specifies the reduction to apply to the output: “none” | “mean” | “sum” | “batchwise_mean”. “none”: no reduction will be applied, “mean”: the sum of the output will be divided by the number of elements in the output, “sum”: the output will be summed.

See https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/loss/losses.py # noqa: E501

catalyst.utils.metrics.focal.reduced_focal_loss(outputs: torch.Tensor, targets: torch.Tensor, threshold: float = 0.5, gamma: float = 2.0, reduction='mean')

Compute reduced focal loss between target and output logits.

Source https://github.com/BloodAxe/pytorch-toolbelt See losses for details.

Parameters

• outputs – Tensor of arbitrary shape

• targets – Tensor of the same shape as input

• reduction (string, optional) –

  Specifies the reduction to apply to the output: “none” | “mean” | “sum” | “batchwise_mean”.

  ”none”: no reduction will be applied, “mean”: the sum of the output will be divided by the number of elements in the output, “sum”: the output will be summed.

  Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction.

  ”batchwise_mean” computes mean loss per sample in batch. Default: “mean”

See https://arxiv.org/abs/1903.01347

catalyst.utils.metrics.iou.iou(outputs: torch.Tensor, targets: torch.Tensor, classes: List[str] = None, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid') → Union[float, List[float]]

Parameters

• outputs (torch.Tensor) – A list of predicted elements

• targets (torch.Tensor) – A list of elements that are to be predicted

• eps (float) – epsilon to avoid zero division

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

IoU (Jaccard) score(s)

Return type

Union[float, List[float]]

catalyst.utils.metrics.iou.jaccard(outputs: torch.Tensor, targets: torch.Tensor, classes: List[str] = None, eps: float = 1e-07, threshold: float = None, activation: str = 'Sigmoid') → Union[float, List[float]]

Parameters

• outputs (torch.Tensor) – A list of predicted elements

• targets (torch.Tensor) – A list of elements that are to be predicted

• eps (float) – epsilon to avoid zero division

• threshold (float) – threshold for outputs binarization

• activation (str) – An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]

Returns

IoU (Jaccard) score(s)

Return type

Union[float, List[float]]

Meters

The meters from torchnet.meters

class catalyst.utils.meters.meter.Meter

Bases: object

Meters provide a way to keep track of important statistics in an online manner.

This class is abstract, but provides a standard interface for all meters to follow.

add(value)

Log a new value to the meter

Parameters

value – Next restult to include.

reset()

Resets the meter to default settings.

value()

Get the value of the meter in the current state.

class catalyst.utils.meters.apmeter.APMeter

Bases: catalyst.utils.meters.meter.Meter

The APMeter measures the average precision per class.

The APMeter is designed to operate on NxK Tensors output and target, and optionally a Nx1 Tensor weight where (1) the output contains model output scores for N examples and K classes that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a sigmoid function); (2) the target contains only values 0 (for negative examples) and 1 (for positive examples); and (3) the weight ( > 0) represents weight for each sample.

add(output, target, weight=None)

Add a new observation

Parameters

• output (Tensor) – NxK tensor that for each of the N examples indicates the probability of the example belonging to each of the K classes, according to the model. The probabilities should sum to one over all classes

• target (Tensor) – binary NxK tensort that encodes which of the K classes are associated with the N-th input (eg: a row [0, 1, 0, 1] indicates that the example is associated with classes 2 and 4)

• weight (optional, Tensor) – Nx1 tensor representing the weight for each example (each weight > 0)

reset()

Resets the meter with empty member variables

value()

Returns the model”s average precision for each class

Returns

1xK tensor, with avg precision for each class k

Return type

ap (FloatTensor)

class catalyst.utils.meters.aucmeter.AUCMeter

Bases: catalyst.utils.meters.meter.Meter

The AUCMeter measures the area under the receiver-operating characteristic (ROC) curve for binary classification problems. The area under the curve (AUC) can be interpreted as the probability that, given a randomly selected positive example and a randomly selected negative example, the positive example is assigned a higher score by the classification model than the negative example.

The AUCMeter is designed to operate on one-dimensional Tensors output and target, where (1) the output contains model output scores that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a signoid function); and (2) the target contains only values 0 (for negative examples) and 1 (for positive examples).

add(output, target)

reset()

value()

class catalyst.utils.meters.averagevaluemeter.AverageValueMeter

Bases: catalyst.utils.meters.meter.Meter

add(value, n=1)

reset()

value()

class catalyst.utils.meters.classerrormeter.ClassErrorMeter(topk=[1], accuracy=False)

Bases: catalyst.utils.meters.meter.Meter

add(output, target)

reset()

value(k=-1)

class catalyst.utils.meters.confusionmeter.ConfusionMeter(k, normalized=False)

Bases: catalyst.utils.meters.meter.Meter

Maintains a confusion matrix for a given calssification problem.

The ConfusionMeter constructs a confusion matrix for a multi-class classification problems. It does not support multi-label, multi-class problems: for such problems, please use MultiLabelConfusionMeter.

Parameters

• k (int) – number of classes in the classification problem

• normalized (boolean) – Determines whether or not the confusion matrix is normalized or not

add(predicted, target)

Computes the confusion matrix of K x K size where K is no of classes

Args: predicted (tensor): Can be an N x K tensor of predicted scores obtained from the model for N examples and K classes or an N-tensor of integer values between 0 and K-1. target (tensor): Can be a N-tensor of integer values assumed to be integer values between 0 and K-1 or N x K tensor, where targets are assumed to be provided as one-hot vectors

reset()

value()

Returns

Confustion matrix of K rows and K columns, where rows corresponds to ground-truth targets and columns corresponds to predicted targets.

class catalyst.utils.meters.mapmeter.mAPMeter

Bases: catalyst.utils.meters.meter.Meter

The mAPMeter measures the mean average precision over all classes.

The mAPMeter is designed to operate on NxK Tensors output and target, and optionally a Nx1 Tensor weight where (1) the output contains model output scores for N examples and K classes that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a sigmoid function); (2) the target contains only values 0 (for negative examples) and 1 (for positive examples); and (3) the weight ( > 0) represents weight for each sample.

add(output, target, weight=None)

reset()

value()

class catalyst.utils.meters.movingaveragevaluemeter.MovingAverageValueMeter(windowsize)

Bases: catalyst.utils.meters.meter.Meter

add(value)

reset()

value()

class catalyst.utils.meters.msemeter.MSEMeter(root=False)

Bases: catalyst.utils.meters.meter.Meter

add(output, target)

reset()

value()

class catalyst.utils.meters.ppv_tpr_f1_meter.PrecisionRecallF1ScoreMeter(threshold=0.5)

Bases: catalyst.utils.meters.meter.Meter

Keeps track of global true positives, false positives, and false negatives for each epoch and calculates precision, recall, and F1-score based on those metrics. Currently, this meter works for binary cases only, please use multiple instances of this class for multi-label cases.

add(output, target)

Thresholds predictions and calculates the true positives, false positives, and false negatives in comparison to the target.

Parameters

• output (torch.Tensor) – prediction after activation function shape should be (batch_size, …), but works with any shape

• target (torch.Tensor) – label (binary) shape should be the same as output’s shape

Returns

None

reset()

Resets true positive, false positive and false negative counts to 0.

value()

Calculates precision/recall/f1 based on the current stored tp/fp/fn counts. :param None:

Returns

(precision, recall, f1)

Return type

tuple of floats

catalyst.utils.meters.ppv_tpr_f1_meter.f1score(precision_value, recall_value, eps=1e-05)

Calculating F1-score from precision and recall to reduce computation redundancy. :param precision_value: precision (0-1) :param recall_value: recall (0-1)

Returns

F1 score (0-1)

catalyst.utils.meters.ppv_tpr_f1_meter.precision(tp, fp, eps=1e-05)

Calculates precision (a.k.a. positive predictive value) for binary classification and segmentation. :param tp: number of true positives :param fp: number of false positives

Returns

precision value (0-1)

catalyst.utils.meters.ppv_tpr_f1_meter.recall(tp, fn, eps=1e-05)

Calculates recall (a.k.a. true positive rate) for binary classification and segmentation :param tp: number of true positives :param fn: number of false negatives

Returns

recall value (0-1)