Utils¶
-
catalyst.utils.checkpoint.
pack_checkpoint
(model=None, criterion=None, optimizer=None, scheduler=None, **kwargs)[source]¶
-
catalyst.utils.checkpoint.
unpack_checkpoint
(checkpoint, model=None, criterion=None, optimizer=None, scheduler=None)[source]¶
-
catalyst.utils.checkpoint.
save_checkpoint
(checkpoint: Dict, logdir: Union[pathlib.Path, str], suffix: str, is_best: bool = False, is_last: bool = False, special_suffix: str = '')[source]¶
-
catalyst.utils.config.
load_config
(path: Union[str, pathlib.Path], ordered: bool = False, data_format: str = None, encoding: str = 'utf-8') → Union[Dict, List][source]¶ Loads config by giving path. Supports YAML and JSON files. :param path: path to config file (YAML or JSON) :type path: str :param ordered: if true the config will be loaded as
OrderedDict
:type ordered: bool :param data_format:yaml
,yml
orjson
. :type data_format: str :param encoding: encoding to read the config :type encoding: str- Returns
Config
- Return type
(Union[Dict, List])
- Raises
Exception – if path
path
doesn’t exists or file format is not YAML or JSON
Examples
>>> load(path="./config.yml", ordered=True)
-
catalyst.utils.config.
save_config
(config: Union[Dict, List], path: Union[str, pathlib.Path], data_format: str = None, encoding: str = 'utf-8', ensure_ascii: bool = False, indent: int = 2) → None[source]¶ Saves config to file. Path must be either YAML or JSON :param config: config to save :type config: Union[Dict, List] :param path: path to save :type path: Union[str, Path] :param data_format:
yaml
,yml
orjson
. :type data_format: str :param encoding: Encoding to write file. Default isutf-8
:type encoding: str :param ensure_ascii: Used for JSON, if True non-ASCII :type ensure_ascii: bool :param characters are escaped in JSON strings.: :param indent: Used for JSON :type indent: int
-
catalyst.utils.hash.
get_hash
(obj: Any) → str[source]¶ Creates unique hash from object following way: - Represent obj as sting recursively - Hash this string with sha256 hash function - encode hash with url-safe base64 encoding
- Parameters
obj – object to hash
- Returns
base64-encoded string
-
catalyst.utils.initialization.
get_optimal_inner_init
(nonlinearity: torch.nn.modules.module.Module, **kwargs) → Callable[[torch.nn.modules.module.Module], None][source]¶ Create initializer for inner layers based on their activation function (nonlinearity).
- Parameters
nonlinearity – non-linear activation
-
catalyst.utils.initialization.
outer_init
(layer: torch.nn.modules.module.Module) → None[source]¶ Initialization for output layers of policy and value networks typically used in deep reinforcement learning literature.
-
catalyst.utils.misc.
copy_directory
(input_dir: pathlib.Path, output_dir: pathlib.Path) → None[source]¶ Recursively copies the input directory
- Parameters
input_dir (Path) – input directory
output_dir (Path) – output directory
-
catalyst.utils.misc.
fn_ends_with_pass
(fn: Callable[[...], Any])[source]¶ Check that function end with pass statement (probably does nothing in any way). Mainly used to filter callbacks with empty on_{event} methods. :param fn: target Callable :type fn: Callable[…, Any]
- Returns
- True if there is pass in the first indentation level of fn
and nothing happens before it, False in any other case.
- Return type
bool
-
catalyst.utils.misc.
format_metric
(name: str, value: float) → str[source]¶ Format metric. Metric will be returned in the scientific format if 4 decimal chars are not enough (metric value lower than 1e-4)
- Parameters
name (str) – metric name
value (float) – value of metric
-
catalyst.utils.misc.
get_fn_argsnames
(fn: Callable[[...], Any], exclude: List[str] = None)[source]¶ Return parameter names of Callable. :param fn: target Callable :type fn: Callable[…, Any] :param exclude: exclude list of parameters :type exclude: List[str]
- Returns
contains parameter names of fn
- Return type
list
-
catalyst.utils.misc.
get_fn_default_params
(fn: Callable[[...], Any], exclude: List[str] = None)[source]¶ Return default parameters of Callable. :param fn: target Callable :type fn: Callable[…, Any] :param exclude: exclude list of parameters :type exclude: List[str]
- Returns
contains default parameters of fn
- Return type
dict
-
catalyst.utils.misc.
get_utcnow_time
(format: str = None) → str[source]¶ Return string with current utc time in chosen format
- Parameters
format (str) – format string. if None “%y%m%d.%H%M%S” will be used.
- Returns
formatted utc time string
- Return type
str
-
catalyst.utils.misc.
is_exception
(ex: Any) → bool[source]¶ Check if the argument is of Exception type
-
catalyst.utils.misc.
maybe_recursive_call
(object_or_dict, method: str, recursive_args=None, recursive_kwargs=None, **kwargs)[source]¶ Calls the
method
recursively for the object_or_dict- Parameters
object_or_dict (Any) – some object or a dictionary of objects
method (str) – method name to call
recursive_args – list of arguments to pass to the
method
recursive_kwargs – list of key-arguments to pass to the
method
**kwargs – Arbitrary keyword arguments
-
catalyst.utils.numpy.
get_one_hot
(label: int, num_classes: int, smoothing: float = None) → numpy.ndarray[source]¶ Applies OneHot vectorization to a giving scalar, optional with label smoothing from https://arxiv.org/abs/1812.01187
- Parameters
label (int) – scalar value to be vectorized
num_classes (int) – total number of classes
smoothing (float, optional) – if specified applies label smoothing from
Bag of Tricks for Image Classification with Convolutional Neural Networks
paper
- Returns
a one-hot vector with shape
(num_classes,)
- Return type
np.ndarray
-
catalyst.utils.plotly.
plot_tensorboard_log
(logdir: Union[str, pathlib.Path], step: Optional[str] = 'batch', metrics: Optional[List[str]] = None, height: Optional[int] = None, width: Optional[int] = None) → None¶
-
catalyst.utils.seed.
set_global_seed
(seed: int) → None[source]¶ Sets random seed into PyTorch, TensorFlow, Numpy and Random.
- Parameters
seed – random seed
-
catalyst.utils.torch.
get_optimizable_params
(model_or_params)[source]¶ Returns all the parameters that requires gradients
-
catalyst.utils.torch.
get_optimizer_momentum
(optimizer: torch.optim.optimizer.Optimizer) → float[source]¶ Get momentum of current optimizer.
- Parameters
optimizer – PyTorch optimizer
- Returns
momentum at first param group
- Return type
float
-
catalyst.utils.torch.
set_optimizer_momentum
(optimizer: torch.optim.optimizer.Optimizer, value: float, index: int = 0)[source]¶ Set momentum of
index
‘th param group of optimizer tovalue
- Parameters
optimizer – PyTorch optimizer
value (float) – new value of momentum
index (int, optional) – integer index of optimizer’s param groups, default is 0
-
catalyst.utils.torch.
get_device
() → torch.device[source]¶ Simple returning the best available device (GPU or CPU)
-
catalyst.utils.torch.
get_available_gpus
()[source]¶ Array of available GPU ids :returns: available GPU ids :rtype: iterable
Examples
>>> os.environ["CUDA_VISIBLE_DEVICES"] = "0,2" >>> get_available_gpus() >>> [0, 2]
>>> os.environ["CUDA_VISIBLE_DEVICES"] = "0,-1,1" >>> get_available_gpus() >>> [0]
>>> os.environ["CUDA_VISIBLE_DEVICES"] = "" >>> get_available_gpus() >>> []
>>> os.environ["CUDA_VISIBLE_DEVICES"] = "-1" >>> get_available_gpus() >>> []
-
catalyst.utils.torch.
get_activation_fn
(activation: str = None)[source]¶ Returns the activation function from
torch.nn
by its name
-
catalyst.utils.torch.
any2device
(value, device: Union[str, torch.device])[source]¶ Move tensor, list of tensors, list of list of tensors, dict of tensors, tuple of tensors to target device.
- Parameters
value – Object to be moved
device (Device) – target device ids
- Returns
Same structure as value, but all tensors and np.arrays moved to device
-
catalyst.utils.torch.
prepare_cudnn
(deterministic: bool = None, benchmark: bool = None) → None[source]¶ Prepares CuDNN benchmark and sets CuDNN to be deterministic/non-deterministic mode
- Parameters
deterministic (bool) – deterministic mode if running in CuDNN backend.
benchmark (bool) – If
True
use CuDNN heuristics to figure out which algorithm will be most performant for your model architecture and input. Setting it toFalse
may slow down your training.
-
catalyst.utils.torch.
process_model_params
(model: torch.nn.modules.module.Module, layerwise_params: Dict[str, dict] = None, no_bias_weight_decay: bool = True, lr_scaling: float = 1.0) → List[Union[torch.nn.parameter.Parameter, dict]][source]¶ Gains model parameters for
torch.optim.Optimizer
- Parameters
model (torch.nn.Module) – Model to process
layerwise_params (Dict) – Order-sensitive dict where each key is regex pattern and values are layer-wise options for layers matching with a pattern
no_bias_weight_decay (bool) – If true, removes weight_decay for all
bias
parameters in the modellr_scaling (float) – layer-wise learning rate scaling, if 1.0, learning rates will not be scaled
- Returns
parameters for an optimizer
- Return type
iterable
Examples
>>> model = catalyst.contrib.models.segmentation.ResnetUnet() >>> layerwise_params = collections.OrderedDict([ >>> ("conv1.*", dict(lr=0.001, weight_decay=0.0003)), >>> ("conv.*", dict(lr=0.002)) >>> ]) >>> params = process_model_params(model, layerwise_params) >>> optimizer = torch.optim.Adam(params, lr=0.0003)
-
catalyst.utils.torch.
set_requires_grad
(model: torch.nn.modules.module.Module, requires_grad: bool)[source]¶ Sets the
requires_grad
value for all model parameters.- Parameters
model (torch.nn.Module) – Model
requires_grad (bool) – value
Examples
>>> model = SimpleModel() >>> set_requires_grad(model, requires_grad=True)
-
catalyst.utils.visualization.
plot_confusion_matrix
(cm, class_names=None, normalize=False, title='confusion matrix', fname=None, show=True, figsize=12, fontsize=32, colormap='Blues')¶ Render the confusion matrix and return matplotlib”s figure with it. Normalization can be applied by setting normalize=True.
-
catalyst.utils.visualization.
render_figure_to_tensor
(figure)¶
-
catalyst.utils.visualization.
plot_metrics
(logdir: Union[str, pathlib.Path], step: Optional[str] = 'epoch', metrics: Optional[List[str]] = None, height: Optional[int] = None, width: Optional[int] = None) → None¶ Plots your learning results.
- Parameters
logdir – the logdir that was specified during training.
step – ‘batch’ or ‘epoch’ - what logs to show: for batches or for epochs
metrics – list of metrics to plot. The loss should be specified as ‘loss’, learning rate = ‘_base/lr’ and other metrics should be specified as names in metrics dict that was specified during training
height – the height of the whole resulting plot
width – the width of the whole resulting plot
Tools¶
-
class
catalyst.utils.tools.frozen_class.
FrozenClass
[source]¶ Bases:
object
Class which prohibit
__setattr__
on existing attributesExamples
>>> class State(FrozenClass):
-
class
catalyst.utils.tools.registry.
Registry
(default_name_key: str, default_meta_factory: Callable[[Union[Type, Callable[[...], Any]], Tuple, Mapping], Any] = <function _default_meta_factory>)[source]¶ Bases:
collections.abc.MutableMapping
Universal class allowing to add and access various factories by name
-
__init__
(default_name_key: str, default_meta_factory: Callable[[Union[Type, Callable[[...], Any]], Tuple, Mapping], Any] = <function _default_meta_factory>)[source]¶ - Parameters
default_name_key (str) – Default key containing factory name when creating from config
default_meta_factory (MetaFactory) – default object that calls factory. Optional. Default just calls factory.
-
add
(factory: Union[Type, Callable[[...], Any]] = None, *factories: Union[Type, Callable[[...], Any]], name: str = None, **named_factories: Union[Type, Callable[[...], Any]]) → Union[Type, Callable[[...], Any]][source]¶ Adds factory to registry with it’s
__name__
attribute or provided name. Signature is flexible.- Parameters
factory – Factory instance
factories – More instances
name – Provided name for first instance. Use only when pass single instance.
named_factories – Factory and their names as kwargs
- Returns
First factory passed
- Return type
(Factory)
-
add_from_module
(module, prefix: Union[str, List[str]] = None) → None[source]¶ Adds all factories present in module. If
__all__
attribute is present, takes ony what mentioned in it- Parameters
module – module to scan
prefix (Union[str, List[str]]) – prefix string for all the module’s factories. If prefix is a list, all values will be treated as aliases.
-
get
(name: str) → Union[Type, Callable[[...], Any], None][source]¶ Retrieves factory, without creating any objects with it or raises error
- Parameters
name – factory name
- Returns
factory by name
- Return type
Factory
-
get_from_params
(*, meta_factory=None, **kwargs) → Union[Any, Tuple[Any, Mapping[str, Any]]][source]¶ Creates instance based in configuration dict with
instantiation_fn
. Ifconfig[name_key]
is None, None is returned.- Parameters
meta_factory – Function that calls factory the right way. If not provided, default is used.
**kwargs – additional kwargs for factory
- Returns
result of calling
instantiate_fn(factory, **config)
-
get_if_str
(obj: Union[str, Type, Callable[[...], Any]])[source]¶ Returns object from the registry if
obj
type is string
-
get_instance
(name: str, *args, meta_factory=None, **kwargs)[source]¶ Creates instance by calling specified factory with instantiate_fn
- Parameters
name – factory name
meta_factory – Function that calls factory the right way. If not provided, default is used
args – args to pass to the factory
**kwargs – kwargs to pass to the factory
- Returns
created instance
-
-
exception
catalyst.utils.tools.registry.
RegistryException
(message)[source]¶ Bases:
Exception
Exception class for all registry errors
-
exception
catalyst.utils.tools.tensorboard.
EventReadingException
[source]¶ Bases:
Exception
An exception that correspond to an event file reading error
-
class
catalyst.utils.tools.tensorboard.
EventsFileReader
(events_file: BinaryIO)[source]¶ Bases:
collections.abc.Iterable
An iterator over a Tensorboard events file
-
class
catalyst.utils.tools.tensorboard.
SummaryItem
(tag, step, wall_time, value, type)¶ Bases:
tuple
-
property
step
¶ Alias for field number 1
-
property
tag
¶ Alias for field number 0
-
property
type
¶ Alias for field number 4
-
property
value
¶ Alias for field number 3
-
property
wall_time
¶ Alias for field number 2
-
property
-
class
catalyst.utils.tools.tensorboard.
SummaryReader
(logdir: Union[str, pathlib.Path], tag_filter: Optional[collections.abc.Iterable] = None, types: collections.abc.Iterable = ('scalar',))[source]¶ Bases:
collections.abc.Iterable
Iterates over events in all the files in the current logdir. Only scalars and images are supported at the moment.
-
__init__
(logdir: Union[str, pathlib.Path], tag_filter: Optional[collections.abc.Iterable] = None, types: collections.abc.Iterable = ('scalar',))[source]¶ Initalize new summary reader
- Parameters
logdir – A directory with Tensorboard summary data
tag_filter – A list of tags to leave (None for all)
types – A list of types to get.
"scalar" and "image" types are allowed at the moment. (Only) –
-
-
catalyst.utils.tools.typing.
Model
¶ alias of
torch.nn.modules.module.Module
-
catalyst.utils.tools.typing.
Criterion
¶ alias of
torch.nn.modules.module.Module
-
class
catalyst.utils.tools.typing.
Optimizer
(params, defaults)[source]¶ Bases:
object
Base class for all optimizers.
Warning
Parameters need to be specified as collections that have a deterministic ordering that is consistent between runs. Examples of objects that don’t satisfy those properties are sets and iterators over values of dictionaries.
- Parameters
params (iterable) – an iterable of
torch.Tensor
s ordict
s. Specifies what Tensors should be optimized.defaults – (dict): a dict containing default values of optimization options (used when a parameter group doesn’t specify them).
-
add_param_group
(param_group)[source]¶ Add a param group to the
Optimizer
s param_groups.This can be useful when fine tuning a pre-trained network as frozen layers can be made trainable and added to the
Optimizer
as training progresses.- Parameters
param_group (dict) – Specifies what Tensors should be optimized along with group
optimization options. (specific) –
-
load_state_dict
(state_dict)[source]¶ Loads the optimizer state.
- Parameters
state_dict (dict) – optimizer state. Should be an object returned from a call to
state_dict()
.
-
state_dict
()[source]¶ Returns the state of the optimizer as a
dict
.It contains two entries:
- state - a dict holding current optimization state. Its content
differs between optimizer classes.
param_groups - a dict containing all parameter groups
-
catalyst.utils.tools.typing.
Scheduler
¶ alias of
torch.optim.lr_scheduler._LRScheduler
-
class
catalyst.utils.tools.typing.
Dataset
[source]¶ Bases:
object
An abstract class representing a
Dataset
.All datasets that represent a map from keys to data samples should subclass it. All subclasses should overwrite
__getitem__()
, supporting fetching a data sample for a given key. Subclasses could also optionally overwrite__len__()
, which is expected to return the size of the dataset by manySampler
implementations and the default options ofDataLoader
.Note
DataLoader
by default constructs a index sampler that yields integral indices. To make it work with a map-style dataset with non-integral indices/keys, a custom sampler must be provided.
Criterion¶
Computes the accuracy.
- It can be used either for:
- multi-class task:
-you can use topk. -threshold and activation are not required. -targets is a tensor: batch_size -outputs is a tensor: batch_size x num_classes -computes the accuracy@k for the specified values of k.
- OR multi-label task, in this case:
-you must specify threshold and activation -topk will not be used (because of there is no method to apply top-k in multi-label classification). -outputs, targets are tensors with shape: batch_size x num_classes -targets is a tensor with binary vectors
Computes the dice metric
- param outputs
A list of predicted elements
- type outputs
list
- param targets
A list of elements that are to be predicted
- type targets
list
- param eps
epsilon
- type eps
float
- param threshold
threshold for outputs binarization
- type threshold
float
- param activation
An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]
- type activation
str
- returns
Dice score
- rtype
double
Source https://github.com/qubvel/segmentation_models.pytorch
- param outputs
A list of predicted elements
- type outputs
torch.Tensor
- param targets
A list of elements that are to be predicted
- type targets
torch.Tensor
- param eps
epsilon to avoid zero division
- type eps
float
- param beta
beta param for f_score
- type beta
float
- param threshold
threshold for outputs binarization
- type threshold
float
- param activation
An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]
- type activation
str
- returns
F_1 score
- rtype
float
-
catalyst.utils.criterion.focal.
sigmoid_focal_loss
(outputs: torch.Tensor, targets: torch.Tensor, gamma: float = 2.0, alpha: float = 0.25, reduction: str = 'mean')¶ Compute binary focal loss between target and output logits.
Source https://github.com/BloodAxe/pytorch-toolbelt See
losses
for details.- Parameters
outputs – Tensor of arbitrary shape
targets – Tensor of the same shape as input
reduction (string, optional) – Specifies the reduction to apply to the output: “none” | “mean” | “sum” | “batchwise_mean”. “none”: no reduction will be applied, “mean”: the sum of the output will be divided by the number of elements in the output, “sum”: the output will be summed.
See https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/loss/losses.py # noqa: E501
-
catalyst.utils.criterion.focal.
reduced_focal_loss
(outputs: torch.Tensor, targets: torch.Tensor, threshold: float = 0.5, gamma: float = 2.0, reduction='mean')¶ Compute reduced focal loss between target and output logits.
Source https://github.com/BloodAxe/pytorch-toolbelt See
losses
for details.- Parameters
outputs – Tensor of arbitrary shape
targets – Tensor of the same shape as input
reduction (string, optional) –
Specifies the reduction to apply to the output: “none” | “mean” | “sum” | “batchwise_mean”.
”none”: no reduction will be applied, “mean”: the sum of the output will be divided by the number of elements in the output, “sum”: the output will be summed.
Note:
size_average
andreduce
are in the process of being deprecated, and in the meantime, specifying either of those two args will overridereduction
.”batchwise_mean” computes mean loss per sample in batch. Default: “mean”
- param outputs
A list of predicted elements
- type outputs
torch.Tensor
- param targets
A list of elements that are to be predicted
- type targets
torch.Tensor
- param eps
epsilon to avoid zero division
- type eps
float
- param threshold
threshold for outputs binarization
- type threshold
float
- param activation
An torch.nn activation applied to the outputs. Must be one of [“none”, “Sigmoid”, “Softmax2d”]
- type activation
str
- returns
IoU (Jaccard) score(s)
- rtype
Union[float, List[float]]
Meters¶
The meters from torchnet.meters
-
class
catalyst.utils.meters.meter.
Meter
[source]¶ Bases:
object
Meters provide a way to keep track of important statistics in an online manner.
This class is abstract, but provides a standard interface for all meters to follow.
-
class
catalyst.utils.meters.apmeter.
APMeter
[source]¶ Bases:
catalyst.utils.meters.meter.Meter
The APMeter measures the average precision per class.
The APMeter is designed to operate on NxK Tensors output and target, and optionally a Nx1 Tensor weight where (1) the output contains model output scores for N examples and K classes that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a sigmoid function); (2) the target contains only values 0 (for negative examples) and 1 (for positive examples); and (3) the weight ( > 0) represents weight for each sample.
-
add
(output, target, weight=None)[source]¶ Add a new observation
- Parameters
output (Tensor) – NxK tensor that for each of the N examples indicates the probability of the example belonging to each of the K classes, according to the model. The probabilities should sum to one over all classes
target (Tensor) – binary NxK tensort that encodes which of the K classes are associated with the N-th input (eg: a row [0, 1, 0, 1] indicates that the example is associated with classes 2 and 4)
weight (optional, Tensor) – Nx1 tensor representing the weight for each example (each weight > 0)
-
-
class
catalyst.utils.meters.aucmeter.
AUCMeter
[source]¶ Bases:
catalyst.utils.meters.meter.Meter
The AUCMeter measures the area under the receiver-operating characteristic (ROC) curve for binary classification problems. The area under the curve (AUC) can be interpreted as the probability that, given a randomly selected positive example and a randomly selected negative example, the positive example is assigned a higher score by the classification model than the negative example.
The AUCMeter is designed to operate on one-dimensional Tensors output and target, where (1) the output contains model output scores that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a signoid function); and (2) the target contains only values 0 (for negative examples) and 1 (for positive examples).
-
class
catalyst.utils.meters.confusionmeter.
ConfusionMeter
(k, normalized=False)[source]¶ Bases:
catalyst.utils.meters.meter.Meter
Maintains a confusion matrix for a given calssification problem.
The ConfusionMeter constructs a confusion matrix for a multi-class classification problems. It does not support multi-label, multi-class problems: for such problems, please use MultiLabelConfusionMeter.
- Parameters
k (int) – number of classes in the classification problem
normalized (boolean) – Determines whether or not the confusion matrix is normalized or not
-
add
(predicted, target)[source]¶ Computes the confusion matrix of K x K size where K is no of classes
Args: predicted (tensor): Can be an N x K tensor of predicted scores obtained from the model for N examples and K classes or an N-tensor of integer values between 0 and K-1. target (tensor): Can be a N-tensor of integer values assumed to be integer values between 0 and K-1 or N x K tensor, where targets are assumed to be provided as one-hot vectors
-
class
catalyst.utils.meters.mapmeter.
mAPMeter
[source]¶ Bases:
catalyst.utils.meters.meter.Meter
The mAPMeter measures the mean average precision over all classes.
The mAPMeter is designed to operate on NxK Tensors output and target, and optionally a Nx1 Tensor weight where (1) the output contains model output scores for N examples and K classes that ought to be higher when the model is more convinced that the example should be positively labeled, and smaller when the model believes the example should be negatively labeled (for instance, the output of a sigmoid function); (2) the target contains only values 0 (for negative examples) and 1 (for positive examples); and (3) the weight ( > 0) represents weight for each sample.