Catalyst API also involves
for convenient metric computation during an experiment run. Its API is quite simple:
You could find all the supported metrics under the Metric API section.
Catalyst Metric API has a default update and compute methods to support per-batch statistic accumulation and final computation during training. All metrics also support update and compute key-value extensions for convenient usage during the run — it gives you the flexibility to store any number of metrics or aggregations you want with a simple communication protocol to use for their logging.
If you haven’t found the answer for your question, feel free to join our slack for the discussion.