fl4health.metrics package

class Accuracy(name='accuracy')

Bases: SimpleMetric

__init__(name='accuracy')

Accuracy metric for classification tasks.

Parameters:

name (str) – The name of the metric.

class BalancedAccuracy(name='balanced_accuracy')

Bases: SimpleMetric

__init__(name='balanced_accuracy')

Balanced accuracy metric for classification tasks. Used for the evaluation of imbalanced datasets. For more information:

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.balanced_accuracy_score.html

class BinarySoftDiceCoefficient(name='BinarySoftDiceCoefficient', epsilon=1e-07, spatial_dimensions=(2, 3, 4), logits_threshold=0.5)

Bases: SimpleMetric

__init__(name='BinarySoftDiceCoefficient', epsilon=1e-07, spatial_dimensions=(2, 3, 4), logits_threshold=0.5)

Binary DICE Coefficient Metric with configurable spatial dimensions and logits threshold.

Parameters:

• name (str) – Name of the metric.

• epsilon (float) – Small float to add to denominator of DICE calculation to avoid divide by 0.

• spatial_dimensions (tuple[int, ...]) – The spatial dimensions of the image within the prediction tensors. The default assumes that the images are 3D and have shape: (batch_size, channel, spatial, spatial, spatial)

• logits_threshold (float | None) – This is a threshold value where values above are classified as 1 and those below are mapped to 0. If the threshold is None, then no thresholding is performed and a continuous or “soft” DICE coefficient is computed.

class F1(name='F1 score', average='weighted')

Bases: SimpleMetric

__init__(name='F1 score', average='weighted')

Computes the F1 score using the sklearn f1_score function. As such, the values of average correspond to those of that function.

Parameters:

• name (str, optional) – Name of the metric. Defaults to “F1 score”.

• average (str | None, optional) –

  Whether to perform averaging of the F1 scores and how. The values of this string corresponds to those of the sklearn f1_score function. See:

  https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html

  Defaults to “weighted”.

class ROC_AUC(name='ROC_AUC score')

Bases: SimpleMetric

__init__(name='ROC_AUC score')

Area under the Receiver Operator Curve (AUCROC) metric for classification. For more information:

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.balanced_accuracy_score.html

class SimpleMetric(name)

Bases: Metric, ABC

__init__(name)

Abstract metric class with base functionality to update, compute and clear metrics. User needs to define __call__ method which returns metric given inputs and target.

Parameters:

name (str) – Name of the metric.

clear()

Resets metrics by clearing input and target lists.

Return type:

None

compute(name=None)

Compute metric on accumulated input and output over updates.

Parameters:

name (str | None) – Optional name used in conjunction with class attribute name to define key in metrics dictionary.

Raises:

AssertionError – Input and target lists must be non empty.

Returns:

A dictionary of string and Scalar representing the computed metric and its associated key.

Return type:

Metrics

update(input, target)

This method updates the state of the metric by appending the passed input and target pairing to their respective list.

Parameters:

• input (torch.Tensor) – The predictions of the model to be evaluated.

• target (torch.Tensor) – The ground truth target to evaluate predictions against.

Return type:

None

class TorchMetric(name, metric)

Bases: Metric

__init__(name, metric)

Thin wrapper on TorchMetric to make it compatible with our Metric interface.

Parameters:

• name (str) – The name of the metric.

• metric (TMetric) – TorchMetric class based metric

clear()

Resets metric.

Raises:

NotImplementedError – To be defined in the classes expending this class.

Return type:

None

compute(name=None)

Compute value of underlying TorchMetric.

Parameters:

name (str | None) – Optional name used in conjunction with class attribute name to define key in metrics dictionary.

Returns:

A dictionary of string and Scalar representing the computed metric and its associated key.

Return type:

Metrics

update(input, target)

Updates the state of the underlying TorchMetric.

Parameters:

• input (torch.Tensor) – The predictions of the model to be evaluated.

• target (torch.Tensor) – The ground truth target to evaluate predictions against.

Return type:

None

Submodules

• fl4health.metrics.base_metrics module
  • Metric
    • Metric.__init__()
    • Metric.clear()
    • Metric.compute()
    • Metric.update()
  • MetricPrefix
    • MetricPrefix.TEST_PREFIX
    • MetricPrefix.VAL_PREFIX
• fl4health.metrics.compound_metrics module
  • EmaMetric
    • EmaMetric.__init__()
    • EmaMetric.clear()
    • EmaMetric.compute()
    • EmaMetric.update()
  • TransformsMetric
    • TransformsMetric.__init__()
    • TransformsMetric.clear()
    • TransformsMetric.compute()
    • TransformsMetric.update()
• fl4health.metrics.efficient_metrics module
  • BinaryDice
    • BinaryDice.__init__()
    • BinaryDice.compute_from_counts()
  • MultiClassDice
    • MultiClassDice.__init__()
    • MultiClassDice.compute_from_counts()
• fl4health.metrics.efficient_metrics_base module
  • BinaryClassificationMetric
    • BinaryClassificationMetric.__init__()
    • BinaryClassificationMetric.compute_from_counts()
    • BinaryClassificationMetric.count_tp_fp_tn_fn()
  • ClassificationMetric
    • ClassificationMetric.__init__()
    • ClassificationMetric.clear()
    • ClassificationMetric.compute()
    • ClassificationMetric.compute_from_counts()
    • ClassificationMetric.count_tp_fp_tn_fn()
    • ClassificationMetric.update()
  • MetricOutcome
    • MetricOutcome.FALSE_NEGATIVE
    • MetricOutcome.FALSE_POSITIVE
    • MetricOutcome.TRUE_NEGATIVE
    • MetricOutcome.TRUE_POSITIVE
  • MultiClassificationMetric
    • MultiClassificationMetric.__init__()
    • MultiClassificationMetric.compute_from_counts()
    • MultiClassificationMetric.count_tp_fp_tn_fn()
• fl4health.metrics.metric_aggregation module
  • evaluate_metrics_aggregation_fn()
  • fit_metrics_aggregation_fn()
  • metric_aggregation()
  • normalize_metrics()
  • uniform_evaluate_metrics_aggregation_fn()
  • uniform_metric_aggregation()
  • uniform_normalize_metrics()
• fl4health.metrics.metric_managers module
  • MetricManager
    • MetricManager.__init__()
    • MetricManager.check_target_prediction_keys_equal()
    • MetricManager.clear()
    • MetricManager.compute()
    • MetricManager.reset()
    • MetricManager.update()
• fl4health.metrics.metrics module
  • Accuracy
    • Accuracy.__init__()
  • BalancedAccuracy
    • BalancedAccuracy.__init__()
  • BinarySoftDiceCoefficient
    • BinarySoftDiceCoefficient.__init__()
  • F1
    • F1.__init__()
  • ROC_AUC
    • ROC_AUC.__init__()
  • SimpleMetric
    • SimpleMetric.__init__()
    • SimpleMetric.clear()
    • SimpleMetric.compute()
    • SimpleMetric.update()
  • TorchMetric
    • TorchMetric.__init__()
    • TorchMetric.clear()
    • TorchMetric.compute()
    • TorchMetric.update()
• fl4health.metrics.metrics_utils module
  • compute_dice_on_count_tensors()
  • threshold_tensor()
• fl4health.metrics.utils module
  • align_pred_and_target_shapes()
  • infer_label_dim()
  • map_label_index_tensor_to_one_hot()