# Ranking¶

## H-score¶

tllib.ranking.hscore.h_score(features, labels)[source]

The H-Score $$\mathcal{H}$$ can be described as:

$\mathcal{H}=\operatorname{tr}\left(\operatorname{cov}(f)^{-1} \operatorname{cov}\left(\mathbb{E}[f \mid y]\right)\right)$

where $$f$$ is the features extracted by the model to be ranked, $$y$$ is the groud-truth label vector

Parameters
• features (np.ndarray) – features extracted by pre-trained model.

• labels (np.ndarray) – groud-truth labels.

Shape:
• features: (N, F), with number of samples N and feature dimension F.

• labels: (N, ) elements in [0, $$C_t$$), with target class number $$C_t$$.

• score: scalar.

## LEEP: Log Expected Empirical Prediction¶

tllib.ranking.leep.log_expected_empirical_prediction(predictions, labels)[source]

Log Expected Empirical Prediction in LEEP: A New Measure to Evaluate Transferability of Learned Representations (ICML 2020).

The LEEP $$\mathcal{T}$$ can be described as:

$\mathcal{T}=\mathbb{E}\log \left(\sum_{z \in \mathcal{C}_s} \hat{P}\left(y \mid z\right) \theta\left(y \right)_{z}\right)$

where $$\theta\left(y\right)_{z}$$ is the predictions of pre-trained model on source category, $$\hat{P}\left(y \mid z\right)$$ is the empirical conditional distribution estimated by prediction and ground-truth label.

Parameters
• predictions (np.ndarray) – predictions of pre-trained model.

• labels (np.ndarray) – groud-truth labels.

Shape:
• predictions: (N, $$C_s$$), with number of samples N and source class number $$C_s$$.

• labels: (N, ) elements in [0, $$C_t$$), with target class number $$C_t$$.

• score: scalar

## NCE: Negative Conditional Entropy¶

tllib.ranking.nce.negative_conditional_entropy(source_labels, target_labels)[source]

Negative Conditional Entropy in Transferability and Hardness of Supervised Classification Tasks (ICCV 2019).

The NCE $$\mathcal{H}$$ can be described as:

$\mathcal{H}=-\sum_{y \in \mathcal{C}_t} \sum_{z \in \mathcal{C}_s} \hat{P}(y, z) \log \frac{\hat{P}(y, z)}{\hat{P}(z)}$

where $$\hat{P}(z)$$ is the empirical distribution and $$\hat{P}\left(y \mid z\right)$$ is the empirical conditional distribution estimated by source and target label.

Parameters
• source_labels (np.ndarray) – predicted source labels.

• target_labels (np.ndarray) – groud-truth target labels.

Shape:
• source_labels: (N, ) elements in [0, $$C_s$$), with source class number $$C_s$$.

• target_labels: (N, ) elements in [0, $$C_t$$), with target class number $$C_t$$.

## LogME: Log Maximum Evidence¶

tllib.ranking.logme.log_maximum_evidence(features, targets, regression=False, return_weights=False)[source]

Log Maximum Evidence in LogME: Practical Assessment of Pre-trained Models for Transfer Learning (ICML 2021).

Parameters
• features (np.ndarray) – feature matrix from pre-trained model.

• targets (np.ndarray) – targets labels/values.

• regression (bool, optional) – whether to apply in regression setting. (Default: False)

• return_weights (bool, optional) – whether to return bayesian weight. (Default: False)

Shape:
• features: (N, F) with element in [0, $$C_t$$) and feature dimension F, where $$C_t$$ denotes the number of target class

• targets: (N, ) or (N, C), with C regression-labels.

• weights: (F, $$C_t$$).

• score: scalar.