ncaa_eval.transform.calibration module

Probability calibration for NCAA basketball model predictions.

Provides calibration wrappers for adjusting model-output probabilities so they are well-calibrated (when the model says 70%, the event happens ~70% of the time).

• IsotonicCalibrator — non-parametric monotonic calibration via sklearn.isotonic.IsotonicRegression. Best with >=1000 calibration samples.

• SigmoidCalibrator — parametric Platt scaling via logistic regression on log-odds. Better for small folds.

Design invariants:

• In-fold only: fit() on training fold predictions, transform() on test fold predictions. Never fit on the data being calibrated.

• goto_conversion was assessed and found not applicable — it removes bookmaker overround from betting odds, which is a fundamentally different problem from calibrating model-predicted probabilities. See Story 4.7 Dev Notes for the full assessment.

class ncaa_eval.transform.calibration.Calibrator(*args, **kwargs)

Bases: Protocol

Protocol for probability calibration transforms.

Both IsotonicCalibrator and SigmoidCalibrator structurally satisfy this protocol.

fit(y_true: ndarray[tuple[Any, ...], dtype[float64]], y_prob: ndarray[tuple[Any, ...], dtype[float64]]) → None

Fit the calibrator on observed labels and predicted probabilities.

transform(y_prob: ndarray[tuple[Any, ...], dtype[float64]]) → ndarray[tuple[Any, ...], dtype[float64]]

Transform raw probabilities into calibrated probabilities.

class ncaa_eval.transform.calibration.IsotonicCalibrator

Bases: object

Non-parametric monotonic probability calibration.

Wraps sklearn.isotonic.IsotonicRegression with y_min=0.0, y_max=1.0, and out_of_bounds="clip" for probability bounds.

Example:

cal = IsotonicCalibrator()
cal.fit(y_true_train, y_prob_train)
calibrated = cal.transform(y_prob_test)

fit(y_true: ndarray[tuple[Any, ...], dtype[float64]], y_prob: ndarray[tuple[Any, ...], dtype[float64]]) → None

Fit the isotonic regression on training fold predictions.

Parameters:

• y_true – Binary labels (0 or 1) from the training fold.

• y_prob – Model-predicted probabilities from the training fold.

transform(y_prob: ndarray[tuple[Any, ...], dtype[float64]]) → ndarray[tuple[Any, ...], dtype[float64]]

Apply calibration to test fold predictions.

Parameters:

y_prob – Model-predicted probabilities to calibrate.

Returns:

Calibrated probabilities in [0, 1].

Raises:

RuntimeError – If fit() has not been called.

class ncaa_eval.transform.calibration.SigmoidCalibrator

Bases: object

Parametric Platt scaling for probability calibration.

Uses logistic regression to fit a sigmoid function mapping raw probabilities to calibrated probabilities. More robust than isotonic regression for small samples (<1000).

Example:

cal = SigmoidCalibrator()
cal.fit(y_true_train, y_prob_train)
calibrated = cal.transform(y_prob_test)

fit(y_true: ndarray[tuple[Any, ...], dtype[float64]], y_prob: ndarray[tuple[Any, ...], dtype[float64]]) → None

Fit Platt scaling parameters on training fold predictions.

Parameters:

• y_true – Binary labels (0 or 1) from the training fold.

• y_prob – Model-predicted probabilities from the training fold.

transform(y_prob: ndarray[tuple[Any, ...], dtype[float64]]) → ndarray[tuple[Any, ...], dtype[float64]]

Apply sigmoid calibration to test fold predictions.

Parameters:

y_prob – Model-predicted probabilities to calibrate.

Returns:

Calibrated probabilities in [0, 1].

Raises:

RuntimeError – If fit() has not been called.