Actual-Estimate Statistical Tests

Tests for comparing ground-truth and estimated model parameters

Statistics for Comparing True and Estimated Parameters

Module Documentation

fitr.criticism.ae_stats.param_ttest(X, Y)

Two-sample t-test for difference between parameters (actual and estimated)

Parameters:

X : ndarray(shape=(n_subjects, nparams))

Y : ndarray(shape=(n_subjects, nparams))

Returns:

res : ndarray(shape=(n_params, 2))

(t-statistic, p-value)

Notes

Arrays X and Y must be the same size

fitr.criticism.ae_stats.paramcorr(X, Y)

Pearson correlation coefficient for actual and estimated parameters

Parameters:

X : ndarray(shape=(n_subjects, nparams))

First array of parameters to be correlated

Y : ndarray(shape=(n_subjects, nparams))

Second array of parameters to be correlated

Returns:

corrs : ndarray(shape=(n_params, 2))

Array of correlation coefficients and p-values

Notes

Arrays X and Y must be the same size