Supervised Classifier Benchmarking

Multiclass Machine Learning Benchmark

Benchmarking 4 supervised model families under Stratified 5-Fold Cross-Validation on 120 task-fMRI contrast maps from the Brainomics Localizer dataset.

Highest Accuracy

Random Forest

Top discrete classification accuracy of 85.83% (max_depth=6, n_estimators=300).

Highest Macro AUROC

Logistic Regression

Top macro AUROC of 0.9656 (85.00% Acc, C=0.01). Selected for network-level interpretability.

Validation Protocol

Stratified 5-Fold

Preserves balanced 30-sample class distribution per fold with in-fold standard scaling to avoid data leakage.

Classifier Performance Benchmark (%)

Comparing 4 supervised models under Stratified 5-Fold Cross-Validation

N = 120 Contrast Maps

Logistic Regression Confusion Matrix (85.00%)

Rows = True Task Labels | Columns = Predicted Task Labels

Selected Interpretable Model

True \ Pred	Working Memory / Math	Language / Reading	Motor Control	Visual Processing
Working Memory / Math	25	3	1	1
Language / Reading	4	22	0	4
Motor Control	1	0	27	2
Visual Processing	1	1	0	28

True: Working Memory / Math → Predicted: Working Memory / Math

25 / 30 Maps (83.3%)

Task-Level Classification Metrics Table (IEEE Table I Replica)

ML Model	Cognitive Task	Accuracy	Precision	Recall	F1-Score	AUROC
Random ForestHighest Acc	Math	85.8%	0.78	0.83	0.81	0.9396
	Language		0.79	0.73	0.76	0.8967
	Motor		0.94	1.00	0.97	0.9996
	Visual		0.93	0.87	0.90	0.9526
Logistic RegressionBest Interpretable	Math	85.0%	0.78	0.83	0.81	0.9667
	Language		0.76	0.73	0.75	0.9211
	Motor		1.00	0.90	0.95	1.0000
	Visual		0.88	0.93	0.90	0.9748
Decision Tree	Math	73.3%	0.88	0.73	0.80	0.8776
	Language		0.58	0.87	0.69	0.8502
	Motor		0.86	0.80	0.83	0.9237
	Visual		0.73	0.53	0.62	0.7785
K-Nearest Neighbors	Math	66.7%	0.72	0.60	0.65	0.8469
	Language		0.53	0.70	0.60	0.7859
	Motor		1.00	0.60	0.75	0.9000
	Visual		0.62	0.77	0.69	0.9019