Confusion Matrix Calculator

Confusion Matrix for Binary Classification

In a binary classification problem, you have two classes: typically called positive and negative (or "class 1" and "class 0", "yes" and "no", etc.). The confusion matrix is a 2×2 table that compares your model's predictions to the actual ground truth. One dimension represents the actual class (what the true label was), and the other represents the predicted class (what your model said).

The four cells of the matrix are:

• True Positive (TP): The model predicted positive, and the actual class was positive. (Correct positive prediction.)
• False Positive (FP): The model predicted positive, but the actual class was negative. (Incorrect positive prediction; also called a "Type I error" or "false alarm.")
• True Negative (TN): The model predicted negative, and the actual class was negative. (Correct negative prediction.)
• False Negative (FN): The model predicted negative, but the actual class was positive. (Incorrect negative prediction; also called a "Type II error" or "miss.")

Visually, the matrix looks like this (rows = actual, columns = predicted):

                   Predicted Positive   Predicted Negative
Actual Positive       TP                   FN
Actual Negative       FP                   TN

All classification metrics (accuracy, precision, recall, etc.) are derived from these four numbers.

Core Binary Classification Metrics

Once you have TP, FP, TN, and FN, you can compute a variety of metrics. Here are the most important ones:

• Accuracy: The proportion of all predictions that were correct.
  Accuracy = (TP + TN) / (TP + FP + TN + FN)

  Accuracy tells you the overall correctness, but can be misleading with imbalanced data.

• Precision (Positive Predictive Value, PPV): Of all the instances your model predicted as positive, how many were actually positive?
  Precision = TP / (TP + FP)

  High precision means few false alarms. Important when the cost of FP is high (e.g., spam filters, expensive follow-up tests).

• Recall (Sensitivity, True Positive Rate, TPR): Of all the instances that were actually positive, how many did your model correctly identify?
  Recall = TP / (TP + FN)

  High recall means you're catching most positives. Critical when missing a positive is costly (e.g., disease detection, fraud detection).

• Specificity (True Negative Rate, TNR): Of all the instances that were actually negative, how many did your model correctly identify as negative?
  Specificity = TN / (TN + FP)

  High specificity means you're good at ruling out negatives. Often paired with sensitivity in medical diagnostics.

• F1 Score: The harmonic mean of precision and recall, balancing both metrics.
  F1 = 2 × (Precision × Recall) / (Precision + Recall)

  F1 is useful when you want a single metric that considers both false positives and false negatives. Ranges from 0 to 1 (higher is better).

• False Positive Rate (FPR): Of all actual negatives, what fraction did you incorrectly call positive?
  FPR = FP / (FP + TN) = 1 − Specificity

  Used in ROC curve analysis. Lower FPR is better.

Class Imbalance and Why Accuracy Isn't Enough

In many real-world problems, classes are imbalanced—one class is much more common than the other. For example, in fraud detection, legitimate transactions vastly outnumber fraudulent ones (often 99:1 or worse). In such cases, a naive model that always predicts the majority class can achieve very high accuracy while being completely useless.

Example: Suppose you have 990 negatives and 10 positives. A model that predicts "negative" for everything gets 99% accuracy (990/1000), but it catches zero of the 10 positives—100% false negatives! The confusion matrix exposes this:

TP = 0,  FP = 0
TN = 990, FN = 10

Accuracy = 99%
Recall = 0% (catches no frauds)
Precision = undefined (no positive predictions)

This is why precision, recall, F1, and balanced accuracy are critical for imbalanced datasets. They focus on how well you handle the minority class, not just overall correctness.

Multiclass Confusion Matrices

When you have more than two classes (e.g., classifying images into "cat," "dog," "bird"), the confusion matrix expands into a larger table. Each row represents an actual class, and each column represents a predicted class. The diagonal cells show correct predictions, and off-diagonal cells show misclassifications.

For each class, you can compute:

• Per-class precision: Of all predictions for class X, how many were actually class X?
• Per-class recall: Of all actual class X instances, how many were correctly predicted as X?
• Per-class F1: Harmonic mean of that class's precision and recall.

To summarize performance across all classes, you can use:

• Macro-average: Compute the metric for each class, then take the unweighted average. Treats all classes equally.
• Micro-average: Pool all TP, FP, FN across all classes, then compute a single metric. Weights by class frequency.
• Weighted average: Average the per-class metrics, weighted by the number of true instances in each class (support).

This calculator supports multiclass confusion matrices, per-class metrics, and macro/micro/weighted F1 scores—helping you understand performance across complex classification tasks.

Mode 1 — Binary Confusion Matrix from TP, FP, TN, FN

1. Select "Binary" from the Classification Type dropdown.
2. Select "Counts" as the Input Mode.
3. Enter your counts:
   • True Positives (TP): Number of correctly predicted positives.
   • False Positives (FP): Number of incorrectly predicted positives (actual negatives).
   • True Negatives (TN): Number of correctly predicted negatives.
   • False Negatives (FN): Number of incorrectly predicted negatives (actual positives).
4. (Optional) Choose normalization: "None" shows raw counts, "Row" shows rates by actual class, "Column" shows rates by predicted class, "All" shows proportions of total.
5. Click Calculate.
6. Review the results:
   • The confusion matrix table (with diagonal = correct, off-diagonal = errors).
   • Accuracy, precision, recall, F1 score, specificity, FPR, balanced accuracy, MCC, Youden's J.
   • ROC and Precision-Recall curves (if applicable).

Use this mode when: You already have TP, FP, TN, FN counts from a model's predictions on a test set, homework problem, or diagnostic test results.

Mode 2 — Multiclass Confusion Matrix

1. Select "Multiclass" from the Classification Type dropdown.
2. Adjust the number of classes: Use the "+ Class" and "− Class" buttons to add or remove classes. The default is 3 classes.
3. Enter counts in the confusion matrix grid: Each cell (i, j) represents the number of instances with actual class i that were predicted as class j. The diagonal represents correct predictions.
4. (Optional) Edit class labels: If supported by the UI, you can rename "Class A", "Class B", etc., to match your problem (e.g., "Cat", "Dog", "Bird").
5. (Optional) Choose normalization to view rates instead of raw counts.
6. Click Calculate.
7. Review the results:
   • The full confusion matrix table.
   • Overall accuracy.
   • Macro, micro, and weighted F1 scores.
   • Per-class precision, recall, F1, and support in a summary table.
   • MCC and Cohen's Kappa for multiclass agreement.

Use this mode when: You have a multiclass classification problem (3+ classes) and want to understand how the model confuses different classes.

Understanding Normalization Options

• None: Shows raw counts (e.g., TP = 85, FP = 15).
• Row (True Rate): Each row sums to 1 (or 100%). Shows what fraction of each actual class was predicted into each predicted class. Useful for understanding recall-like metrics per class.
• Column (Predicted Rate): Each column sums to 1 (or 100%). Shows what fraction of each predicted class came from each actual class. Useful for precision-like insights.
• All: Each cell is divided by the total number of instances, so the entire matrix sums to 1. Useful for proportional visualization.

Normalization is especially helpful when comparing confusion matrices of different sizes or when you want to focus on rates rather than raw counts.

General Tips for All Modes

• Use consistent definitions: Make sure you know which class is "positive" and which is "negative" (or which labels correspond to which classes in multiclass).
• Check for warnings: The calculator will flag issues like zero denominators (e.g., no positive predictions → precision undefined) or very small sample sizes.
• Compare multiple scenarios: Run the calculator for different models, thresholds, or cross-validation folds to see how metrics change.
• Copy or export results: Use the results for reports, homework, slide decks, or further analysis.

Core Binary Classification Formulas

Let N = TP + FP + TN + FN (total number of instances).

• Accuracy:
  Accuracy = (TP + TN) / N
• Precision (PPV):
  Precision = TP / (TP + FP), if (TP + FP) > 0

  If no positive predictions, precision is undefined.

• Recall / Sensitivity / TPR:
  Recall = TP / (TP + FN), if (TP + FN) > 0

  If no actual positives, recall is undefined.

• Specificity / TNR:
  Specificity = TN / (TN + FP), if (TN + FP) > 0
• F1 Score:
  F1 = 2 × (Precision × Recall) / (Precision + Recall), if both > 0

  Harmonic mean balances precision and recall.

• False Positive Rate (FPR):
  FPR = FP / (FP + TN) = 1 − Specificity
• Balanced Accuracy:
  Balanced Accuracy = (Recall + Specificity) / 2

  Useful for imbalanced datasets; weights positive and negative classes equally.

• Matthews Correlation Coefficient (MCC):
  MCC = (TP×TN − FP×FN) / sqrt((TP+FP)(TP+FN)(TN+FP)(TN+FN))

  Ranges from -1 to +1. Considers all four cells; robust to imbalance.

Multiclass Averages (Conceptual)

For multiclass problems, you compute precision, recall, and F1 for each class (treating it as "this class vs all others"), then average:

• Macro-average: Compute metric for each class, then take the mean. Treats all classes equally, regardless of size.
  Macro-F1 = (F1_class1 + F1_class2 + ... + F1_classK) / K
• Micro-average: Pool all TP, FP, FN across classes, then compute a single metric. Weighs by class frequency.
  Micro-F1 = 2 × (Precision_micro × Recall_micro) / (Precision_micro + Recall_micro)

  Where Precision_micro = sum(TP) / sum(TP + FP) across all classes.

• Weighted average: Average per-class metrics, weighted by support (number of true instances per class).

Worked Example 1 — Balanced Binary Case

Worked Example 2 — Imbalanced Case

1. Medical Diagnostic Test Evaluation (Homework or Real Data)

Scenario: A biostatistics student is given data from a diagnostic test for a disease. Out of 500 patients, 50 have the disease. The test results are: TP = 45, FN = 5, TN = 430, FP = 20.

Using the calculator: Enter the counts, calculate sensitivity (recall = 90%), specificity (95.6%), and predictive values. The student uses these metrics in a homework write-up, discussing the trade-off between sensitivity (catching diseases) and specificity (avoiding false alarms).

Result: The calculator shows that the test has high sensitivity (good at detecting disease) but some false positives. The student learns that no test is perfect and that metric choice depends on the clinical context.

2. Fraud Detection Model Validation

Scenario: A data scientist trains a fraud detection model and evaluates it on a validation set. The confusion matrix shows low precision (many false positives) but high recall (catches most frauds).

Using the calculator: Input the TP, FP, TN, FN counts, compute precision, recall, and F1. Compare multiple models or thresholds to find the sweet spot that minimizes false alarms while maintaining high fraud catch rate.

Result: The calculator helps visualize the precision-recall trade-off, guiding threshold tuning and model selection for deployment.

3. Spam Email Filter Performance Check

Scenario: A team builds a spam filter and tests it on 1,000 emails (900 legitimate, 100 spam). The confusion matrix shows TP = 95 spam caught, FP = 10 legitimate emails mistakenly blocked, TN = 890, FN = 5 spam missed.

Using the calculator: Compute precision (95/(95+10) ≈ 90.5%), recall (95%), F1, and FPR (1.1%). The team sees that precision is good (few real emails blocked) and recall is high (few spam missed).

Result: The tool confirms the filter is effective, with low false positive rate (important for user experience) and high recall (important for spam reduction).

4. Customer Churn Prediction for Marketing

Scenario: A company predicts which customers will churn (cancel subscriptions). The confusion matrix reveals that the model has moderate precision but low recall—it misses many churners.

Using the calculator: Enter counts, see recall is only 60%, meaning 40% of churners are not flagged. The marketing team decides to adjust the decision threshold to increase recall (catch more churners) at the cost of more false positives (wasted outreach).

Result: The calculator helps the team understand the business trade-off: more false positives mean wasted marketing spend, but higher recall means fewer missed opportunities to retain customers.

5. Machine Learning Course Assignment

Scenario: Students in an ML course train a classifier on the Iris dataset (multiclass: 3 flower species). They compute a 3×3 confusion matrix from test predictions and need to report per-class precision/recall and macro-average F1.

Using the calculator: Select multiclass mode, enter the 3×3 counts, calculate. The tool shows per-class precision/recall and macro/micro F1. Students see which classes are easiest/hardest to predict and discuss why in their report.

Result: The calculator automates the tedious arithmetic, letting students focus on interpretation and learning.

6. Credit Scoring Model Evaluation

Scenario: A bank evaluates a credit scoring model that predicts loan default (positive = default, negative = repay). The confusion matrix shows high recall (catches most defaults) but low precision (many false alarms, denying credit to good borrowers).

Using the calculator: Input counts, review precision, recall, F1. The bank uses these metrics to calibrate the model's risk tolerance and ensure regulatory compliance.

Result: The calculator provides quantitative backing for business decisions about acceptable default rates vs credit access.

7. Image Classification Model Debugging

Scenario: A computer vision practitioner trains a model to classify images into "cat", "dog", "bird". The multiclass confusion matrix reveals that the model often confuses "dog" and "cat" but almost never mistakes "bird".

Using the calculator: Enter the 3×3 confusion matrix, see per-class recall and the off-diagonal counts. The practitioner identifies that dog/cat confusion is the main error and decides to collect more training data or engineer better features to distinguish these classes.

Result: The confusion matrix guides targeted model improvement rather than generic "increase accuracy" goals.

8. A/B Testing for Recommendation System

Scenario: A product team tests two recommendation algorithms (A and B) and measures whether users click on recommended items (positive = click, negative = no click). They build confusion matrices for both models on the same test set.

Using the calculator: Run the tool twice (once for each model's confusion matrix), compare precision, recall, and F1. Model A has higher recall (shows more relevant items) but lower precision (more irrelevant items), while Model B has the opposite trade-off.

Result: The team uses the confusion matrix metrics to decide which model aligns better with business goals (e.g., if showing more relevant items is more important than avoiding irrelevant ones, choose Model A).

1. Swapping Positive and Negative Labels

Interpreting metrics incorrectly because you mixed up which class is "positive" (e.g., calling "not fraud" the positive class when fraud is actually the positive class). Solution: Always clarify your class definitions before entering counts. "Positive" should be the class of interest (disease, fraud, churn, etc.).

2. Confusing Precision with Recall

Assuming that high precision means you're catching most positives (that's recall's job). Precision tells you how many of your positive predictions are correct; recall tells you how many actual positives you found. Solution: Remember: precision = "of predicted positives, how many are right?", recall = "of actual positives, how many did we catch?"

3. Focusing Only on Accuracy

Relying on accuracy as the sole metric when classes are imbalanced. A 99% accurate model might catch zero positives if positives are 1% of data. Solution: Use precision, recall, F1, and balanced accuracy to understand minority-class performance. Accuracy should never be the only metric you report.

4. Ignoring the Cost Asymmetry of Errors

Treating false positives and false negatives as equally bad when they have very different real-world costs. Example: In cancer screening, FN (missed cancer) is far more costly than FP (unnecessary follow-up test). Solution: Choose metrics that align with your problem's cost structure. Prioritize recall for high-cost FNs, precision for high-cost FPs.

5. Using Tiny Sample Sizes

Drawing strong conclusions from confusion matrices with very small counts (e.g., TP = 3, FN = 1 → "75% recall!"). Small samples have high variance; a single error drastically changes metrics. Solution: Collect more data or use confidence intervals / cross-validation. Be cautious about interpreting results from small test sets.

6. Misinterpreting Macro vs Micro Averages (Multiclass)

Not realizing that macro-average treats all classes equally (good for balanced evaluation) while micro-average weighs by class frequency (good when you care more about common classes). Solution: Understand what each averaging method emphasizes. Use macro-average for class-balanced insights, micro-average when larger classes are more important.

7. Comparing Models Without Consistent Test Sets

Evaluating two models on different test sets and comparing their confusion matrices directly. Different data distributions can make comparisons meaningless. Solution: Always compare models on the same held-out test set or cross-validation folds.

8. Forgetting That Metrics Are Threshold-Dependent

For probabilistic classifiers (output probabilities), the confusion matrix depends on the decision threshold (e.g., predict positive if probability > 0.5). Changing the threshold changes TP, FP, TN, FN and thus all derived metrics. Solution: Explore multiple thresholds using ROC and PR curves, not just a single confusion matrix. The calculator can help you understand each threshold's trade-offs.

9. Overinterpreting F1 Score Alone

Using F1 as the sole metric without checking precision and recall separately. F1 can hide important information: a model with 100% precision but 10% recall has a low F1, but you might care more about that 100% precision in some contexts. Solution: Always look at precision, recall, and F1 together, not just F1 in isolation.

10. Not Checking for Label Leakage or Data Issues

Achieving suspiciously perfect metrics (e.g., 100% accuracy) because of label leakage (test labels leaked into training) or data errors (duplicates, mislabeled examples). Solution: Sanity-check results. If metrics seem too good to be true, audit your data pipeline and train/test splits before celebrating.

1. Choose Metrics Based on Problem Domain and Costs

Different applications prioritize different metrics. Safety-critical problems (disease detection, fraud) often prioritize recall (don't miss any positives). Low-stakes false positives (spam filters) prioritize precision (don't annoy users with false alarms). Balanced problems use F1. Solution: Before running the calculator, identify which error type is more costly in your problem and choose your target metric accordingly.

2. Use ROC and Precision-Recall Curves for Threshold Selection

A single confusion matrix is just one point on a curve. ROC curves (TPR vs FPR) and PR curves (Precision vs Recall) show how metrics change across all thresholds. Solution: If your classifier outputs probabilities, explore multiple thresholds using these curves (the calculator shows them if data supports it). Choose the threshold that best balances your business trade-offs, not just the default 0.5.

3. Use Confusion Matrices to Guide Feature Engineering and Data Collection

Off-diagonal cells in a multiclass confusion matrix reveal which classes the model confuses. Example: If "dog" and "wolf" are often confused, you might add features that distinguish size or context (domestic vs wild). Solution: Treat the confusion matrix as a diagnostic tool—identify systematic errors and address them with targeted improvements.

4. Aggregate Confusion Matrices Across Cross-Validation Folds

In k-fold cross-validation, you get k confusion matrices (one per fold). You can sum them element-wise to get an aggregate confusion matrix, then compute overall metrics. Solution: Use the calculator to compute metrics for the aggregated matrix, giving a more stable estimate of performance than any single fold.

5. Report Confidence Intervals or Bootstrap Estimates

Confusion matrix metrics are point estimates; they have uncertainty. For small test sets, report confidence intervals (via bootstrap or exact methods). Solution: While the calculator gives point estimates, supplement them with uncertainty quantification in reports or papers to show robustness.

6. Compare Confusion Matrices Side-by-Side for Model Selection

When choosing between models, compute confusion matrices for each on the same test set and compare metrics side-by-side. Solution: Use the calculator for each model, then build a comparison table (Model A: precision 85%, recall 90%; Model B: precision 90%, recall 80%). This makes trade-offs explicit.

7. Understand the Relationship Between Metrics and Business KPIs

Connect classification metrics to real-world outcomes. Example: For a recommendation system, precision might map to user satisfaction (relevant recommendations), and recall might map to revenue (showing more relevant items increases purchases). Solution: Frame confusion matrix metrics in business language when presenting to stakeholders: "Increasing recall by 10% could reduce customer churn by 5%."

8. Use Normalized Confusion Matrices for Interpretability

Raw counts can be hard to interpret when classes have very different sizes. Row normalization shows recall per class (what fraction of each actual class was correctly predicted). Column normalization shows precision per class. Solution: Use the calculator's normalization options to visualize rates instead of counts, making patterns clearer.

9. Combine Quantitative Metrics with Qualitative Error Analysis

Confusion matrices give you the what (which errors occur), but not the why. Solution: After computing metrics with the calculator, manually inspect a sample of misclassified instances (FP, FN, or off-diagonal multiclass errors) to understand root causes and guide model debugging.

10. Document Assumptions and Present Results Transparently

When presenting confusion matrix results, be explicit: "We define positive as class X, used threshold 0.6, and evaluated on a test set of N examples with class distribution Y." Solution: Clear documentation builds trust and allows others to reproduce or challenge your analysis. The calculator provides the math; you provide the context.