Understanding Logistic Regression
Educational Tool Only
This calculator is strictly for educational purposes. The coefficients you enter are hypothetical values for demonstration. In real applications, coefficients are estimated from training data using maximum likelihood estimation. Never use this tool for real medical, financial, or consequential decisions.
What is Logistic Regression?
Logistic regression is a statistical method for binary classification—predicting outcomes that have exactly two possible values (yes/no, pass/fail, spam/not spam). Unlike linear regression which predicts continuous values, logistic regression predicts the probability that an observation belongs to a particular class.
The key insight is using the sigmoid function to transform any real-valued linear combination into a probability between 0 and 1.
The Mathematics
Step 1: Linear Predictor (z)
The linear predictor combines inputs with their coefficients. This value can range from -∞ to +∞.
Step 2: Sigmoid Transformation
The sigmoid function "squashes" the linear predictor to a value between 0 and 1, which we interpret as the probability of class 1.
Step 3: Classification Decision
The default threshold is 0.5, but it can be adjusted based on the relative costs of false positives vs false negatives.
Interpreting Coefficients
β₀ (Intercept): The log-odds of the positive class when all features are zero. A positive intercept means the baseline probability is greater than 50%.
βᵢ (Feature coefficients): The change in log-odds for a one-unit increase in feature xᵢ, holding other features constant.
- Positive β: Feature increases probability of class 1
- Negative β: Feature decreases probability of class 1
- Larger |β|: Stronger effect on the outcome
Odds Ratios
The odds ratio for a coefficient is e^β. It represents how the odds multiply for each one-unit increase in the feature:
- OR = 1: No effect on odds
- OR > 1: Increased odds of positive class
- OR < 1: Decreased odds of positive class
Example: If β = 0.5, then OR = e^0.5 ≈ 1.65, meaning a one-unit increase in that feature multiplies the odds by 1.65 (65% increase in odds).
Real-World Applications
In actual practice, logistic regression requires:
- Training data: Historical examples with known outcomes
- Maximum Likelihood Estimation: Statistical method to find optimal coefficients
- Model validation: Testing on held-out data, cross-validation
- Statistical significance: p-values, confidence intervals for coefficients
- Model diagnostics: ROC curves, AUC, calibration plots
Popular applications include credit scoring, medical diagnosis, spam detection, churn prediction, and A/B test analysis.
Frequently Asked Questions
Related Math & Statistics Tools
Regression Calculator
Fit linear and polynomial regression models to your data
Probability Calculator
Calculate probabilities for various distributions
Normal Distribution
Explore the Gaussian distribution and z-scores
Descriptive Statistics
Calculate mean, median, standard deviation, and more
Confidence Interval
Estimate population parameters with confidence intervals
Bayesian Update Visualizer
See how prior beliefs update with new evidence
Correlation Significance Calculator
Test whether a correlation is statistically significant