Transform Random Variables and Track Mean/SD

It centers the data at zero and scales by the standard deviation: z = (x − μ) / σ. The transformed variable has mean 0 and SD 1 by construction. Shape is preserved (symmetric stays symmetric, skewed stays skewed); only location and scale change. Useful for: comparing values across different distributions on a common scale, removing scale dependence in ML models that use distance metrics (k-NN, k-means, SVM with RBF kernel), and reading deviations in standard-deviation units (z = 2.5 means "2.5 SDs above the mean" regardless of the original units).

Min-max maps your data to [0, 1] via (x − min) / (max − min). Z-score gives mean 0 and SD 1. Use min-max when you need bounded output (neural network inputs, image pixel values), but be aware: it's extremely sensitive to outliers. One extreme value squeezes everything else into a tiny slice of [0, 1]. Z-score is more robust to outliers and is what statistics defaults to. For ML feature scaling, robust scaling (median and IQR) is the safe default when you don't know whether outliers exist.

Distance-based and gradient-based algorithms care about scale. k-nearest-neighbors with one feature ranging 0-1 and another ranging 0-1000 effectively only uses the second feature in distance calculations. Gradient descent for neural networks converges much slower with un-scaled inputs because gradient steps are mismatched to feature scales. Tree-based models (random forest, gradient boosting) don't care about scale. Linear regression coefficients depend on scale (β changes if you change units), so for interpretation, standardize first if comparing coefficient magnitudes across predictors.

Subtract the median, divide by the IQR. Resistant to outliers because median and IQR don't move under extreme values, unlike mean and SD. sklearn.preprocessing.RobustScaler is the standard implementation. Use when you can't visually inspect for outliers and want a safe default. Use when downstream models are sensitive to outliers but you want to keep them in the dataset. The cost: the transformed data don't have nice statistical properties (mean isn't exactly 0, SD isn't exactly 1), but for most ML pipelines that doesn't matter.

x = z · σ + μ. The reverse of z = (x − μ) / σ. If you computed z = 1.5 with μ = 100 and σ = 15, then x = 1.5 · 15 + 100 = 122.5. The inversion needs the same μ and σ used in the forward transform. For ML pipelines, this means the scaler object holds the parameters: sklearn's StandardScaler.fit() learns μ and σ from training data; .inverse_transform() applies the inverse. Saving these parameters (or the fitted scaler) is essential when you deploy a model to production data.

From the definitions. Mean is linear: E[aX + b] = a·E[X] + b. Variance is quadratic: Var(aX + b) = E[(aX + b − E[aX + b])²] = E[a²(X − E[X])²] = a²·Var(X). The constant b drops out because it shifts both X and E[X] by the same amount, leaving the deviation unchanged. The factor a gets squared because the deviation is squared. Standard deviation, being the square root, scales by |a|, with absolute value because SD must be non-negative. The asymmetry surprises people but follows directly from the algebra.

Linear (Y = aX + b): preserves shape, mean and SD transform predictably, the inverse is also linear. Z-score, Fahrenheit-to-Celsius, percentage points to proportions are all linear. Nonlinear (log, square root, square, exp): changes shape, breaks the closed-form rules for E and Var, often non-invertible without further constraints. Log transforms are common because they pull in long right tails (income, response times, counts). Box-Cox and Yeo-Johnson generalize the log family with a tunable parameter. For ML feature engineering, nonlinear transforms are routine; the price is losing the simple linear-algebra rules.

Fit the scaler on training data only. Apply the same fitted scaler to validation and test sets. If you fit on the full dataset (train + test), you've leaked test-set information into the training process, which inflates apparent performance and can mask real-world failures. sklearn handles this correctly via the fit/transform pattern: scaler.fit(X_train), then scaler.transform(X_test). Pipeline objects (sklearn.pipeline.Pipeline) chain the scaler with the model so cross-validation respects the train/test boundary. Manual standardization with np.mean(X) and np.std(X) on the full data is the most common cause of leakage I see in production code reviews.