- Type
- Supervised classification algorithm
- Output
- Probability between 0 and 1
- Core function
- Logistic / sigmoid
- Trained by
- Maximum likelihood estimation
- Related
- Linear regression, Neural network
- Type
- Supervised classification algorithm
- Output
- Probability between 0 and 1
- Core function
- Logistic / sigmoid
- Trained by
- Maximum likelihood estimation
- Related
- Linear regression, Neural network
Overview
Logistic regression is a supervised learning algorithm used to estimate the probability that an input belongs to a particular class. Despite its name it is a classification method rather than a regression method in the usual sense. It is one of the most widely used models in statistics, epidemiology, finance and machine learning because it is simple, fast to train, and produces interpretable, probabilistic outputs.
In its basic form, logistic regression handles binary classification, distinguishing between two outcomes such as fraud or legitimate, default or repay, disease or healthy. Extensions handle problems with more than two classes.
How it works
Logistic regression computes a weighted sum of the input features, the same linear combination used in linear regression, and then passes the result through the logistic or sigmoid function. The sigmoid maps any real number to a value between 0 and 1, which can be interpreted as a probability. In plain notation, the model computes z = w*x + b and then p = 1 / (1 + e^(-z)).
A decision threshold, commonly 0.5, converts the probability into a class label. The model parameters — the weights and the bias — are estimated from training data by maximising the likelihood of the observed labels, which is equivalent to minimising a loss called binary cross-entropy or log loss. Because this loss is convex, optimisation by gradient descent reliably finds the global optimum.
Multiclass extension
For problems with more than two classes, multinomial logistic regression, also called softmax regression, replaces the sigmoid with the softmax function to produce a probability distribution over all classes. An alternative is the one-versus-rest scheme, which trains a separate binary classifier for each class.
Regularisation and interpretation
To prevent overfitting, logistic regression is often trained with L1 or L2 regularisation, which penalise large weights. L1 regularisation can drive some weights to zero, performing implicit feature selection. A key strength of the model is interpretability: each weight indicates the direction and strength of a feature's influence on the outcome, and exponentiating a weight yields an odds ratio that domain experts can reason about.
Strengths and limitations
Logistic regression is computationally cheap, robust on linearly separable problems, and resistant to overfitting when regularised. Its main limitation is that it models a linear decision boundary in feature space, so it struggles with complex non-linear relationships unless features are engineered or transformed. For highly non-linear problems, tree ensembles or neural networks usually perform better, but logistic regression remains a strong, interpretable baseline.
Applications
The method underpins credit scoring, medical diagnosis, marketing response models, spam filtering and click-through-rate prediction. It also forms the final layer of many neural networks, where a sigmoid or softmax output effectively reproduces logistic regression on learned features.
Logistic regression is a workhorse in Malaysia's financial sector, where Maybank, CIMB, RHB and Bank Rakyat use it for credit scoring and probability-of-default models. Its transparency is valued because Bank Negara Malaysia expects financial institutions to explain and validate credit decisions, and interpretable odds ratios make logistic models easier to defend to regulators than opaque alternatives.
In healthcare, researchers at Malaysian institutions such as Universiti Malaya Medical Centre and the Ministry of Health use logistic regression for clinical risk prediction, including models for dengue severity, cardiovascular risk and hospital readmission, areas where the relationship between risk factors and outcomes must remain interpretable to clinicians.
Insurers and takaful operators regulated under the Securities Commission Malaysia and Bank Negara Malaysia apply logistic models for claims and underwriting decisions. The method is a standard component of data science training subsidised by HRD Corp and taught across Malaysian universities, and any model trained on customer records must comply with the Personal Data Protection Act 2010.