- Field
- AI security and robustness
- Core threat
- Crafted inputs that fool models
- Attack types
- Evasion, poisoning, extraction, inference
- Notable methods
- FGSM, PGD
- Defences
- Adversarial training, detection
- Standards
- NIST AI adversarial taxonomy
- Field
- AI security and robustness
- Core threat
- Crafted inputs that fool models
- Attack types
- Evasion, poisoning, extraction, inference
- Notable methods
- FGSM, PGD
- Defences
- Adversarial training, detection
- Standards
- NIST AI adversarial taxonomy
Adversarial machine learning is the study of how machine learning systems can be deliberately manipulated, and of the techniques used to defend against such manipulation. A central finding of the field is that models which perform well on ordinary data can be highly vulnerable to inputs that have been crafted to deceive them. In the best-known example, adding a small, carefully computed perturbation to an image, often imperceptible to a human, can cause an image classifier to confidently assign the wrong label.
These crafted inputs are called adversarial examples, and they reveal that the patterns a model learns are not always the robust, human-like features one might assume. The field has become increasingly important as machine learning is deployed in security-sensitive settings such as autonomous vehicles, fraud detection, medical diagnosis, and content moderation.
Categories of attack
Adversarial attacks are commonly grouped by their goal and the stage of the pipeline they target. Evasion attacks occur at inference time, modifying an input so the model produces an incorrect output; an attacker might subtly alter network traffic so that malicious activity appears normal to an intrusion-detection system. Poisoning attacks corrupt the training data so that the resulting model behaves incorrectly or contains a hidden backdoor. Model extraction attacks query a deployed model repeatedly to steal a functional copy of it. Inference attacks, including membership inference, attempt to recover information about the training data.
A further distinction is between white-box attacks, where the attacker knows the model's internal parameters and can compute perturbations directly, and black-box attacks, where the attacker can only observe inputs and outputs and must estimate how to fool the model.
Methods and defences
Two classic techniques for generating adversarial examples are the Fast Gradient Sign Method (FGSM), which perturbs an input in the direction that most increases the model's error, and Projected Gradient Descent (PGD), a stronger iterative method that is widely used to benchmark robustness. Defending against these attacks is difficult. The most effective general approach is adversarial training, in which the model is trained on adversarial examples alongside normal data so that it learns to resist them, though this raises training cost and can reduce accuracy on clean inputs. Other defences include input preprocessing, detecting anomalous inputs, and ensemble methods, but no defence is universally robust, and the field remains an ongoing contest between attacks and countermeasures.
Relevance in 2025 and 2026
As of 2025, adversarial machine-learning research concentrates on automotive systems, healthcare, electrical power and energy systems, and large language models. Attacks on vision-language models and on the vision-LiDAR fusion used in autonomous driving have grown, and large language models face their own adversarial pressures through prompt injection and jailbreaking. Standards bodies including the United States National Institute of Standards and Technology have published taxonomies of adversarial threats to guide defenders.
| Attack | Stage | Goal | |--------|-------|------| | Evasion | Inference | Cause misclassification | | Poisoning | Training | Corrupt or backdoor the model | | Extraction | Inference | Steal the model | | Membership inference | Inference | Reveal training data |
Adversarial machine learning sits at the intersection of AI and cybersecurity, an area overseen in Malaysia by the National Cyber Security Agency (NACSA) and CyberSecurity Malaysia. As Malaysian banks, telecommunications operators, and government services adopt AI for fraud detection, identity verification, and threat monitoring, these very systems become targets, and understanding adversarial robustness is essential to securing them.
The financial sector is a particular concern. Banks regulated by Bank Negara Malaysia (BNM) increasingly rely on machine-learning models for transaction monitoring and anti-money-laundering screening; evasion attacks that disguise fraudulent activity as legitimate would directly undermine these controls. Similarly, facial-recognition and electronic know-your-customer systems used in digital banking and by the National Registration Department could be targeted by adversarial inputs.
Malaysia's growing connected-vehicle and smart-manufacturing sectors, including the electronics corridor in Penang and Kulim, depend on computer-vision models that are susceptible to adversarial perturbation. Critical infrastructure protected under the national cyber-security framework, including power and water utilities, faces analogous risks where machine learning is used for monitoring and control.
The Malaysia AI Governance and Ethics framework and related guidance emphasise the safety, security, and reliability of AI systems. Adversarial robustness is a practical requirement for meeting these principles. Universities such as Universiti Teknologi Malaysia and Universiti Kebangsaan Malaysia conduct research in AI security, and HRD Corp-supported cybersecurity training increasingly incorporates the AI dimension as adversarial threats become more prominent across Southeast Asia.