Hypothesis Testing in Presence of Adversaries

Authors: Jorge González-Ortega (ICMAT-UCM), David Ríos (ICMAT-CSIC), Fabrizio Ruggeri and Refik Soyer

Source: The American Statistician vol. 75, iss. 1

Date of publication: 2021



Hypothesis testing is one of the fundamental problems in statistical inference. Though subject to debate, it has been thoroughly studied from a decision theoretical perspective, both from the frequentist and Bayesian points of view. In recent years, there has been an increasing interest in issues related with hypothesis testing problems in which hostile adversaries perturb the data observed by a decision maker as a way to confound her about the relevant hypothesis so as to attain some objectives. These cover applications like online fraud or spam detection and fooling classifiers in applied domains like national and homeland security, cybersecurity and automated driving systems and forms part of the emergent field of adversarial machine learning.

In this article, using concepts from Adversarial Risk Analysis (ARA), Jorge González-Ortega, David Ríos Insua, Fabrizio Ruggeri and Refik Soyer provide an alternative novel approach to the Adversarial Hypothesis Testing (AHT) problem, formerly based typically on standard game-theoretic concepts. They consider an agent, called the defender (she), who needs to assess which of several hypotheses holds, based on observations from a source that might have been perturbed by another agent, which they designate attacker (he) and study the AHT problem from the defender’s perspective. In doing this, the defender formulates a Bayesian decision making problem but has to forecast the attacker’s decision; this is a non-standard forecasting problem as it entails strategic elements. They make such forecast by simulating from the attacker’s problem, taking into account the uncertainty over the attacker’s beliefs and preferences.

The article begins by introducing what they term the Adversarial Statistical Decision Theory problem, extending the standard Statistical Decision Theory formulation to consider an adversarial variation in which the attacker tries to modify the dataflow observed by the defender to confound her and, consequently, attain a profit. After that, they pose the AHT problem formally and provide a conceptual solution focusing on a binary point hypothesis testing, as well as illustrating it with a simple numerical example and presenting a game theoretic perspective for comparison purposes. Then, they describe in depth an application in relation with batch acceptance. This problem consists of deciding whether to accept a batch of items received over a period of time, some of which could be faulty, thus entailing potential security and/or performance problems. This type of issues arises in areas such as screening containers at international ports, filtering batches of electronic messages or admitting packages of perishable products or electronic components, among others.

Further applications may be found in the context of, for example, adversarial signal processing, such as in Electronic Warfare (EW) where pulse/signal environment is generally very complex with many different radars transmitting simultaneously. Time interval between two pulses emitted by a threat radar is defined as a Pulse Repetition Interval (PRI). PRI tracking is an important problem in naval EW applications as knowledge of the PRI is used to defend ships against radar-guided missiles. The signals received may be jammed by hostile radars and this results in missing pulses due to reduced sensitivity of the receiver.