AI-Supported Targeting: Precision or Civilian Casualties?

AI-supported targeting: enhancing precision or increasing civilian casualties?

Despite claims of enhanced precision, the deployment of AI-Decision Support Systems (AI-DSS)in modern battlefields, including their use in target identification, selection, and engagement, risks exacerbating civilian suffering. This stems from the fundamental limitations of AI and the potential for catastrophic misapplication.

AI-DSS rely on predictions based on pattern recognition and classification, generalizing from training data to recommend targets. This inherent reliance on past data means that the system's ability to identify targets depends on its capacity to find similarities between available information about a given population and the data it was trained with. Research by MIT highlights the inherent limitations of this approach.

The problem is compounded by the inherent imperfections of AI. These systems are never perfect and are prone to "failure," particularly in the complex and unpredictable environment of a real-life battlefield. Even with meticulous development, the countless variables involved in the “fog of war” are impossible to perfectly pre-program. For example, an AI-DSS trained to identify collaborators of enemy commanders through various means (social media, communication intercepts, etc.)could misclassify individuals as targets based on perceived, yet tenuous, linkages with the enemy. Innocent civilians could be targeted due to shared school attendance or casual acquaintances. Furthermore, AI might even “invent” nonexistent patterns, leading to the targeting of innocent civilians. Studies on autonomous weapons emphasize the ethical ramifications of these inaccuracies.

While human operators retain the ultimate authority to use force, AI-DSS recommendations significantly influence their decision-making. Studies show that military personnel often prioritize action over inaction in time-sensitive situations, leading to "automation bias." This bias inclines operators toward accepting AI recommendations without sufficient verification, reducing meaningful human control. The speed and scalability of AI-powered targeting enhance this risk, increasing the potential for mass targeting and minimizing the opportunity for careful assessment.

The devastating potential impact on civilian lives is substantial. Research suggests that AI's use in targeting decisions could lead to far greater casualties, particularly in urban environments. The claim of “collateral damage” often serves as a justification for civilian casualties; military forces may claim to have believed in their system, but the question of accountability remains a significant challenge. Human Rights Watch reports highlight the absence of accountability in such situations.

Even if a system boasts a seemingly low error rate, this figure can be deceptive. The dynamic nature of data flows in AI systems means that any reported success or failure rate constantly fluctuates. Furthermore, the legal review mechanisms envisioned in Article 36 of Additional Protocol I may not be sufficient to prevent violations of IHL. The lack of transparency regarding the specifics of the AI systems used by armed forces hinders monitoring and oversight. Even with a low purported margin of error, the sheer volume of targets identified by AI systems still places countless civilian lives at risk. Can we accept such casualties as the result of a simple "glitch" or "hallucination"?