Technology and Innovation in Solving Global Problems
The Possible Use of AI-Driven Systems in Contemporary Armed Conflicts – Benefits
Machines could take over tasks from soldiers because they lack negative psychophysical features such as fatigue, discouragement, emotions, fear, and the fear of losing life. This means they won't act out of panic, vengeance, or hatred, reducing the risk of excessive behavior in stressful situations. Furthermore, their consistent operational capacity ensures reliability in critical tasks.
AI-driven systems are more capable of carrying out 3D (dull, dirty, and dangerous) work due to their lack of emotions or physical exhaustion. They possess divisible/shared intelligence, can calculate and operate at digital speed, making them more capable of collecting and processing new information, and they do not forget orders. This superior processing speed can lead to quicker and more accurate decision-making in complex scenarios.
The Possible Use of AI-Driven Systems in Contemporary Armed Conflicts – Benefits
Machines may expand the battlefield by penetrating enemy lines and maintaining a presence in the operational theatre for longer durations than humans. Combat robots will not endanger civilian lives by exposing them to hostile fire, as there is no need for self-defense, and will minimize their own losses, contributing to the mitigation of the negative impacts of armed conflicts. Additionally, robots' precise targeting capabilities can also help reduce collateral damage.
The Possible Use of AI-Driven Systems in Contemporary Armed Conflicts – Concerns
Besides legal and ethical controversies, a question arises whether fully autonomous weapons systems (AWS), equipped with artificial intelligence and guided by an algorithm, will be able to make decisions in a way that accounts for numerous variables while ensuring compliance with the prohibitions and restrictions adopted for the implementation of a combat mission, since contemporary military operations take place in a complex legal, military, and political environment. Specifically, can these systems adapt to unforeseen circumstances and nuances in real-time?
Autonomous Weapon Systems (AWS) – Searching for a Path in the Maze of Definitions
Definitions of AWS:
U.S. Department of Defense (2012): An AWS is a weapon that, once activated, can select and engage targets without further intervention by a human operator, including human-supervised autonomous weapon systems that allow human operators to override the system but can select and engage targets without further human input after activation.
International Committee of the Red Cross (2016): An AWS may be any weapon system with autonomy in its critical functions—that is, a weapon system that can select (search for, detect, identify, track, or select) and attack (use force against, neutralize, damage, or destroy) targets without human intervention.
Autonomous Weapon Systems (AWS) – Searching for a Path in the Maze of Definitions
Definitions of AWS:
European Parliament (2018): Lethal Autonomous Weapon Systems (LAWS) refer to weapon systems without meaningful human control over the critical functions of selecting and attacking individual targets.
Newport ROE Handbook (2022): An autonomous weapon system is a weapon system that, once activated, can select and engage targets without further intervention by a human operator including human-supervised autonomous weapon systems that allow human operators to override the system but can select and engage targets without further human input after activation.
Human-supervised autonomous weapon system: An autonomous weapon system that is designed to provide human operators with the ability to intervene and terminate engagement.
Semi-autonomous weapon system: A weapon system that, once activated, is intended to only engage individual targets or specific target groups that have been selected by a human operator.
Autonomous Weapon Systems (AWS) – Searching for a Path in the Maze of Definitions
Definitions of LAWS:
Government of China (GGE, 2018): LAWS should include but not be limited to the following 5 basic characteristics:
Lethality: sufficient payload (charge) and means to be lethal.
Autonomy: the absence of human intervention and control during the entire process of executing a task.
Impossibility for termination: once started, there is no way to terminate the device.
Indiscriminate effect: the device will execute the task of killing and maiming regardless of conditions, scenarios, and targets.
Evolution: through interaction with the environment, the device can learn autonomously and expand its functions and capabilities in a way exceeding human expectations.
Autonomous, Automated, Automatic
Automated systems, like the American Phalanx or Russian Kashtan, independently locate and destroy incoming rockets or missiles. These systems are programmed with specific targets and responses but don't deviate from their programming. For example, the Phalanx system uses radar to identify and engage incoming threats, acting as a last line of defense for naval vessels.
Autonomous, Automated, Automatic
Unmanned systems can be grouped into three categories:
Remotely operated systems
Automated systems
Systems that operate autonomously
Future actions are foreseeable in automatic systems due to the inability to react differently to a given factor. In automated systems the ability to operate is enhanced but at the same time limited to the types of situations that have been pre-programmed in a limited way. In the case of autonomous systems, the effect of actions is difficult to predict since the system is given a high level of decision-making in a changing and complex environment. This unpredictability raises significant challenges for ensuring compliance with international law and ethical standards.
Autonomous, Automated, Automatic
Autonomous systems are "systems that can operate without direct human control or supervision in dynamic, unstructured, open environments based on feedback information from a variety of sensors."
An autonomous weapon system is understood as "being composed of disparate soft- and hardware elements that work together – including sensors, algorithmic targeting and decision-making mechanisms, and the weapon itself."
Autonomous, Automated, Automatic
Autonomy in weapon systems can be categorized according to three different traits:
The human-machine command-and-control relationship.
The sophistication of the machine’s decision-making process.
The types of decisions or functions being made autonomous.
Regarding the first trait, systems can be classified based on whether they receive inputs by a human operator to perform their functions in:
“Human-in-the-loop” of the targeting decision, referring to systems that select targets and deliver force upon human command. In these systems, a human reviews and approves each target selected by the AI.
“Human-on-the-loop,” capable of selecting targets and delivering force without human interaction but remaining under the oversight of humans so that humans retain the power to override the machine’s action. This allows for quicker responses while still maintaining a level of human supervision.
“Human-out-of-the-loop,” which refers to systems that select targets and deliver force autonomously without humans being able to intervene during the process. These systems raise the most significant ethical and legal concerns due to the lack of human oversight.
Autonomous, Automated, Automatic
The lack of “human control/human intervention” is a crucial element of the definitions of AWS. A fully autonomous system could be deployed without any established communication network and would independently respond to a changing environment and decide how to achieve its pre-programmed goals. However, this independence also introduces risks related to unintended consequences and the potential for violations of international law.
Autonomous, Automated, Automatic
“Autonomous functioning” refers to the ability of a system, platform, or software, to complete a task without human intervention, using behaviors resulting from the interaction of computer programming with the external environment.
Tasks or functions executed either by a platform, or distributed between a platform and other parts of the system, may be performed using a variety of behaviors, which may include reasoning and problem-solving, adaptation to unexpected situations, self-direction, and learning.
Artificial Intelligence and Algorithm
Artificial Intelligence (AI) is defined as "an intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans and other animals."
This term is applied particularly when a machine mimics cognitive functions that are associated with human minds, such as learning and problem-solving. AI systems use algorithms and data to perform tasks that typically require human intelligence.
Artificial Intelligence and Algorithm
EU Regulation 2024/1689 of the European Parliament and of the Council:
A key characteristic of AI systems is "their capability to infer."
This capability refers to the process of obtaining the outputs, such as predictions, content, recommendations, or decisions, which can influence physical and virtual environments, and to a capability of AI systems to derive models or algorithms, or both, from inputs or data. The accuracy and reliability of these inferences are critical for the safe and ethical deployment of AI.
The techniques that enable inference while building an AI system include machine learning approaches that learn from data how to achieve certain objectives and logic- and knowledge-based approaches that infer from encoded knowledge or symbolic representation of the task to be solved. Machine learning algorithms can identify patterns and make predictions based on large datasets.
The regulation also clarifies that AI systems are designed to "operate with varying levels of autonomy, meaning that they have some degree of independence of actions from human involvement and of capabilities to operate without human intervention. The adaptiveness that an AI system could exhibit after deployment, refers to self-learning capabilities, allowing the system to change while in use." This adaptability introduces both opportunities and challenges for oversight and control.
Artificial Intelligence and Algorithm
Main characteristics of AI systems:
Perception of the environment, including consideration of the complexity of the real world. This involves the use of sensors and data processing to understand the surrounding environment.
Information processing, encompassing the collection and interpretation of input data. This includes analyzing and filtering data to extract relevant information.
Action-taking, which involves task execution (including adaptation and response to environmental changes) while maintaining a certain level of autonomy. The actions taken should align with the set objectives.
Achievement of specific goals. The AI system should be designed to achieve specific, well-defined goals.
Artificial Intelligence and Algorithm
AI is typically grouped into the following three categories:
Artificial Narrow Intelligence (ANI) – This is an AI that can execute one particular decision type; ANI is also described as Weak AI; currently, all AI methods we use are examples of ANI. Examples include spam filters and recommendation systems.
Artificial General Intelligence (AGI) – This is an AI that can execute multiple functions and is comparable to human intellectual abilities; AGI is also known as Strong AI; no AI system has yet achieved general intelligence. AGI would be capable of performing any intellectual task that a human being can.
Artificial Super Intelligence (ASI) – AI that surpasses human intelligence; ASI is also categorized as Strong AI and also does not exist yet. ASI could potentially solve problems and make decisions far beyond the capabilities of human beings.
Artificial Intelligence and Algorithm
The distinction between Strong AI and Weak AI essentially boils down to the difference between being intelligent and acting intelligently.
Strong AI denotes a “mind” that is truly intelligent and self-aware. A strong AI could possess consciousness and subjective experience.
Weak AI refers to what is currently available, namely systems that exhibit intelligent behaviors despite being “ordinary” computers. These systems are designed to perform specific tasks but do not possess general intelligence or self-awareness.
Artificial Intelligence and Algorithm
An algorithm can be defined as a precisely specified computational procedure that, based on certain values provided (input), produces certain values (referred to as output).
Algorithms allow, among others, to make a “decision” or “choice” based on programmed parameters and specific inputs, and therefore replace human judgment or decision- making processes. However, the reliance on algorithms also introduces concerns about bias and fairness.
Typical AI algorithms that are data-driven “are inherently brittle, which means that such algorithms cannot generalize and can only consider the quantifiable variables identified early on in the design stages when the algorithms are originally coded." This lack of adaptability can lead to failures in unexpected situations.
Artificial Intelligence and Algorithm
Until the level at which ideally autonomous systems are created – i.e. those created by other autonomous systems – is achieved, the algorithm will always be the work of a human, and therefore its functioning will be subject to some extent to human control. Nonetheless, the complexity of advanced algorithms can make it difficult to fully understand and control their behavior.
Artificial Intelligence and Algorithm
An algorithm is a regulation for performing a specific task – it is a sequence of actions or a procedure leading to the completion of a specific task or problem solution in a finite time.
AWS functions through algorithms and data, yet it may not sufficiently address intricate and unpredictable scenarios in a battlefield context. The computer adheres to a defined procedure, processing extensive data to reach a particular goal. The relationships between the data can be so intricate that offering explanations may be notably challenging or even impossible. Once the algorithm is activated and begins making decisions, it becomes challenging to ascertain the rationales behind those decisions. This difficulty arises because the algorithm does not analyze causes to derive consequences; instead, it derives solutions by processing extensive data. Determining the motivation behind the outcome would necessitate an analytical task comparable to the one performed by the machine.
Algorithm and Targeting Checklist
Is the intended target of the attack a military objective? – If the answer is NO, the attack must be aborted. If the answer is YES, then proceed to:
Could the attack result in destruction and civilian casualties (collateral damage)? – If the answer is YES, then proceed to:
Is there an alternative military target that would achieve the same military advantage while posing a lower risk to civilians or civilian objects? – If the answer is NO, then proceed to:
Have all practically feasible precautions been taken to minimize destruction and civilian casualties (e.g., warnings, evacuation time, choice of means of warfare, direction of attack)? – If the answer is YES, then proceed to:
Would the anticipated destruction and civilian casualties be excessive in relation to the concrete and direct military advantage anticipated? – If the answer is YES, the attack must be aborted. If the answer is NO, then proceed to:
Are the proposed weapons, methods, and means of warfare prohibited under the given circumstances? – If the answer is YES, the attack must be aborted. If the answer is NO, the attack may be conducted, taking into account the prevailing situation and adhering to the outlined targeting procedure.
Algorithm and ROE Card
ROE Card (Soldier’s Card) – contains simplified summaries of the actual ROE and are intended to provide the personnel to which they are issued with clear and understandable instructions on the use of force relevant to their level of decision- making.
Rules of engagement often involve complex and context-dependent decisions. They may require assessing the intentions and actions of potential targets, and respecting LOAC principles (e.g., distinction, proportionality). These tasks are challenging for autonomous systems. Ensuring that autonomous systems can adhere to these principles is a major area of research and development.
Algorithm and ROE Card
The more "unpredictable" the nature of the environment, the greater the need for assessment and decision-making. In highly dynamic and uncertain environments, human judgment remains crucial for making ethical and legal decisions.
The complexity and scope of tasks that will be carried out by AI-driven systems will certainly be wider and wider, and these systems will have to perform not only tasks based on skills/abilities and rules, but also tasks requiring knowledge and experience. This evolution necessitates continuous adaptation and refinement of AI capabilities.
Relative strengths of computer vs human information processing
Computers excel at rules and skills, while humans excel at knowledge and expertise. Combining the strengths of both humans and computers can lead to more effective and ethical outcomes.
The First Restriction – The Environment
Since AI systems operate and learn based on the data they receive, data is central to both the development of the system and the determination of its operational behavior. The quality and relevance of the data used to train AI systems are critical for their performance.
Belligerents must ensure that the system is trained with appropriate data. If the system autonomously collects data and learns, it must be programmed to collect only relevant data and use it appropriately. Therefore, the system must be trained with a focus on the environment in which it will be deployed and under conditions representative of that environment. This includes accounting for cultural, social, and political factors.
The First Restriction – The Environment
Considering the global trend towards urbanization, it appears that future conflicts are likely to increasingly occur in densely populated battlefields, which are conducive to high levels of uncertainty. Urban environments present unique challenges for AI systems due to their complexity and unpredictability.
The uncertainty and dynamic nature of these environments require systems to make accurate predictions and timely actions. AI systems must be capable of adapting to changing conditions and making decisions in real-time.
Modern battlefields present highly adversarial multi-agent environments, necessitating engagement with cooperative, neutral, and competitive agents simultaneously, including allied agents, civilians, third parties, and enemy agents. This complexity requires AI systems to differentiate between various agents and act accordingly.
While air and maritime environments are relatively simple with few navigational obstacles, land navigation presents numerous challenges such as irregular terrain, vegetation, unstable surfaces, and a higher number of agents (both human and automated). Due to these difficulties, current autonomous capabilities are typically deployed only in “controlled” environments. However, even controlled environments can present unexpected challenges.
The Second Restriction – Tasks
It is possible to distinguish five different “task areas” into which automation can be implemented:
Mobility (platform movement and navigation),
Health (survival management, e.g. refueling or self- repair functions),
Interoperability (communication with other agents in the environment),
Intelligence (analysis of tactical or strategic battlefield data) and
Force (detection, identification and engagement of targets).
The Second Restriction – Tasks
The first three tasks (Mobility, Health and Interoperability) are often referred to as “operational functions” while the latter two (Intelligence and Force) are often referred to as “critical functions”. Critical functions carry the most significant ethical and legal implications.
It is frequently the critical task areas that are deemed more problematic. Since critical functions are those related to targeting, this implies that the tasks assigned to AWS are limited to combat operations where the use of force is essentially indispensable. This limitation helps mitigate the risk of unintended consequences.
If AWS are granted a certain degree of decision-making autonomy, which is inherent in the ROE, in situations characterized by poor situational awareness and significant battlefield dynamics – conditions that entail a high degree of uncertainty and unpredictability – there is a risk that AWS may fail, thereby potentially even leading to breaches of international humanitarian law. Ensuring that AWS adhere to IHL is a paramount concern.
The Second Restriction – Tasks
AWS could be applicable in situations of deliberate targeting operations that may commence weeks, months, or even years before an attack on a target occurs, allowing ample time to employ a strategic approach. These operations allow for careful planning and consideration of potential consequences.
The use of AWS should be avoided in dynamic targeting situations, where the level of uncertainty and unpredictability increases. Dynamic targeting requires quick decision-making, which can be challenging for AWS.
Although dynamic targeting operations follow similar decision-making steps, the timeline for deciding to attack a target is significantly shortened, allowing military forces to respond more swiftly and effectively to rapidly changing situations. Human judgment is often necessary in these scenarios.
Machine Learning – Opportunity or Challenge?
The future deployment of AWS will face limitations related to the environments in which they operate and the tasks/functions they are expected to perform. These limitations must be carefully considered to ensure the safe and ethical use of AWS.
The characteristics of current AI systems significantly impair our ability to predict the system’s outputs and understand the reasons behind them. The lack of transparency and explainability is a major challenge for the deployment of AI systems.
The unpredictability of AI systems is of a different nature and degree depending on the AI techniques employed, namely hand-coded programming or machine learning.
Hand-coded programming – It involves programmers developing a model of the world, including its logical rules, which are embedded into the system to enable autonomous operation. Since this model is created by humans, the system’s internal processes are understandable and transparent. By providing specific inputs, one can observe the system’s responses and trace the reasoning behind its outcomes. However, this handcrafted approach is limited to environments that can be reduced to clear mathematical rules. Given the unpredictable and constantly changing nature of armed conflicts, this approach is generally unsuitable for such scenarios. As a result, handcrafted AI is only applicable in highly controlled and predictable environments.
Machine Learning – Opportunity or Challenge?
Machine learning – This technique enables AI systems to improve their performance by expanding their knowledge. Machine learning algorithms can identify patterns and make predictions based on large datasets.
Machine learning is defined by a dual purpose:
Scientific (understanding and mechanically generating phenomena related to temporal changes and adapting reasoning) and
Practical (automatically acquiring foundational knowledge based on examples).
Machine learning can be defined as the improvement of outcomes through experience, which is closely related to generalization. The ability to generalize from past experiences is crucial for adapting to new situations.
Machine learning is a technique that allows algorithms to extract correlations from data with minimal supervision. However, the extraction of correlations does not necessarily imply causation, which can lead to flawed decision-making.
Machine Learning – Opportunity or Challenge?
Machine learning enables a system to learn how to perform tasks by analyzing large amounts of data, identifying correlations, and building a representation of the world. This approach allows AI systems to adapt to complex and dynamic environments.
Unlike traditional programming, it does not require explicit instructions; instead, developers must design a structure that allows the system to learn and adapt to changing conditions, providing it with extensive, well-selected data relevant to its operating environment. The design of the learning structure is critical for ensuring the system learns appropriate behaviors.
For instance, when deployed in armed conflict settings, a machine learning system might develop its own criteria for applying a set of rules based on observations made on the battlefield. Due to these technical characteristics, machine learning systems are particularly well-suited for operating in complex environments where hand-coded programming would be inadequate. However, the system's reliance on data also introduces potential biases.
Machine Learning – Opportunity or Challenge?
The primary issue with machine learning is its dependence on training data for learning. The quality and representativeness of the training data are crucial for the system's performance.
The system’s effectiveness relies on data that accurately represents the environment in which it will operate to prevent biases or significant flaws. Biases in the training data can lead to discriminatory or unethical outcomes.
In complex scenarios, the range of inputs is far greater than in handcrafted systems, potentially altering the system’s parameters and increasing uncertainty about what the system will learn and how it will respond to new inputs. Managing this uncertainty is a significant challenge.
Machine Learning – Opportunity or Challenge?
The “black box” (or opacity) problem – It refers to complex algorithms whose implementation and operation are opaque, meaning that the internal functioning of the method is difficult to comprehend. This lack of transparency makes it difficult to understand and trust the system's decisions.
Researchers are unable to fully explain the functioning or the outcomes of AI – Neither programmers nor users can fully understand how, and why, a given output was produced, which entails certain risks, such as actions that may not align with the original intentions of the AI system creators. This indicates that the system is unable to provide justifications for its estimations or decisions, lacking what is known as explainability. Explainability is crucial for accountability and trust.
Even in the case of deterministic AI systems, predictability is still limited. It is unrealistic to expect that these systems will encounter exactly the same inputs in real-world scenarios as they did during the pre-deployment assessment. Therefore, continuous monitoring and evaluation are necessary.
Machine Learning – Opportunity or Challenge?
At various levels of military command, preceding the final decision to deploy an AI system, steps should be taken to determine whether the involvement of AI would facilitate or hinder the achievement of operational objectives and whether this can be accomplished in compliance with IHL and ROE. A thorough risk assessment is essential before deploying AI systems.
It is also essential to anticipate the risks associated with the use of AI systems and to develop appropriate parameters and constraints to mitigate these risks and ensure adherence to the relevant norms and rules. This includes establishing clear lines of responsibility and accountability.
However, given the inherent unpredictability of complex and dynamic environments like armed conflicts, it is highly doubtful that developers and their organizations could certify that a system would respond safely or appropriately to any input or conditions it might encounter. The limitations of AI systems must be acknowledged and addressed.
Drone Swarms – Elements of Definition
A swarm is a group of individual systems that interact and operate as a collective with a common goal. The collective behavior of swarms can enable them to accomplish complex tasks.
Swarms are a novel type of weapon system in which there is great military interest because of their potential for enabling new types of missions. However, the use of swarms also raises new ethical and legal challenges.
However, because swarms operate as collectives, there are real limitations to the extent of control that humans can exert over them. The decentralized nature of swarms makes it difficult to predict and control their behavior.
The individual units of a swarm are unmanned systems, but a swarm can also contain manned systems (such as a swarm of one aircraft and many drones) or static sensors. The combination of different types of systems can enhance the capabilities of the swarm.
Nonetheless, the individual units, or nodes, are usually not very advanced. The utility of swarms derives from the fact that the whole is better than the sum of its parts. The emergent behavior of swarms can be greater than the sum of the individual components.
Drone Swarms – Elements of Definition
Through coordination and task distribution, swarms can accomplish complex missions, giving them three major benefits.
Swarms are:
a) scalable, as it is easy to change the size of the swarm depending on the mission; Scalability allows for adapting to different mission requirements.
b) adaptable, as they can be used for different types of missions; and Adaptability makes swarms versatile for various applications.
c) robust, because if a single node fails, other nodes can take over. Robustness ensures that the swarm can continue functioning even if individual nodes fail.
Drone Swarms – Elements of Definition
Swarms invariably need a human commander at the mission level, giving overarching guidance, but delegated with a wide range of tasks to be carried out autonomously. Human oversight is necessary to ensure that the swarm operates within ethical and legal boundaries.
For instance, an individual could task a swarm of missiles with a set of targets but allow the missiles to coordinate among themselves to choose the target to strike. This delegation of tasks allows the swarm to adapt to changing conditions.
Agents in the swarm must be homogeneous: They must have the same physical characteristics, the same programming, and the same sensors. Homogeneity simplifies the coordination and control of the swarm.
Sensors are important because the rules used to guide swarm behaviour are often based on environmental factors outside of the swarm. Environmental awareness is crucial for the swarm to make informed decisions.
Finally, the agents in an autonomous swarm must be able to communicate with each other. Communication enables the swarm to coordinate its actions and share information.
The current state of the art in military applications is limited to swarms ranging in size from tens to hundreds of unmanned aircraft.
Potential Benefits of the Military Use of Swarming Drones
The possibilities to use drone swarms in a military context:
a) lethal missions (combat operations); The use of lethal missions raises significant ethical and legal concerns.
b) non-lethal missions (e.g., they may be tasked to search a defined area to find wounded soldiers (combat SAR), or they may be used to map large areas). Non-lethal missions can provide valuable support without the use of force.
Swarming drones can be used in three ways by military forces:
a) to attack,
b) to defend, and
c) to provide support functions, such as intelligence, surveillance, and reconnaissance (ISR). Each of these uses presents different challenges and opportunities.
Swarming Drones as a Challenge for International Law
Article 36 of AP I of 1977 – International law requires legal reviews of new weapons and of new means and methods of warfare while they are under study or development. Critical analysis at an early stage of whether the weapons under development do not pose major strategic, ethical and legal risks is therefore essential. This requirement ensures that new weapons comply with international law.
Responsibility and control: First and foremost, swarms raise questions about the quality of human control over the use of weapons and their effects. The difficulty of predicting the behavior of swarms raises concerns about accountability.
Because there is no universal model for understanding what emergent behaviours will arise from simple rules, it is questionable whether a person in charge of a swarm is able to sufficiently predict its behaviour to make the required ethical and legal assessments and be responsible for it. The unpredictability of swarm behavior makes it difficult to assign responsibility.
Swarming Drones as a Challenge for International Law
When the number of elements in a swarm increases, human control must shift increasingly to the swarm as a whole, rather than micromanaging individual elements. This shift requires new approaches to human-machine interaction.
Swarms have two relational levels of autonomy:
a) between the human and the swarm, and This relationship requires careful consideration of the appropriate level of human control.
b) between the nodes and the swarm as a whole. The interactions between nodes within the swarm also raise questions about autonomy and control.
Even if humans decide which targets to select and engage, the nodes could have a certain level of freedom in deciding how to execute this. This delegation of decision-making raises concerns about compliance with IHL.
Ultimately, this suggests that a “level of autonomy” approach would not be the ideal solution for dealing with swarms. A more holistic approach that considers the entire system is needed.
Swarming Drones as a Challenge for International Law
Providing the programming of all the algorithms correctly equates for distinction, proportionality, duty to take precautions etc., the swarm may still be IHL compliant. However, ensuring compliance with IHL requires careful attention to the design and programming of the swarm.
The reality is that such autonomy does, as with all autonomous systems raise the question as to whether the swarm could distinguish its targets correctly, balance the loss of civilian life against the direct and concrete military advantage and take the necessary precautions without entirely without the presence of a reasonable military commander. The absence of human intervention raises concerns about the ability of the swarm to make ethical and legal decisions.
It is less about whether the swarm does or does not violate IHL and more about how the operations fit within the existing framework. Integrating swarms into existing legal and ethical frameworks is a major challenge.
Clearly, if the swarm is incapable of fulfilling the distinction requirement, its use is inherently unlawful. The ability to distinguish between combatants and non-combatants is a fundamental principle of IHL.
Swarming Drones as a Challenge for International Law
The limited opportunities to alter the behaviour of a swarm after launch, and the reliance on indirect control methods, can lead to adverse incidents – crashes, fratricides (so-called friendly fire) and even civilian casualties. The lack of direct control raises the risk of unintended consequences.
In the context of an armed conflict this lack of predictability poses a challenge to the protection of civilians against dangers arising from military operations. Protecting civilians is a paramount concern in armed conflict.
During swarm engagements, it is likely that some nodes will be shot down and left on the battlefield. The presence of unexploded ordnance poses a threat to civilians after the conflict.
If they contain unexploded ordnance, this could kill civilians. In other words, swarms of explosive micro-drones could pose a post- conflict risk to civilians when they fail to explode as intended and become explosive remnants of war (ERW). The presence of ERW can have long-lasting consequences for civilian populations.
Swarming Drones as a Challenge for International Law
Slaughterbots – small, expendable, explosive weapons deployed in swarms to attack individual people – elaborated by opponents of autonomous weapons, call into question the long-standing legal protection of combatants against exploding projectiles and assumptions about what constitutes superfluous injury or unnecessary suffering. The use of such weapons raises serious ethical concerns.
St. Petersburg Declaration Renouncing the Use, in Time of War, of Explosive Projectiles Under 400 Grammes Weight of 1868: “The Contracting Parties engage mutually to renounce, in case of war among themselves, the employment by their military or naval troops of any projectile of a weight below 400 grammes, which is either explosive or incendiary, on the grounds that such weapons cause unnecessary suffering and are morally unacceptable in armed conflict.