1. Introduction

In September 2020, Pope Francis addressed the 75th meeting of the United Nations General Assembly and said, ‘new forms of military technology, such as lethal autonomous weapons systems … irreversibly alter the nature of warfare, detaching it further from human agency’.¹ This is a recent example in a long line of statements from heads of state, international organisations, non-governmental organisations, pressure groups and others commenting on the significance of ‘autonomous weapons’ which, according to the International Committee of the Red Cross (ICRC), can select and attack targets without human intervention after deployment.² Indeed, the ICRC has suggested that the deployment of such weapons would represent a ‘paradigm shift’ in warfare by removing humans from combat and delegating it to machines.³ In this sense, combat involving autonomous technology is more than merely an incremental move along the spectrum of remote warfare—a phenomenon represented most obviously by ‘drones’, which remain under the direct control of humans, albeit in distant bunkers.⁴ Rather, it represents an entirely new genus of warfare and one that must be understood, analysed and planned for before it enters onto the world stage.

Despite the significance of the issue, and the need to plan for the advent of autonomous weapons, as McFarland notes, ‘there is no conventional [ie treaty-based] or customary international law specifically directed’ at autonomous weapons.⁵ The consequence of the lack of bespoke law for autonomous weapons is that, as matters stand, their operation will be governed by existing international law—including international humanitarian law (IHL). What this means in practice is far from clear and is subject to much debate. Nonetheless, the notion of applying existing IHL—most of which emanates from the 19th and 20th centuries and which assumes determinations made on the battlefield will remain human made—to conflicts involving self-directed robots is fascinating. This article will explore the issue by providing an exposition of the existing debate and then analysing the legality of autonomous weapons specifically through the lens of the principles of IHL. The overarching argument of the piece is that the deployment of autonomous weapons would not yet be legal owing to technological limitations that will be clearly articulated. However, the piece also demonstrates that the direction of travel for overcoming those limitations has already been set and explains why autonomous weapons may be legally deployable sooner than many critics would care to admit. Indeed, it will be demonstrated that autonomous weapons have the potential to comply more closely with the principles of IHL than humans. It should be noted that this article does not seek to address the compatibility of autonomous weapons with other aspects of international law (such as international human rights law) or the morality of such weapons. These considerations undoubtedly have an important part to play in the wider autonomous weapons debate; however, they are not the focus of this particular piece.

2. The autonomous weapons debate

Much has already been written about the compatibility of autonomous weapons with IHL and international law more generally. Broadly, there are two positions on the subject. The first is that the deployment of autonomous weapons would be unlawful because such machines would be unable to comply with one or more aspects of IHL. The second is that we should reserve judgement on the matter because it may turn out that autonomous weapons are able to abide by humanitarian law adequately or, indeed, better than humans in some scenarios. It is helpful to provide an overview of those polarised positions to set the scene.

A. Critics of autonomous weapons

There is an abundance of material from international and non-governmental organisations to support the position that the deployment of autonomous weapons would be illegal. Human Rights Watch (HRW) has been one of the most vocal critics of the development of autonomous weapons and has taken the view that they ‘should be banned and [that] governments should urgently pursue that end’ (cf. the point below that States are investing billions of dollars into this technology).⁶ For example, HRW has argued that autonomous weapons ‘would not have the ability to sense or interpret the difference between soldiers and civilians’.⁷ This is, in part, because individuals’ intentions are often relevant to their status (eg in the context of civilians who ‘directly participate in hostilities’⁸) whereas ‘fully autonomous weapons would not possess [the] human qualities necessary to assess those intentions’.⁹ This deficiency in understanding of contextual factors would, HRW argues, render autonomous weapons unable to comply with the principle of distinction. Similarly, when it comes to proportionality, HRW highlights that implementation of this principle demands more than a mere balancing of quantitative data and that ‘a robot could not be programmed to duplicate the psychological processes in human judgment that are necessary to assess proportionality’ (cf the arguments raised by Schmitt, Boothby, Ekelhof and others below to the effect that autonomous weapons may be able to leverage sophisticated programming and processing power to number-crunch their way through these complex issues).¹⁰ Another non-governmental organisation, ‘Article 36’ (which campaigns to prevent unnecessary harm caused by new weapons), stated that automatic target recognition was already blurring the line between the responsibilities of humans and machines and that ‘any military attack should be through deliberate human reasoning’ (cf the point below made by Boothby and, separately, Thurnher to the effect that human reasoning can be warped by emotions such as fear, anger and vengeance).¹¹ The late United Nations Special Rapporteur on extrajudicial killings, Christof Heyns, expressed a range of concerns, including the apparent accountability deficit in the event that an autonomous weapon was to violate IHL. He observed that, ‘without the promise of accountability, deterrence and prevention are reduced, resulting in lower protection of civilians and potential victims of war crimes’.¹² Of course, in response to that point, one may argue that the various international criminal courts have a pretty dismal record for holding perpetrators to account even in the absence of autonomous weapons. One may also argue that technological solutions could be found to allow crimes committed via autonomous weapons to be traced back to their operators, commanders and programmers. For example, regarding programmers, the author has recommended that a list of contributors to an autonomous weapon’s code—similar to the credits that come at the end of a movie—could be created during development. This list could be coupled with a ‘black box’ style recorder that operates during deployment in order to improve the viability of attributing wrongfulness in cases where things go wrong.¹³ If this could be achieved, the deployment of autonomous weapons need not necessarily lead to an accountability gap.

Many academics also hold the view that autonomous weapons would be incompatible with IHL. For Ulgen, the use of autonomous weapons in armed conflict would be an affront to the concept of humanity, as the decision to take a life would be delegated to a machine, thus undermining the value of human existence (cf the argument made by Boothby below that autonomous weapons may be able to implement IHL more rigorously than humans thus making autonomous warfare more respectful of human existence than conventional warfare).¹⁴ Sharkey does not believe that autonomous weapons would be able to comply with distinction because ‘humans understand one another in a way that machines cannot … cues can be very subtle, and there are an infinite number of circumstances where lethal force is inappropriate’.¹⁵ Similarly, he argues that, after an attack has begun, the number of courses of action that open up becomes ‘vast and could cause chaotic robot behavior with deadly consequences’ (cf the point below that emotions, or the ‘fog of war’, can render humans unpredictable on the battlefield).¹⁶ For McFarland, the problem is that the deployment of autonomous weapons would result in the replacement of ‘on the spot’ human-made decisions with ‘broader policy-like choices’ which might be less well tailored to specific scenarios.¹⁷ Finally, Amoroso and Giordano oppose the use of autonomous weapons because, like Heyns, they do not believe that international criminal law is robust enough to attribute individual criminal responsibility where the use of autonomous weapons leads to violations of international law.¹⁸

B. Proponents of autonomous weapons

Inevitably though, as has been alluded to already, there is another side to the coin. Perhaps most obviously, many powerful States are heavily invested in the development of autonomous weapons with the effect that the HRW-led call for a ban seems doomed to fail. These States include China, Japan, India, Israel, Russia, South Korea, the UK and the USA.¹⁹ According to Allen, China’s leadership believes that being at the forefront in artificial intelligence is critical to the future of global military power.²⁰ Indeed, China has spent an estimated $4.5 billion on drone technology, including autonomous drones, according to the Stockholm International Peace Research Institute.²¹ According to Bendett, Russia has established a defence research organisation to investigate autonomy and robotics called the ‘Foundation for Advanced Studies’ and has also inaugurated an annual conference on the ‘Robotisation of the Armed Forces of the Russian Federation’.²² Furthermore, the Chairman of the Federation Council’s Defence and Security Committee observed that autonomous systems will soon be able to ‘replace a soldier on the battlefield [or] a pilot in an aircraft cockpit’.²³ Indeed, in 2017, Russia was already working on artificial intelligence-guided missiles that could decide to switch targets mid-flight without any remote human ‘pilot’ being involved.²⁴ In the USA, weaponry with integrated autonomy has been a principal component of the national security strategy since 2012²⁵ and, more recently, Gettinger estimated that spending in 2019 for ‘autonomy, teaming [and] swarms’ was $866 million.²⁶ Finally, in the UK, according to the Foreign, Commonwealth and Development Office, ‘at present [the UK does] not see the need for a prohibition on the use of [autonomous weapons] as international humanitarian law already provides sufficient regulation for this area’.²⁷ More tangibly, the aerial combat vehicle, ‘Taranis’, is being developed by the British aerospace manufacturer BAE Systems (in liaison with the Royal Air Force).²⁸ Taken collectively, Baker argues that these positions demonstrate that the ‘major powers appear reluctant to impose any limits on these systems beyond what international law already prescribes’.²⁹ If this is correct, ‘at some point in the future, [autonomous weapons] will find their way into the battlespace’.³⁰ The advocates for an autonomous weapons ban seem to be shouting into the wind.

Moreover, a number of academics have supported, or at least remained open to, the legality of autonomous weapons. Schmitt takes the view that existing IHL would be perfectly adequate to govern these machines because ‘each of [its] obligations applies fully to the use of autonomous weapon systems’.³¹ On matters such as their compliance with the principle of distinction, he observed that IHL would require ‘full use of onboard … sensors that could boost the reliability of target identification’.³² Likewise, Boothby takes the view that autonomous weapons ‘with further development may meet or even exceed [the applicable] legal criteria’.³³ In particular, he notes that advances in artificial intelligence may ‘enable an autonomous system to react appropriately, for example, to the absence, or presence of, particular kinds of target at a specific location scheduled for an attack by refraining from the planned attack’ (the matter of artificial intelligence is considered more fully below).³⁴ He also highlights that ‘robotic technologies will not be distorted by fear, anger, vengeance, amnesia, tiredness or other peculiarly human fallibilities’.³⁵ Anderson and Waxman similarly argue that autonomous weapons might prove to be more precise and less harmful than existing approaches to warfare, with the effect that their deployment would likely prove to be a positive development for IHL-compliance (the issue of machine targeting is also considered more fully below).³⁶ Thurnher grounds his support for autonomous weapons in the fact that they would not be hindered by human frailties stating that, ‘whereas human judgement is often clouded by anger, revenge, fear, and fatigue, autonomous machines may offer the promise of removing those worries from the battlefields of the future’.³⁷ He argues that, assuming solutions can be found for the remaining technological difficulties, the ‘use of autonomous weapons should not be deemed unlawful’.³⁸ Finally, Ekelhof argues that ‘States cannot solve the multiple challenges [of modern warfare] simply by throwing more people at the problem’ and that, when it comes to implementing distinction, ‘autonomous technology could speed up the process, processing large amounts of data so as to discern interrelationships and dependencies that human beings would fail to recognize’.³⁹

In short, there is a wide spread of views in academia and beyond on whether autonomous weapons have the capacity to comply with IHL. For every criticism, there is a response. For every response, there is a counter-response. The fact that the technology is still emerging—and is thus unknown and untested—has precluded anything resembling a conclusive resolution to the debate.

C. The United Nations regulatory debate

The multiplicity of views that have been expressed on the subject of autonomous weapons have been echoed within the halls of the United Nations in the context of negotiations held under the auspices of the Convention on Conventional Weapons (CCW). At the 2016 meeting of the CCW, it was decided that a Group of Governmental Experts (GGE) should be formed to ‘explore and agree on possible recommendations on options related to emerging technologies in the area of [autonomous weapons], in the context of the objectives and purposes of the Convention, taking into account all proposals—past, present and future’.⁴⁰ However, the progress of the GGE has been limited and, far from moving towards a ban or heavy regulation, the latest sessions have instead been characterised by a permissive tone grounded in the application of existing IHL.

The GGE 2019 session is a case in point. It placed more emphasis on the ways in which autonomous weapons could be made compatible with the existing legal regime than on creating a new treaty to ban or heavily restrict the phenomenon. This tack was encapsulated in the addition of a new guiding principle to the effect that ‘human-machine interaction, which may take various forms and be implemented at various stages of the life cycle of a weapon, should ensure that the potential use of … autonomous weapons … is in compliance with applicable international law [emphasis added]’.⁴¹ Again, the focus here was on existing IHL rather than any potential future IHL. More evidence of a permissive approach grounded in existing IHL came in the form of the statement that autonomous weapons ‘may be useful for enhancing the implementation of IHL … given the potential for emerging technologies to reduce human error and to increase precision in attacks’.⁴² Although the language in the full text is somewhat nuanced, this statement could be read as suggesting that autonomous weapons, outfitted with the appropriate technologies, may be better than humans at implementing the law in some instances. If correct, this is a significant point—hinting that the key to better IHL implementation going forward is more technology, not more regulation. Ultimately, the GGE 2019 session led to no decision on the form or content of any future regulation (such as a legally binding treaty or a political declaration).⁴³

The GGE 2020 session may have been expected to witness some further incremental progress on the issue. Of course, 2020 was not a normal year and no decisions were taken; except to postpone further discussions until 2021.⁴⁴ When it did finally occur, the GGE 2021 session (held in December 2021) resulted in an agreement to do little more than extend the GGE’s mandate to consider the issue and schedule a further 10 discussion days in 2022.⁴⁵ The outcome of the GGE 2021 session, or lack thereof, prompted a backlash from activist groups. The Campaign to Stop Killer Robots observed that ‘it is clear that a minority of States including the US and Russia … are committed to using the consensus rule in the CCW to … block progress towards the international legal response that is urgently needed’.⁴⁶ The pandemic notwithstanding, it seems that after years of deliberations the GGE could indeed have made more progress on the issue of autonomous weapons if certain key States were genuinely minded to do so. However, it seems they are not so minded and that the GGE’s contribution to the autonomous weapons debate is therefore likely to be diluted.

3. Considering autonomous weapons through the lens of the principles of IHL

In the absence of a settled and definitive regulatory position, the matter of the compatibility of autonomous weapons with IHL has been left—perhaps deliberately—in limbo. Yet, while the regulatory debate stalls, technological advances in the autonomous weaponry field continue to be made and States push closer and closer to the use of offensive autonomous weapons in combat. As Haner and Garcia put it, while ‘United Nations member states have made little progress in discussions of [autonomous weapons] the nature of warfare is transforming before our eyes’.⁴⁷ Indeed, in June 2021, Reuters reported that ‘the first autonomous drone attacks in history were taking place on a largely unwatched battlefield in Libya’ with the Turkish military as the operator.⁴⁸ While those reports are as yet unconfirmed, they nevertheless stress the urgency of this issue. We no longer have time to wait for a ban, for a new protocol to the CCW or for any other ‘new’ regulation. We must determine whether the deployment of autonomous weapons would be compatible with IHL as it currently stands. It is submitted that the best way to do this is to strip back the layers of the law and to analyse the issue by considering the compatibility of autonomous weapons with the principles of IHL. After all, the principles go to the heart of the regime and can act as a familiar and reliable guide to an emerging and complex subject.

The principles of IHL are not fixed in any one definitive list and so they must be identified. In Nuclear Weapons, the International Court of Justice (ICJ) enumerated its four ‘cardinal principles’ of IHL as distinction,⁴⁹ military necessity,⁵⁰ the prohibition on unnecessary suffering⁵¹ and proportionality.⁵² The UK Ministry of Defence has stated that ‘four fundamental principles still underlie the law of armed conflict [namely] military necessity, humanity, distinction, and proportionality’.⁵³ Denmark’s Ministry of Defence offers the same list.⁵⁴ New Zealand, through its Defence Force, supplies this quartet but with the addition of ‘non-discrimination’.⁵⁵ Academics offer their own nuanced variations on the theme. For example, Droege opines that ‘the core rules of IHL on the conduct of hostilities [are] the principles of distinction, proportionality and precaution,’⁵⁶ while Kolb argues that principles such as ‘distinction, precaution and proportionality’ are essential.⁵⁷ For present purposes, these lists have been synthesised into the following provisional shortlist: (i) humanity; (ii) military necessity; (iii) distinction; (iv) proportionality and (v) precaution. The extent to which these concepts are indeed ‘principles’ and whether autonomous weapons might comply with each of them will now be considered in turn.

A. Humanity and military necessity

We turn first to humanity and military necessity. Humanity is perhaps best encapsulated in Additional Protocol I to the Geneva Conventions which provides that, at all times, ‘civilians and combatants remain under the protection and authority of the … principles of humanity’.⁵⁸ Military necessity is captured in the Hague Regulations 1907 which provide that it is forbidden ‘to destroy or seize the enemy’s property, unless such destruction or seizure be imperatively demanded by the necessities of war’.⁵⁹ Given these pronouncements, and the myriad others in the same vein, it is clear that humanity and military necessity have a key role to play in IHL.⁶⁰ However, in the view of the author, there is a fundamental misapprehension surrounding the specific nature of these concepts in the regime. More precisely, it is submitted that they do not actually possess the key features of legal ‘principles’.

Dworkin undertook ground-breaking work on ‘principles’ and arrived at two key observations. Dworkin’s first observation was that principles must carry specific, and different, weightings so that one may triumph over the others in the event of a clash. More precisely, he noted that ‘when principles intersect [the resolution depends on] the relative weight of each’ principle.⁶¹ However, humanity and military are precluded from carrying different weightings. This is because IHL relies on a strict balance being maintained between humanity and military necessity in order to operate properly. As Dinstein put it, IHL is ‘predicated on a subtle equilibrium between the two diametrically opposed stimulants of military necessity and humanitarian considerations’.⁶² Dworkin’s second observation was that principles must have the capacity to supersede positive rules so that coherence in the regime over which they preside is maintained.⁶³ As Raz summarised, ‘all rules are subject to all principles and may be overridden by any of them in particular circumstances’.⁶⁴ However, this does not work in the context of IHL and neither humanity nor military necessity can be bestowed with the capacity to supersede positive rules. On military necessity, for example, Additional Protocol I prohibits combatants from harvesting organs from captives.⁶⁵ This rule could not simply be ‘trumped’ by military necessity if a commander needed a heart transplant and the only viable donor was one of his prisoners.⁶⁶ Indeed, if the positive rules of IHL were subject to military necessity in this manner, the protections offered by the regime would be badly undermined. The same would be true, in the reverse, of humanity. If humanity was to be applied as a principle, it could undermine the legitimate freedoms afforded to combatants to wage war. Thus, neither the ‘weightings’ nor ‘supersession’ criteria are satisfied by humanity or military necessity. Consequently, they are not principles of IHL.⁶⁷ It is likely that considerations such as these—even if not expressly articulated—explain the choice of the ICJ not to include ‘humanity’ in its list of ‘cardinal principles’ and the choice of academics such as Droege and Kolb not to include either ‘humanity’ or ‘military necessity’ in their respective lists.

Instead, rather than being ‘principles’ themselves, humanity and military necessity provide the counterbalanced framework that underpins IHL. The effect is that they should be considered—as the author has explained previously and in more detail in this journal—as ‘pillars’ of IHL.⁶⁸ These pillars manifest in the ‘true’ principles of distinction, proportionality and precaution (discussed below) and the phalanx of IHL rules. Their DNA is woven into those principles and rules. When it comes to autonomous weapons, the practical significance of this finding is that States need not allow for the notions of humanity or military necessity as separate, free-standing, considerations from the rest of IHL. Instead, provided they observe the true principles and rules of IHL, these considerations will be accounted for automatically. To take the example of distinction, civilians may not be targeted (because it would be inhumane) yet combatants may be targeted (because it is militarily necessary). In other words, humanity and military necessity are inherent in distinction and it is in this indirect manner that they come to be realised. The finding that humanity and military necessity do not require their own discrete implementation supports the broader argument of this article, namely that autonomous weapons may be able to comply with IHL more readily than some critics believe. This is because it obviates the need to distil these vague concepts into precisely articulated algorithms—a task that has hitherto been viewed as one of the main stumbling blocks to the development and deployment of autonomous weapons.

B. Distinction

The next question is whether autonomous weapons could comply with the principle of distinction in IHL. This principle is captured in the Additional Protocol I rule that one must ‘distinguish between the civilian population and combatants … and accordingly … direct … operations only against military objectives’.⁶⁹ Of course, hitherto, humans have been responsible for distinguishing friend from foe on the battlefield. They decide, for example, if a person is an enemy soldier based on the clothing they wear and the equipment they are carrying or whether an individual has been rendered hors de combat due to surrender.⁷⁰ Distinction is a crucial battlefield principle because, as Green summarised, combatants are a ‘legitimate object of attack [only for] … as long as they are capable of fighting, willing to fight or resist capture’.⁷¹ Humans implement distinction by considering a range of factors, such as the raising of hands in the air (to indicate surrender) or the collapsed or disorientated appearance of a foe (indicating incapacitation).⁷² Of course, were autonomous weapons to be deployed, machines would implement distinction. To do this, as the author has explained previously, machines would need to possess advanced skills in observation and recognition as well as sophisticated judgement-making ability.⁷³ Only once all three skillsets are mastered could robust distinction assessments be made.

Firstly, regarding the observation abilities of machines, one need look no further than the recent efforts of US defence manufacturer Raytheon. Raytheon, jointly with Exyn Technologies, has developed ‘mapping autonomous drones’ that can fully perceive their surrounding environment.⁷⁴ Exyn’s drones are essentially mechanised surveyors that can be flown into dangerous or awkward environments such as mines or derelict buildings to map those locations and instantaneously produce three-dimensional virtual models. Those models can then be scrutinised by surveyors, rescue personnel, mine managers and so on to help them in their work. According to Raytheon, these drones can ‘map dense urban environments in 3-D [and] dig deep to reveal tunnels, urban undergrounds and natural cave networks’.⁷⁵ They complete these feats using a combination of sensors including cameras, radar and lidar (which is similar to radar but uses infrared light). The company explains that the system collects 300,000 data points per second in order to map its environment and that it is sensitive enough to detect even dangling wires.⁷⁶ Thus, robots can ‘observe’ more, and with greater speed, than humans.⁷⁷ Secondly, regarding the recognition abilities of machines, they have also proven to be superior to humans. We can see this, for example, in the automated security guard context, where Patriot One’s ‘PatScan’ system can ‘recognise’ the inputs it receives and flag them as dangerous if appropriate. The system is essentially a collection of cameras, magnetic sensors, particle detectors and other scanning apparatuses that form a unit built into a doorframe, turnstile or other entryway to determine if anyone passing through poses a threat. According to Patriot One, the system can identify ‘when a weapon is present, whether overt or concealed’.⁷⁸ The system is even able to identify ‘specific weapon types’ with 94.7% accuracy.⁷⁹ Patriot One claims that PatScan is ‘faster, more accurate and more effective’ than human security guards.⁸⁰ This technology is currently used in commercial contexts but it is easy to imagine it being converted to military applications including, for present purposes, target acquisition by autonomous weapons. Thus, there is little doubt that machines also possess recognition capabilities at least on par with humans.

However, in spite of the successes in machine observation and recognition, machine judgement still appears to be lagging behind that of humans. Judgement is critical in the context of distinction in IHL because people can move between being targetable combatants and non-targetable civilians, or vice versa, without any change in their physical appearance. For example, a person might be dressed in army fatigues and carrying a machine gun in the middle of a battlefield. From observation and recognition alone, that person would no doubt be classified as a ‘combatant’. However, the person may not be targeted if they have become hors de combat due to capture, surrender or incapacitation.⁸¹ These are contextual considerations that humans are usually able to interpret quite well but that machines struggle with due to their highly nuanced nature. The difficulties in getting machines to appreciate contextual shifts continue to hamper advancement in the related context of automated agricultural processes. For example, the Danish firm ‘Agrointelli’ is developing a system called ‘RoboWeedMaPS’. This system operates as a sophisticated boom sprayer attached to a tractor that patrols crops in order to spray weeds with pesticides while leaving harvestable produce undisturbed.⁸² As such, the system operates on the same basis as an autonomous weapon in the sense that it must separate the ‘targetable’ from the ‘untargetable’. However, even in this simplified, low stakes, environment, contextual considerations continue to vex artificial intelligence. As a senior researcher on the project, Jorgensen, explained: ‘it only takes a small beetle to eat a leaf and the plant doesn’t look like the one in the image at all [or] the stems can be so thin that … it looks as though the leaves aren’t connected [or] if it’s cold in spring, some weeds turn completely purple even though they’re normally green’.⁸³ The result of the difficulties posed by contextual factors is that the system operates with a margin of error. This may be acceptable in the context of food production but it would not be tolerable in a battlespace environment where human lives are at stake. Currently then, autonomous weapons are not able to comply with the principle of distinction.

In the longer term of course, that may change. It is difficult to say for sure if or when artificial intelligence might reach a sufficiently high level to be able to make context-sensitive judgements. In an attempt to forecast how machine intellect may develop, Müller and Bostrom surveyed hundreds of artificial intelligence experts asking, ‘by what year would you see a (10 per cent/50 per cent/90 per cent) probability for … high level machine intelligence to exist?’. The median response for 10% probability was 2022, the median response for 50% probability was 2040 and the median response for 90% probability was 2075.⁸⁴ Walsh (a respected expert in this field) believes that machines will have achieved ‘human level’ cognition capability by 2062.⁸⁵ In essence then, it seems likely that sometime between 2040 and 2062 we will have artificial intelligence that is sufficiently advanced to deal with the full gamut of distinction assessments. If true, that would dispel the HRW argument above that autonomous weapons ‘would not have the ability to sense or interpret the difference between soldiers and civilians’.⁸⁶ In short, autonomous weapons could well be capable of complying with the principle of distinction in a couple of decades. It is a speculative point, but the general trend of rapid technological advancement over at least the last century—and the consensus of expert opinion—seems to suggest that it is a matter of when, not if, the requisite level of artificial intelligence will be reached.

C. Proportionality

In addition to being able to distinguish civilian from combatant, an autonomous weapon would need to be able to comply with the principle of proportionality in IHL. This manifests in the Additional Protocol I rule that prohibits attacks that would cause collateral damage ‘excessive in relation to the concrete and direct military advantage anticipated’.⁸⁷ As noted above, HRW is of the view that implementation of this principle demands more than a mere balancing of quantitative data and that ‘a robot could not be programmed to duplicate the psychological processes in human judgment that are necessary to assess proportionality’.⁸⁸ It is submitted that HRW is incorrect here and that proportionality does involve a quantitative calculation; albeit a very complex one. More specifically, proportionality in this context is simply a permutation of utilitarianism which, according to Bentham, is satisfied by any action ‘when the tendency it has to augment the happiness of the community is greater than any it has to diminish it’.⁸⁹ The parallel between utilitarianism and proportionality is apparent. Just as utilitarianism is satisfied when a given action will result in more ‘happiness’ than ‘unhappiness’; proportionality is satisfied when an attack will result in more military gain than collateral harm. Thus, proportionality in IHL, rightly or wrongly, is a principle in the utilitarian vein.⁹⁰

Consequently, the issue becomes whether an autonomous weapon might be able to implement proportionality by measuring military gain and collateral damage against each other. On this matter, the so-called ‘collateral damage estimation methodologies’ (CDEMs) present a starting point. CDEMs are complex processes. To take the example of the US CDEM, a five-stage analytical framework is employed to assess collateral damage based on considerations such as the area of effect of different weapon types, the demographics in the intended attack area and the impact of timing on the likely level of civilian casualties.⁹¹ It is through processes such as CDEM implementation that the utilitarian framework that Bentham identified can render real-world values, which may be used in proportionality calculations. For example, while Bentham would have talked in general terms about the ‘extent’ (ie scope or ambit) of a ‘pain’,⁹² the US CDEM involves drawing a radius around the target within which persons or objects may be affected by most conventional weapons and identifying ‘all collateral concerns’ which would be impacted within that radius.⁹³ Similarly, whereas Bentham would talk generally about the intensity (ie level or degree) of a pain,⁹⁴ the US CDEM considers details such as ‘potential [chemical, biological and radiological] plume and environmental hazards’ which may magnify the effects of an attack.⁹⁵ These more specific and tangible distillations of generic concepts are inherently more manageable and quantifiable than something phrased more broadly.

Of course, there is then the task of taking the collateral damage that will be caused by an attack and comparing it to the military gain that will be yielded. This would seem to be impossible because, at first glance, the two values are incommensurable: proverbial apples and oranges. However, the scholarship of Newton and May offers the beginnings of a potential solution to this problem. They state that:

The key to making proportionality manageable is to have weighing that can be done between things that are similar, not dissimilar, whenever feasible. It is much easier if the value of the military objective can be couched in terms of lives to be protected or saved so that the costs of such an operation, also often drawn in stark terms of the risk of loss of non-combatant lives, can be assessed more straightforwardly.⁹⁶

In other words, the solution to the incommensurability problem is to calculate collateral damage in terms of lives that will be lost or injuries that will be caused (perhaps using a CDEM as noted above) and then to calculate military gain in terms of the lives that will be saved or the injuries that will be prevented (perhaps using an equivalent of the CDEM that is created to assess military gain systematically). These two values can then be compared on a like-for-like basis to render a robust proportionality assessment.

In short, the key point is that proportionality is, in fact, capable of being broken down into sufficiently concrete, and comparable, values for implementation by artificial intelligence. Again, this would be a complex process and it would likely require the sort of ‘high-level’ machine intelligence discussed above by Müller and Bostrom that may only be available in twenty years or so. Still, the direction of travel is set and so it seems likely that, in time, autonomous weapons may be able to comply with this principle of IHL too.

D. Precaution

Finally, autonomous weapons must be able to comply with the principle of precaution in attack in IHL, which comprises an amalgam of verification,⁹⁷ proportionality,⁹⁸ mitigation of means and methods⁹⁹ and the duty to provide warnings¹⁰⁰. The inclusion of verification and proportionality are largely procedural reiterations of the distinction and proportionality principles discussed above and need not be recapitulated. However, the reference to mitigation of means and methods of warfare and the reference to warnings are new and discrete requirements.

In terms of mitigation of means, these are any weapons and they range from canisters of incapacitating gases to nuclear bombs. The precautionary obligation here is to make a ‘choice of means … with a view to avoiding, and in any event to minimizing, incidental loss of civilian life’.¹⁰¹ Autonomous weapons have the potential to discharge means-based precaution better than human combatants for two key reasons. Firstly, unlike human soldiers, autonomous weapons could be equipped with a wide range of different means of warfare due to their effectively unlimited physical strength. This would give them a wider selection of means to choose from in any given engagement. This is useful because, as the Harvard Program on Humanitarian Policy and Conflict Research Manual (HPCR Manual) puts it, ‘an attacker ought to choose a weapon with greater precision or lesser explosive force if doing so would minimize the likelihood of collateral damage, assuming the selection is militarily feasible [emphasis added]’.¹⁰² Clearly, if a wider panoply is available it becomes more feasible to use a weapon of lesser force. Secondly, the specifications of modern weapons are often highly complex. They cover matters such as penetrative proficiency, blast radii, duration of effect and surface impact modelling.¹⁰³ Humans require much training to become familiar with these specifications and still struggle to grapple with this level of detail on the battlefield. Machines, however, can retain all this detail perfectly throughout even the longest and most demanding deployments. Therefore, it is possible that they would be able to make better determinations about which weapons suit which situations than humans.¹⁰⁴

In terms of mitigation of methods, methods are ‘tactics … such as timing, angle and altitude of attack’.¹⁰⁵ The precautionary obligation here is to make a ‘choice of methods … with a view to avoiding, and in any event to minimizing, incidental loss of civilian life’.¹⁰⁶ Regarding timing, the US IHL manual states that ‘attacking a military objective when civilians are less likely to be present may be appropriate [and that] it may be appropriate to wait until enemy forces have departed from populated areas before attacking’.¹⁰⁷ Regarding angles of attack, the HPCR Manual observes that this is ‘one of the factors that determine where a bomb may land if it falls short of, or beyond, the target [and so] to spare a building located, for example, to the west of a target, it may be advisable to attack from the north or the south’.¹⁰⁸ In essence then, mitigation of methods of attack is largely about timing and angles. Again, autonomous weapons might be able to implement this obligation better than humans. The US Handbook of Naval Operations notes that mitigation of methods decisions must be made ‘in light of all the facts known or reasonably available’.¹⁰⁹ The amount of information available to an autonomous weapon is likely to exceed the amount available to a human. This could be in the form of data from sensors, information received from other military sources or indeed archived material. All this information could be synthesised quickly to decide on attack timing and vectors that would minimise the level of collateral damage. The ability to perform these analyses well is becoming more and more important in modern warfare given that, as Melzer notes, conflicts are increasingly waged in tightly packed urban areas ‘where the margin for error vanishes’.¹¹⁰ Machines could use their unique mathematical talents and their immunity to the fog of war to achieve the best possible outcomes in such environments.¹¹¹

Regarding the requirement to deliver warnings, according to Additional Protocol I, ‘effective advance warning shall be given of attacks which may affect the civilian population, unless circumstances do not permit’.¹¹² The HPCR Manual notes that warnings may be given ‘for instance, through dropping leaflets or broadcasting’.¹¹³ Still, as the USA notes, ‘there is no set form for warnings [and they] may be general, communicated to the national leadership of the enemy State, or delivered to the civilian population through military information support operations’.¹¹⁴ Nonetheless, there are some general ground rules and ‘warnings ought to be as specific as circumstances permit’¹¹⁵ with the degree of specificity required depending on factors such as ‘the length of time [given] prior to the attack’.¹¹⁶ For Thurnher, the warning must be sufficient to provide the civilian population ‘the opportunity to take measures to avoid the dangers’.¹¹⁷ In this context also, autonomous weapons have the potential to discharge precaution better than humans. Firstly, issuing warnings is a relatively simple matter. It requires only the ability to take actions such as sounding a siren to advise of an imminent attack. Indeed, Thurnher noted that ‘it seems reasonable to assume that nations could design autonomous systems to have broadcast or loudspeaker-type capabilities [or] other types of system’.¹¹⁸ One could even imagine autonomous weapons being outfitted with more warning systems than would be feasible for a human to carry, better placing them to implement effective warnings. Secondly, the deployment of autonomous weapons could make the delivery of warnings less risky for attackers, which makes them more feasible which, in turn, makes warnings more likely and thus improves the position of civilians. For human soldiers, there is usually a direct relationship between the level of warning they give to their targets and the level of risk to which they subject themselves. This is because giving a warning can result in greater exposure to the enemy in terms of physical proximity or duration. However, when autonomous weapons are used, there is no additional risk to the lives of the attackers. The autonomous weapon itself is more likely to be destroyed, but it is only an object after all and its destruction is worthwhile if it means that civilians were given their proper warning.¹¹⁹

4. Conclusion

Autonomous weapons remain a hot topic in IHL discourse. As yet, there is no consensus on how best to regulate this technology or, indeed, whether to give it any new regulation at all. In the absence of any ban, treaty, handbook or other form of regulation, this article has submitted that the best way to view autonomous weapons is through the lens of the principles of IHL. Throughout this short exposition of that approach, certain conclusions have emerged. The first is that humanity and military necessity are not, in fact, principles of IHL at all owing to their failure to carry the key traits of principles. The second is that to comply with the true principles of IHL—distinction, proportionality and precaution—autonomous weapons would need to be highly sophisticated in terms of the level of artificial intelligence they possess. The third is that, at present, limitations on artificial intelligence and the inability of machines to form sound, context-sensitive, judgements based on external stimuli mean that they do not possess the necessary sophistication. The fourth, however, is that advances in technology have brought machines to the cusp of complying with IHL. The final piece of the puzzle will be for machines to develop ‘high level’ intelligence and, according to experts that may only be a couple of decades away.

Footnotes

1

Pope Francis, ‘The Future we Want, the United Nations we Need: Reaffirming our Collective Commitment to Multilateralism–Confronting Covid-19 through Effective Multilateral Action’ (United Nations General Assembly, New York, 25 September 2020) <https://bit.ly/3kQ2tum> accessed 1 January 2022.

2

ICRC, Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons (ICRC 2020) 8.

3

ICRC, ‘Autonomous Weapon Systems – Q&A’ (ICRC, 12 November 2014) para 13 <bit.ly/2ixib2p> accessed 1 January 2022.

4

S Casey-Maslen, ‘Pandora’s Box? Drone Strikes under Jus ad Bellum, Jus in Bello, and International Human Rights Law’ (2012) 94 International Review of the Red Cross 597, 606–16.

5

T McFarland, Autonomous Weapon Systems and the Law of Armed Conflict (CUP 2020) 85.

6

Human Rights Watch, Losing Humanity: The Case Against Killer Robots (Human Rights Watch 2012) 2.

7

ibid 30.

8

N Melzer, Interpretive Guidance on the Notion of Direct Participation in Hostilities under International Humanitarian Law (ICRC 2009).

9

Human Rights Watch (n 6) 31.

10

ibid 33.

11

O Bowcott, ‘UK Opposes International Ban on Developing “Killer robots”’ The Guardian (London, 13 April 2015) para 19.

12

United Nations Human Rights Council, ‘Report of the Special Rapporteur on Extrajudicial, Summary or Arbitrary Executions, Christof Heyns’ (9 April 2013) UN Doc A/HRC/23/47 para 75.

13

E Winter, ‘The Accountability of Software Developers for War Crimes Involving Autonomous Weapons: The Role of the Joint Criminal Enterprise Doctrine’ (2021) 83(1) University of Pittsburgh Law Review 51, 84.

14

O Ulgen, ‘Human Dignity in an Age of Autonomous Weapons: Are We in Danger of Losing an “Elementary Consideration of Humanity”?’ (2017) 17 Baltic Yearbook of International Law 167.

15

N Sharkey, ‘Killing Made Easy’ in P Lin, K Abney and GA Bekey (eds), Robot Ethics: The Ethical and Social Implications of Robotics (MIT Press 2012) 118.

16

N Sharkey, ‘Automated Killers and the Computing Profession’ (2007) 40(11) Computer 122, 123.

17

T McFarland, ‘Factors Shaping the Legal Implications of Increasingly Autonomous Military Systems’ (2015) 97 International Review of the Red Cross 1313, 1335.

18

D Amoroso and B Giordano, ‘Who Is to Blame for Autonomous Weapons Systems’ Misdoings?’ in E Carpanelli and N Lazzerini (eds), Use and Misuse of New Technologies: Contemporary Challenges in International and European Law (Springer 2019) 211.

19

J Haner and D Garcia, ‘The Artificial Intelligence Arms Race: Trends and World Leaders in Autonomous Weapons Development’ (2019) 10(3) Global Policy 331, 332.

20

GC Allen, Understanding China’s AI Strategy: Clues to Chinese Strategic Thinking on Artificial Intelligence and National Security (Center for a New American Security, 6 February 2019) para 2 <https://bit.ly/3u1epyj> accessed 1 January 2022.

21

Stockholm International Peace Research Institute, Military Expenditure Database (Stockholm International Peace Research Institute 2020) <https://bit.ly/2PInm0Q> accessed 1 January 2022.

22

S Bendett, ‘Red Robots Rising: Behind the Rapid Development of Russian Unmanned Military Systems’ The Strategy Bridge (12 December 2017) para 3 <https://bit.ly/3u36L6M> accessed 01 January 2022.

23

ibid para 18.

24

T O’Connor, ‘Russia’s Military Challenges U.S. and China by Building a Missile That Makes Its Own Decisions’ Newsweek (27 July 2017) <https://bit.ly/2Pg0Lsj> accessed 1 January 2022.

25

United States Department of Defense, ‘Autonomy in Weapons Systems’ Directive Number 3000.09 (21 November 2012).

26

D Gettinger, ‘Summary of Drone Spending in the FY 2019 Defense Budget Request’ (2018) Center for the Study of the Drone at Bard College 1, 1.

27

Bowcott (n 11) para 7.

28

BAE Systems, ‘Taranis’ (BAE Systems 2021) <http://bit.ly/2uamk2j> accessed 1 January 2022.

29

D-P Baker and others, ‘Introducing Guiding Principles for the Development and Use of Lethal Autonomous Weapon Systems’ (2020) E-International Relations <https://bit.ly/3bngewB> accessed 1 January 2022.

30

MN Schmitt, ‘Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics’ (2013) Harvard National Security Journal 5 <https://bit.ly/ 2WCnpwb> accessed 01 January 2022.

31

ibid 23.

32

ibid.

33

WH Boothby, ‘Highly Automated and Autonomous Technologies’ in WH Boothby (ed), New Technologies and the Law in War and Peace (CUP 2019) 159.

34

ibid 150.

35

ibid 160.

36

K Anderson and MC Waxman, Law and Ethics for Autonomous Weapon Systems: Why a Ban Won’t Work and How the Laws of War Can (Stanford University, The Hoover Institution 10 April 2013) 6 <bit.ly/2pBFnO9> accessed 1 January 2022.

37

JS Thurnher, ‘Feasible Precautions in Attack and Autonomous Weapons’ in W Heintschel von Heinegg, R Frau and T Singer (eds), Dehumanization of Warfare: Legal Implications of New Weapon Technologies (Springer 2018) 114.

38

ibid 110.

39

M Ekelhof, ‘Lifting the Fog of Targeting’ (2018) 71(3) Naval War College Review 61, 79 and 83.

40

Convention on Conventional Weapons, ‘Report of the 2016 Informal Meeting of Experts on Lethal Autonomous Weapons Systems (LAWS)’ (10 June 2016) UN Doc CCW/CONF.V/2 Annex 14 para 3.

41

Group of Governmental Experts, ‘Report of the 2019 Session of the Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems’ (25 September 2019) UN Doc CCW/GGE.1/2019/3 para 16.

42

ibid para 22(c).

43

ibid para 25(b).

44

Campaign to Stop Killer Robots, ‘Diplomatic Talks Re-Convene’ (Campaign to Stop Killer Robots, 25 September 2020) para 8 <https://bit.ly/3lQ2oIz> accessed 1 January 2022.

45

Group of Governmental Experts, ‘Draft Report of the 2021 Session of the Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems’ (8 December 2021) UN Doc CCW/GGE.1/2021/CRP.1.

46

I Jones, ‘Historic Opportunity to Regulate Killer Robots Fails as a Handful of States Block the Majority’ (Campaign to Stop Killer Robots, 17 December 2021) para 4 <https://bit.ly/32Rtw4Z> accessed 1 January 2022.

47

Haner and Garcia (n 19) 331.

48

P Apps, ‘New Era of Robot War May be Underway Unnoticed’ Reuters (London, 10 June 2021) para 1.

49

Nuclear Weapons Advisory Opinion [1996] ICJ Rep 226 paras 78, 92 and 95.

50

ibid paras 30, 32, 43 and 48.

51

ibid paras 77, 78 and 95.

52

ibid paras 41, 44 and 46.

53

United Kingdom Ministry of Defence, The Manual of the Law of Armed Conflict (OUP 2004) 21.

54

Danish Ministry of Defence, Military Manual on International Law Relevant to Danish Armed Forces in International Operations (Defence Command Denmark 2016) 145–155.

55

New Zealand Defence Force, Manual of Armed Forces Law (New Zealand Defence Force 2019) 4.6.1.

56

C Droege, ‘Get Off My Cloud: Cyber Warfare, International Humanitarian Law, and the Protection of Civilians’ (2012) 94(886) International Review of the Red Cross 533, 553.

57

R Kolb, Advanced Introduction to International Humanitarian Law (Edward Elgar 2014) 76.

58

Protocol Additional to the Geneva Conventions of 12 August 1949, and relating to the Protection of Victims of International Armed Conflicts (adopted 08 June 1977, entered into force 07 December 1978) 1125 UNTS 3, art 1(2).

59

Convention (IV) respecting the Laws and Customs of War on Land and its annex: Regulations concerning the Laws and Customs of War on Land (adopted 18 October 1907, entered into force 26 January 1910) 187 CTS 227, art 23(g).

60

E Winter, ‘Pillars not Principles: The Status of Humanity and Military Necessity in the Law of Armed Conflict’ (2020) 25 Journal of Conflict and Security Law 1, 10–21.

61

RM Dworkin, ‘The Model of Rules’ (1967) 35 University of Chicago Law Review 14, 27.

62

Y Dinstein, The Conduct of Hostilities under the Law of International Armed Conflict (3^rd edn, CUP 2016) 9.

63

Dworkin (n 61) 29.

64

J Raz, ‘Legal Principles and the Limits of Law’ (1972) 81(5) Yale Law Journal 823, 837.

65

Additional Protocol I (n 58) art 11(2)(b).

66

Winter (n 60) 8.

67

ibid 9–10.

68

ibid.

69

Additional Protocol I (n 58) art 48.

70

ibid art 41(2)(b).

71

L Green, The Contemporary Law of Armed Conflict (Juris Publishing 2000) 124.

72

E Winter, ‘The Compatibility of Autonomous Weapons with the Principle of Distinction in the Law of Armed Conflict’ (2020) 69(4) International and Comparative Law Quarterly 845, 868.

73

ibid 846.

74

Airsoc, ‘Where Hazards Lurk’ (Airsoc, January 2020) para 5 <https://bit.ly/2Tzvcb5> accessed 1 January 2022.

75

ibid para 3.

76

ibid para 6.

77

Winter (n 72) 853–59.

78

T Maddox, ‘PatScan Platform Detects Hidden Weapons, Chemicals, and Bombs’ (TechRepublic, 10 January 2020) <https://tek.io/2IdOFZB> accessed 1 January 2022.

79

Patriot One Technologies, ‘About’ (Patriot One Technologies) <https://bit.ly/3dT3UWH> accessed 1 January 2022.

80

ibid.

81

Additional Protocol I (n 58) art 41(2).

82

Agrointelli, ‘RoboWeedMaPS’ (Agrointelli) <https://bit.ly/3dAkMkZ> accessed 1 January 2022.[AQ]

83

RN Jorgensen, RoboWeedMaPS: How Deep Learning Can Help Farmers Get Rid of Weeds (Aarhus University Department of Engineering, 14 January 2019) <https://bit.ly/30aI4rm> accessed 1 January 2022.

84

VC Müller and N Bostrom, ‘Future Progress in Artificial Intelligence: A Survey of Expert Opinion’ in Vincent C Müller (ed), Fundamental Issues of Artificial Intelligence (Springer 2016).

85

T Walsh, 2062: The World that AI Made (La Trobe University Press 2018).

86

Human Rights Watch (n 6) 30.

87

Additional Protocol I (n 58) arts 51(5)(b) and 57(2)(a)(iii).

88

Human Rights Watch (n 6) 33.

89

J Bentham, The Principles of Morals and Legislation (Prometheus Books 1988) 3.

90

E Winter, ‘Autonomous Weapons in Humanitarian Law: Understanding the Technology, Its Compliance with the Principle of Proportionality and the Role of Utilitarianism’ (2018) 6(1) Groningen Journal of International Law 183, 194.

91

E Winter, ‘The Compatibility of the Use of Autonomous Weapons with the Principle of Precaution in the Law of Armed Conflict’ (2020) 58(2) Military Law and the Law of War Review 240, 262.

92

Bentham (n 89) 30.

93

US Department of Defense, ‘No-Strike and the Collateral Damage Estimation Methodology’ (2012) CJCSI 3160.01A D-A-7 <https://bit.ly/2Zzzggj> accessed 1 January 2022.

94

Bentham (n 89) 30.

95

US Department of Defense (n 93) D-A-7.

96

M Newton and L May, Proportionality in International Law (OUP 2014) 285.

97

Additional Protocol I (n 58) art 57(2)(a)(i).

98

ibid art 57(2)(a)(iii).

99

ibid art 57(2)(a)(ii).

100

ibid art 57(2)(c).

101

ibid art 57(2)(a)(ii).

102

Program on Humanitarian Policy and Conflict Research at Harvard University, HPCR Manual on International Law Applicable to Air and Missile Warfare (CUP 2013) 145.

103

Winter (n 91) 267.

104

ibid 265–68.

105

HPCR Manual (n 102) 145.

106

Additional Protocol I (n 58) art 57(2)(a)(ii).

107

Office of General Counsel, Department of Defense Law of War Manual (US Department of Defense 2015) para 5.11.2.

108

HPCR Manual (n 102) 145.

109

US Navy, US Marine Corps and US Coastguard, The Commander’s Handbook on the Law of Naval Operations (US Naval War College 2007) para 8.3.1.

110

N Melzer, ‘The Principle of Distinction between Civilians and Combatants’ in A Clapham and P Gaeta (eds), The Oxford Handbook of International Law in Armed Conflict (OUP 2014) 298.

111

Winter (n 91) 268–71.

112

Additional Protocol I (n 58) art 57(2)(c).

113

HPCR Manual (n 102) 153.

114

Office of General Counsel (n 107) para 5.11.1.1.

115

HPCR Manual (n 102) 153.

116

ibid 154.

117

Thurnher (n 37) 111.

118

ibid 112.

119

Winter (n 91) 271–73.

© Oxford University Press 2022.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

© Oxford University Press 2022.