The Ethics of Autonomous Weapons Systems

November 21 -
 21, 2014

The Conference

Autonomous Weapons Systems (AWS) are defined by the U.S. Department of Defense as “a weapon system(s) that, once activated, can select and engage targets without further intervention by a human operator.” Since the crucial distinguishing mark of human reasoning is the capacity to set ends and goals, the AWS suggests for the first time the possibility of eliminating the human operator from the battlefield.  The development of AWS technology on a broad scale, therefore, represents the potential for a transformation in the structure of war that is qualitatively different from previous military technological innovations. 

In May, the first Meeting of Experts on Lethal Autonomous Weapons Systems was held at the United Nations in Geneva.  The participants recognized the potential of AWS to alter radically the nature of war, as well as a variety of ethical dilemmas such weapons systems raise.  Worldwide concern has been growing about the idea of developing weapons systems that take human beings “out of the loop,” though the precise nature of the ethical challenges to developing such systems, and even possible ethical benefits, have not yet been clearly identified. 

The idea of fully autonomous weapons systems raises a host of intersecting philosophical, psychological, and legal issues. For example, it sharply raises the question of whether moral decision-making by human beings involves an intuitive, non-algorithmic capacity that is not likely to be captured by even the most sophisticated of computers?  Is this intuitive moral perceptiveness on the part of human beings ethically desirable?  Does the automaticity of a series of actions make individual actions in the series easier to justify, as arguably is the case with the execution of threats in a mutually assured destruction scenario?  Or does the legitimate exercise of deadly force should always require a “meaningful human control?”  If the latter is correct, what should be the nature and extent of a human oversight over an AWS? 

Additional questions arise with regard to the very definition of an AWS.  Should the definition focus on the system’s capabilities for autonomous target selection and engagement, or on the human operator’s use of such capabilities?  Should the human operator’s pre-engagement intention have a decisive bearing on the system’s definition as an AWS?  Furthermore, AWS present a unique challenge to the way legal responsibility in combat should be assessed.  If a given AWS is merely applying a set of preprogrammed instructions, then, presumably its designers and operators are the ones morally responsible for its behavior.  But if the AWS in question is a genuine moral discerner in its own right, that appears to shift the locus of responsibility to the automated system itself.  And if this is the case, what are the implications for legal liability?  Who, if anyone, should bear the legal liability for decisions the AWS makes? 

The purpose of this conference is to address such questions by bringing together distinguished scholars and practitioners from the academy, civil society, government service and the military, to engage in two days of constructive discussion and exploration of the moral and legal challenges posed by Autonomous Weapons Systems


Friday, Nov. 21, 2014 

8:30 a.m. – 9:15 a.m.Registration & Breakfast – Silverman 147
9:15 a.m. – 9:30 a.m.Welcome Remarks: Dean Wendell Pritchett, Interim Dean, Penn Law and Bill Burke-White, Director of the Perry World House
9:30 a.m. – 10:45 a.m.Session 1 – Human Morality and the Problem of Intelligent MachinesSilverman 147 
Moderator: Claire Finkelstein
10:45 a.m. – 11:15 a.m.Break
11:15 a.m. – 12:30 p.m.Session 2 – The Law and Ethics of Autonomous Weapons SystemsSilverman 147
Moderator: Jens Ohlin
12:30 p.m. – 1:45 p.m. Lunch – Silverman 245, Levy Conference Room
Lunch Keynote – Keeping Weapons Control in Human Hands
Professor Noel Sharkey, University of Sheffield
1:45 p.m.Walk to Van Pelt Library, 3420 Walnut Street, for final session, keynote address, cocktail hour and dinner
2:15 p.m. – 3:30 p.m.Session 3 – The Rationality and Morality of Automaticity
Meyerson Conference Room, Van Pelt Library, 2nd Floor 
Moderator: Duncan MacIntosh
3:30 p.m. – 4:00 p.m.Break
4:00 p.m. – 5:30 p.m.Keynote Panel – Should “Killer Robots” be Banned?

The Class of 78 Pavilion, Van Pelt Library, 6th Floor
Moderator: Professor Bill Burke-White, Director, Perry World House
Participants: Mr. Charles A. Blanchard, Arnold & Porter, LLP; former General Counsel and Chief Ethics Officer for the U.S. Air Force, Ms. Bonnie Docherty Human Rights Watch; Harvard Law School, Major General (ret.) Robert LatiffUniversity of Notre Dame, Professor Wendell WallachYale University
5:30 p.m. – 6:30 p.m.Cocktail Reception – Moelis Terrace, Van Pelt Library, 6th Floor
6:30 p.m. – 8:00 p.m.Dinner –   Seminar Rooms, Van Pelt Library , 6th Floor 

Saturday, Nov. 22, 2014

8:45 a.m. – 9:30 a.m.Breakfast – Silverman 147
9:30 a.m. – 10:45 a.m.Session 4 – Uniquely Human? On Intuition, Mercy and Moral Decision MakingSilverman 147
Moderator: Sharon Lloyd
10:45 a.m. – 11:15 a.m.Break
11:15 a.m. – 12:30 p.m.Session 5 – On the Concept of Meaningful Human ControlSilverman 147
Moderator: Michael Horowitz
12:30 p.m. – 2:15 p.m.Lunch – Silverman 245, Levy Conference Room 
2:15 a.m. – 3:30 p.m.Session 6 – Responsibility for Acts of Intelligent MachinesSilverman 147
Moderator: Kenneth Anderson


Professor Kenneth Anderson

American University

Professor Ronald Arkin

Georgia Institute of Technology

Professor Peter Asaro

The New School for Public Engagement; Stanford Law School

Ms. Katherine Baker

Department of State

Mr. Charles A Blanchard

Arnold & Porter, LLP; former General Counsel and Chief Ethics Officer for the U.S. Air Force

Professor William W. Burke-White

University of Pennsylvania Law School

Mr. William Craven

Federal Systems Inc

Professor Mary Cummings

Duke University

Ms. Bonnie Docherty

Human Rights Watch; Harvard Law School

Professor Claire Finkelstein

University of Pennsylvania Law School

CAPT (Ret) Robert G. Fuller, Jr.

Former Navy Jag and Penn Law Alum

Ms. Ashling Gallagher

University of Pennsylvania

Professor Kevin Govern

Ave Maria School of Law

Professor Oren Gross

University of Minnesota Law School

Mr. Paul Haaga


CAPT USN (Ret) Professor Mark Hagerott

US Naval Academy Center for Cyber Studies

Mr. Christopher Harland

International Committee of the Red Cross

Professor Michael Horowitz

University of Pennsylvania

Ms. Jessie Ilkson

Department of Defense

Professor Deborah Johnson

University of Virginia

Professor Vijay Kumar

University of Pennsylvania

MAJ GEN (Ret) Robert Latiff

University of Notre Dame

Professor Sharon Lloyd

University of Southern California

Professor Duncan MacIntosh

Dalhousie University

Professor Christopher W. Morris

University of Maryland

Professor Jens Ohlin

Cornell University School of Law

Professor Daniel D. Perlmutter

University of Pennsylvania

Professor Connie Rosati

University of Arizona

Professor Harvey Rubin

University of Pennsylvania

Mr. Ilya Rudyak

University of Pennsylvania

Mr. Paul Scharre

Center for a New American Security

Professor Matthias Scheutz

Tufts School of Engineering

Professor Noel Sharkey

University of Sheffield

Professor Markus Wagner

University of Miami

Professor Wendell Wallach

Yale University

Ms. Gay Walling

University of Pennsylvania

Dr. Stephen Xenakis

Center for Translational Medicine; Physicians for Human Rights

Mr. Jules Zacher

Council for a Livable World

Background Readings

Recent Articles on Autonomous Weapons Systems

Michael C. Horowitz and Paul Scharre
The Morality of Robotic War, NYT, May 26, 2015

Nicholas Carr
Automation Makes Us Dumb, WSJ, Nov 22, 2014

Jane Wakefield,
Robots Face New Creativity Test, BBC News, Nov, 21, 2014

Michael C. Horowitz and Paul Scharre
Killer Robots Save Lives. At Least Some of Them Do, Politico, Nov 19, 2014

John Markoff
Fearing Bombs That Can Pick Whom to Kill, NYT, Nov 11, 2014

Human Morality and the Problem of Intelligent Machines

Ronald  Arkin
Lethal Autonomous Systems and the Plight of the Non-combatant

John Searle
Minds, Brains, and Programs, 3 The Behavioral and Brain Sciences 417 (1980).

Robert Sheckley
Watchbird, Galaxy, Feb. 1953, at 74

David Cole
The Chinese Room Argument, The Stanford Encyclopedia of Philosophy (June 2014)

The Law and Ethics of Autonomous Weapons Systems

Keith Abney, Patrick Lin and George Bekey.
Autonomous Military Robotics: Risk, Ethics, and Design, Cal Poly (2008)

Kenneth Anderson, Matthew C. Waxman
Law and Ethics for Autonomous Weapons Systems: Why a Ban Won’t Work and How the Laws of War Can, Policy Review 2013, at 1.

U.S. Department of Defense (DoD) policy on autonomy in weapons: DoD Directive 3000.09
Autonomy in Weapon Systems, (2012)

Additional Protocol I to the Geneva Conventions (Article 1 par. 2, Article 36)

Mary Ellen O’Connell
The Legal and Ethical Requirement That Humans Make Near-Time Lethal Decisions, in The American Way of Bombing Changing Ethical and Legal Norms from Flying Fortresses to Drones (Matthew Evangelista and Henry Shue editors, 2014)

Colin Allen, Wendell Wallach
Framing Robot Arms Control, 15 Ethics and Information Technology 125 (2013)

Chantal Grut
The Challenge of Autonomous Lethal Weapon Systems to International Law, 18.1 Journal of Conflict and Security Law 5 (2013)

Michael N. Schmitt
Autonomous Weapon Systems and International Humanitarian Law: Reply to the Critics, Harvard National Security Journal Features (Feb. 5, 2013, 2:07 PM)

UK Ministry of Defense, Joint Doctrine Note 2/11
The UK Approach to Unmanned Aircraft Systems

The Rationality and Morality of Automaticity

Larry Alexander
The Doomsday Machine: Proportionality, Punishment and Prevention, 63.2 The Monist 199 (1980)

David Gauthier
Assure and Threaten, 690-721 Ethics, 104,4 (1994)

Gregory Kavka
The Toxin Puzzle, 33-36 Analysis 43,1 (1983)

Uniquely Human? On Intuition, Mercy and Moral Decision Making

Martha Nussbaum
Equity and Mercy, 22.2 Philosophy and Public Affairs 83 (1993)

Deborah G. Johnson and Merel Noorman
Artefactual Agency and Artefactual Moral Agency

Noel E. Sharkey
The Evitability of Autonomous Robot Warfare, 94 International Review of the Red Cross 787 (2012)

On the Concept of Meaningful Human Control

Peter Asaro
On Banning Autonomous Weapon Systems: Human Rights, Automation, and the Dehumanization of Lethal Decision-Making, in Ethics of 21st Century Military Conflict 257-269 (Erica L. Gaston & Patti Tamara Lenard eds. 2012).

Shane Harris
Out of the Loop: The Human-free Future of Unmanned Aerial Vehicles

Ezzio Di Nucci & Fillipo Santoni de Dio
Who’s Afraid of Robots? Fear of Automation and the Ideal of Direct Control, in Roboethics in Film (Fiorella Battaglia & Nathalie Weidenfield eds. forthcoming).

Geoffrey S. Corn
Autonomous Weapon Systems: Managing the Inevitability of ‘Taking the Man out of the Loop’. Social Science Research Network, June 14, 2014.

William Marra and Sonia McNeil
Understanding ‘The Loop’: Regulating the Next Generation of War Machines. Harvard Journal of Law and Public Policy, 36 no. 3 (2013).

Mary Ellen O’Connell
Banning Autonomous Killing: The Legal and Ethical Requirement That Humans Make Near-Time Lethal Decisions

Responsibility for Acts of Intelligent Machines

Daniel C. Dennett
When Hal Kills, Who’s to Blame? Computer Ethics, in Hal’s Legacy: 2001’s Computer as Dream and Reality 351-365 (David Stork ed. 1997).

Peter M. Asaro
A Body to Kick But Still No Soul to Damn: Legal Perspectives on Robotics, in Robot Ethics: The Ethical and Social Implications of Robotics 169-186 (Keith Abney, George Bekey, & Patrick Lin eds., 2011)

Deborah G. Johnson
Technology with No Human Responsibility?

Deborah G. Johnson and Merel Noorman
Responsibility Practices in Robotic Warfare

Deborah G. Johnson and Merel Noorman
Negotiating Autonomy and Responsibility in Military Robots

Merel Noorman
Computing and Moral Responsibility, The Stanford Encyclopedia of Philosophy (June 2014)

Reports and Memoranda

International Committee of the Red Cross
Autonomous Weapons Systems Technical, Military, Legal and Humanitarian Aspects (November 1, 2014)

International Committee of the Red Cross
Expert Meeting Report: Autonomous Weapon Systems, Technical, Military, Legal and Humanitarian Aspects (Geneva, Switzerland 26 to 28 March 2014)


Gabriella Blum
An Emerging Threats Essay: Invisible Threats

Bonnie Docherty
Shaking the Foundations: the Human Rights Implications of Killer Robots, Human Rights Watch (2014)

Mark Hagerott
Robots on the Battlefield: Contemporary Issues and Implications for the future, Chapter 3: Robots, Cyber, History, and War

Mark Hagerott
Limiting automation in a cyber-insecure world

Michael Hastings
The Rise of the Killer Drones: How America Goes to War in Secret

Christof Heyns
Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions

Michael C. Horowitz
Coming Next in Military Tech, 70.1 Bulletin of the Atomic Scientists 54 (2014)

Michael C. Horowitz
The Looming Robotics Gap: Why America’s Global Dominance in Military Technology is Starting to Crumble, Foreign Policy, May 5, 2014

Aaron M. Johnson
The Morality of Autonomous Robots, Journal of Military Ethics, Vol. 12, No. 2, 2013.

Deborah G. Johnson and Merel Noorman
Recommendations for Future Development of Artificial Agents

Deborah G. Johnson and Thomas M. Powers
Computers as Surrogate Agents, in Moral Philosophy and Information Technology 251-269 (Jeroen van de Hoen & John Weckert eds. 2008).

John F. Keane and Stephen S. Carr
A Brief History of Early Unmanned Aircraft

Gary E. Marchant et al.
International Governance of Autonomous Military Robots, 12 U. PA L. Rev. 2010 (2011)

John Markoff
Scientists Consider Repurposing Robots for Ebola

John Markoff
Fearing Bombs That Can Pick Whom to Kill

Nicholas Marsh
Defining the Scope of Autonomy: Issues for the Campaign to Stop Killer Robots, Peace Research Institute Oslo (

Markus Wagner
The Dehumanization of International Humanitarian Law: Legal, Ethical, and Political Implications of Autonomous Weapons Systems

Wendell Wallach
Terminating the Terminator: What to do About Autonomous Weapons

Contact us

For any questions regarding the conference or registration, please contact: Jennifer Cohen at

Share The Ethics of Autonomous Weapons Systems on:

The Ethics of Autonomous Weapons Systems