An autonomous drone ‘does what it is told, killing whatever is inside, combatant or child’. Photograph: Houthi Military Media Center Handout/EPAView image in fullscreenAn autonomous drone ‘does what it is told, killing whatever is inside, combatant or child’. Photograph: Houthi Military Media Center Handout/EPALettersAutonomous drones and the future of war in an AI-driven worldMachines that kill without distinction, answerable to no one, leave no one safe, writes military surgeon Dr Darren MannStuart Russell is right that AI’s builders fear losing control of it (Experts are warning: our AI arms race is putting humanity at risk, 11 August). But it is not only a future superintelligence’s problem; it is already loaded into weapons, where a loss of control kills a civilian today, not humanity tomorrow.An autonomous drone told to clear a “kill box” is the genie your other columnists described (How do we prevent AI agents from going rogue? It starts with a new kind of measurement, 18 July): it does what it is told, killing whatever is inside, combatant or child. The gap between a target and a lawful target is the principle of distinction, which no instruction closes, because the instruction was lawful.Russell doubts asking a superintelligence how to control itself. In war the check need not be another opaque mind but a deterministic one: its rulebook the Geneva conventions, its judgments auditable, returned to a human. We need not trust it; we can check it. Some of us are working on it.A further danger is peculiar to war. Autonomy is sold as combat without our own dead, and a bloodless war is easier to start. But that is a fallacy: you can take the human from the targeting loop, not the nation from the war. Ground must be won and held; the dead still come, in attack or defence. Armies no longer go to war; nations do.If we cannot teach a drone to keep the laws of war, we will not teach a greater intelligence to keep faith with all we value. To abandon those laws for survival is the confusion of interests with values that our strategists warn against; here the two are one. Machines that kill without distinction, answerable to no one, leave no one safe. Our interests do not merely permit us to hold the line; they demand it.Dr Darren MannMilitary surgeon, LondonExplore more on these topicsTechnologyDrones (military)AI (artificial intelligence)MilitarylettersShareReuse this content