Story image

When AI goes rogue - a look into its possible futures

28 May 2018

What happens when artificial intelligence (AI) goes bad? According to the Electronic Frontier Foundation, AI and machine learning will bring benefits in diverse areas such as transport, health, art and science, but we’ve already seen things go horribly wrong.

Today’s computers are inherently insecure so they’re a poor choice for high-stakes machine learning systems and AI – and according to the Electronic Frontier Foundation, we need to consider the implications these new technologies may have for computer security.

Earlier this year the Electronic Frontier Foundation was one of six institutions that released a report called The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation.

Also involved in the report was the Future of Humanity Institute, the University of Oxford, the Centre for the Study of Existential Risk, the University of Cambridge, the Center for a New American Security, and OpenAI.

The report looks at AI’s potential impact on digital security, physical security, and political security.

It says there are specific security-relevant properties of AI, including its dual use for civilian and military purposes; its scalability; the ability for its algorithms to be rapidly distributed; and its ability to exceed human capabilities.

They can also expand existing threats, introduce new threats, and alter the typical character of threats, allowing attacks to be more versatile, effective, and targeted.

In terms of digital security, AI could influence email attacks such as spear phishing to become more automated – and it could even eliminate the need for the attacker to speak the same language as the target.

“Many important IT systems have evolved over time to be sprawling behemoths, cobbled together from multiple different systems, under-maintained and — as a consequence — insecure. Because cybersecurity today is largely labour-constrained,” the report notes.

AI could also target malware’s behaviour that it becomes impossible for humans to control in a manual way. The Stuxnet malware is a clear example of how a malware cannot receive commands once it infects computers.

In addition to automation of social engineering attacks, AI could also automatically discover vulnerabilities, automate hacking processes by evading detection and responding to behavioural changes from the target; it could mimic human-like denial-of-service attacks, and exploit legitimate AI itself.

Althrough offensive use of AI has only publicly been disclosed through experiments by white hat hackers, the report says it’s only a matter of time before it is used for malicious consequences – if it is not already happening.

AI could disrupt physical security by repurposing commercial AI systems for terrorism – for example using autonomous vehicles to cause crashes. It could enable distributed swarming attacks for surveillance, and it could increase the scale of attacks.

AI could also affect political security by allowing states to automate surveillance platforms.

“State surveillance powers of nations are extended by automating image and audio processing, permitting the collection, processing, and exploitation of intelligence information at massive scales for myriad purposes, including the suppression of debate,” the report says.

It could also achieve highly realistic videos to support fake news reports, manipulate information availability; automate influencing campaigns; and hyper-personalise disinformation campaigns.

The report recommends four approaches to responsible AI use:

1. Policymakers should collaborate closely with technical researchers to investigate, prevent, and mitigate potential malicious uses of AI.

2. Researchers and engineers in artificial intelligence should take the dual-use nature of their work seriously, allowing misuse-related considerations to influence research priorities and norms, and proactively reaching out to relevant actors when harmful applications are foreseeable.

3. Best practices should be identified in research areas with more mature methods for addressing dual-use concerns, such as computer security, and imported where applicable to the case of AI.

4. Actively seek to expand the range of stakeholders and domain experts involved in discussions of these challenges.

Norwegian aluminium manufacturer hit hard by LockerGoga ransomware attack
“IT systems in most business areas are impacted and Hydro is switching to manual operations as far as possible.”
Slack doubles down on enterprise key management
EKM adds an extra layer of protection so customers can share conversations, files, and data while still meeting their own risk mitigation requirements.
Security professionals want to return fire – Venafi
Seventy-two percent of professionals surveyed believe nation-states have the right to ‘hack back’ cybercriminals.
Alcatraz AI to replace corporate badges with AI security
The Palo Alto-based startup supposedly leverages facial recognition, 3D sensing, and machine learning to enable secure access control.
Unencrypted Gearbest database leaves over 1.5mil shoppers’ records exposed
Depending on the countries and information requirements, the data could give hackers access to online government portals, banking apps, and health insurance records.
Mozilla launches Firefox Send, an encrypted file transfer service
Mozille Firefox has launched a free encrypted file transfer service that allows people to securely share files from any web browser – not just Firefox.
Ransomware’s decline equals cryptomining’s rise
ESET’s Security Days Conference recently took place to go over the current threat environment and what to look out for next.
IoT and DDoS attacks: A match made in heaven
A10 Network’s Adrian Taylor uses findings from a number of reports to illustrate his point that advances in technology are facilitating cybercrime.