In collaboration with the US Cybersecurity and Infrastructure Agency (CISA), the FBI’s Internet Crime Complaint Center (IC3) and twenty partner organzations worldwide, the UK National Cyber Security Centre released guidelines for secure artificial intelligence (AI) development, at the end of November.
The guidelines document contends that AI systems are subject to novel security vulnerabilities that need to be considered alongside standard cyber security threats. When the pace of development is high – as is the case with AI – security can often be a secondary consideration. Security must be a core requirement, not just in the development phase, but throughout the life cycle of the system.
For this reason, the guidelines are broken down into four key areas within the AI system development life cycle: secure design, secure development, secure deployment, and secure operation and maintenance. For each section, the agencies suggest considerations and mitigations that will help reduce the overall risk to an organisational AI system development process.
- Secure design: Guidelines that apply to the design stage of the AI system development life cycle. It covers understanding risks and threat modelling, as well as specific topics and trade-offs to consider on system and model design.
- Secure development: Guidelines that apply to the development stage of the AI system development life cycle, including supply chain security, documentation, and asset and technical debt management.
- Secure deployment: Guidelines that apply to the deployment stage of the AI system development life cycle, including protecting infrastructure and models from compromise, threat or loss, developing incident management processes, and responsible release.
- Secure operation and maintenance: Guidelines that apply to the secure operation and maintenance stage of the AI system development life cycle, with guidelines on actions particularly relevant once a system has been deployed, including logging and monitoring, update management and information sharing.
The guidelines follow a ‘secure by default’ approach, and are aligned closely to practices defined in the NCSC’s Secure development and deployment guidance, NIST’s Secure Software Development Framework, and ‘secure by design principles’ published by CISA, the NCSC and international cyber agencies.
They prioritize:
- taking ownership of security outcomes for customers
- embracing radical transparency and accountability
- building organisational structure and leadership so secure by design is a top business priority
Further reading
Guidelines for secure AI system development. Report. November 28, 2023. UK National Cyber Security Centre, FBI Internet Crime Complaint Center (IC3), et al.
Why it matters
Artificial intelligence (AI) systems have the potential to bring many benefits to society. However, for the opportunities of AI to be fully realised, it must be developed, deployed and operated in a secure and responsible way. Cyber security is a necessary precondition for the safety, resilience, privacy, fairness, efficacy and reliability of AI systems.
AI systems are subject to novel security vulnerabilities that need to be considered alongside standard cyber security threats. When the pace of development is high – as is the case with AI – security can often be a secondary consideration. Security must be a core requirement, not just in the development phase, but throughout the life cycle of the system.
Participating agencies:
UK National Cyber Security Centre (NCSC) the US Cybersecurity and Infrastructure Security Agency (CISA), and the following international partners:
- National Security Agency (NSA)
- Federal Bureau of Investigations (FBI)
- Australian Signals Directorate’s Australian Cyber Security Centre (ACSC)
- Canadian Centre for Cyber Security (CCCS)
- New Zealand National Cyber Security Centre (NCSC-NZ)
- Chile’s Government CSIRT
- National Cyber and Information Security Agency of the Czech Republic (NUKIB)
- Information System Authority of Estonia (RIA)
- National Cyber Security Centre of Estonia (NCSC-EE)
- French Cybersecurity Agency (ANSSI)
- Germany’s Federal Office for Information Security (BSI)
- Israeli National Cyber Directorate (INCD)
- Italian National Cybersecurity Agency (ACN)
- Japan’s National center of Incident readiness and Strategy for Cybersecurity (NISC)
- Japan’s Secretariat of Science, Technology and Innovation Policy, Cabinet Office
- Nigeria’s National Information Technology Development Agency (NITDA)
- Norwegian National Cyber Security Centre (NCSC-NO)
- Poland Ministry of Digital Affairs
- Poland’s NASK National Research Institute (NASK)
- Republic of Korea National Intelligence Service (NIS)
- Cyber Security Agency of Singapore (CSA)
