Lessons Learned from Developing Secure AI Workflows.pdf
0 likes203 views
Lessons Learned from Developing Secure AI Workflows
Lessons Learned from Developing Secure AI Workflows.pdf
- 1. SESSION ID: #RSAC Elie Bursztein Lessons learned from developing secure AI workflows Google @elie SAT-W09
- 2. © 2024 RSA Conference LLC or its affiliates. The RSA Conference logo and other trademarks are proprietary. All rights reserved. Disclaimer Presentations are intended for educational purposes only and do not replace independent professional judgment. Statements of fact and opinions expressed are those of the presenters individually and, unless expressly stated to the contrary, are not the opinion or position of RSA Conference™ or any other co-sponsors. RSA Conference does not endorse or approve, and assumes no responsibility for, the content, accuracy or completeness of the information presented. Attendees should note that sessions may be audio- or video-recorded and may be published in various media, including print, audio and video formats without further notice. The presentation template and any media capture are subject to copyright protection.
- 3. 3
- 5. Model weaponization Model Backdoor Prompt Subversion PII leaks Infrastructure vulnerability Hallucinations Excessive agency Biases
- 6. SAIF site SAIF Secure AI framework
- 7. Today: Explore AI system components risks and controls with concrete examples
- 8. The solutions explored in this talk are product agnostic - use your favorite systems and models
- 9. Application AI system tour map Infrastructure Data Model Governance and Assurance
- 11. Data
- 12. Data Securely collect, store, and manage the data used by models for training, fine-tuning and retrieval purposes
- 13. Data Modules Data Sources Data Filtering and Processing External Sources Data sources Data filtering and processing
- 14. Data poisoning Data Modules Data Sources Data Filtering and Processing External Sources
- 15. Many products include user reporting flows that can be abused Gmail Gemini
- 16. Gmail manual reporting false flags AI-specific risks
- 17. Data Modules Data Sources Data Filtering and Processing External Sources Training data sanitization
- 18. Perform data validation using anomaly detection and supervised classifiers Controls
- 19. Data Modules Data Sources Data Filtering and Processing External Sources Data & model poisoning
- 20. Data Modules Data Sources Data Filtering and Processing Training data sanitization Training data management User data management
- 21. Prevent unauthorized data access via encryption at rest and access control Controls
- 22. Infrastructure
- 23. Securely train, fine-tune, and serve AI models
- 24. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Framework code Model and Data Storage Training, Tuning and Evaluation Model serving Model and Data Storage Model Serving
- 25. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Model source tampering
- 26. https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/ Payload Types distribution 60% 40% 20% 0% Potential Object Hijack Arbitrary Code Execution Pickle Deserialization Pingback Software Opening File Write Reverse Shell Hugging Face model files backdoored Classic risks
- 28. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Unauthorized training data
- 30. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Model poisoning
- 31. Backdoor model code to get remote access Example of layer acting as backdoor that can be added at anypoint https://splint.gitbook.io/cyberblog/security-research/tensorflow-remote-code-execution-with-malicious-model Classic risks
- 32. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Secure-by-default ML tooling
- 33. Controls Implement verifiable model provenance using cryptography https://github.com/google/model-transparency
- 34. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Model exfiltration
- 35. Bearer Token exposure & loss Classic risks
- 37. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Model and data access control
- 38. Ensure that model & data access requires authentication and API keys are stored as secrets Controls
- 39. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Model exfiltration Model deployment tampering Model poisoning Unauthorized training data Model source tampering
- 40. Infrastructure Modules Training, Tuning and Evaluation Model and Frameworks Code Model and Data Storage Model Serving Model and data access control Secure-by-default ML tooling Security as default not as optional Adversarial training and testing Adversarial training and testing Privacy enhancing technologies
- 41. Models
- 42. Safely process user’s inputs and model’s outputs
- 43. Model Modules Model Output Handling Model input handling Model Model output handling Model Input Handling Model
- 44. Model Modules Model Output Handling Model Input Handling Model Unsafe model output
- 45. Un-sanitized output lead to arbitrary code execution https://github.com/advisories/GHSA-fprp-p869-w6q2 Classic risks
- 46. Model Modules Model Output Handling Model Input Handling Model Adversarial training and testing
- 47. Organize red team exercises to test model safety & security Controls
- 48. Model Modules Model Output Handling Model Input Handling Model Prompt injection
- 49. Invisible UTF-8 characters are not escaped https://twitter.com/rez0__/status/1745545813512663203 Classic risks
- 50. Model Modules Model Output Handling Model Input Handling Model Input validation and sanitization Output validation and sanitization
- 51. Implement dedicated input & output security classifiers and code sanitizers https://github.com/google/model-transparency Controls
- 52. Model Modules Model Output Handling Model Input Handling Model Sensitive data disclosure
- 53. Privacy enhancing technologies Infrastructure Modules Model and Data Storage Model Serving Training, Tuning and Evaluation Model and Frameworks Code Model Modules Model and Data Storage Model Serving Training, Tuning and Evaluation
- 54. Differential privacy training to ensure the model doesn’t learn and recall PII https://openreview.net/pdf?id=Q42f0dfjECO Controls
- 55. Model Modules Model Output Handling Model Input Handling Model Sensitive data disclosure Prompt injection Unsafe model output Model evasion Inferred sensitive data
- 56. Model Modules Model Output Handling Model Input Handling Model Input validation and sanitization Adversarial training and testing Output validation and sanitization
- 57. Applications
- 58. Securely integrate models into complex applications
- 59. Application Modules Application Model Plugin Applications Model plugins Users External Sources
- 60. Application Modules Application Model Plugin Users External Sources Unauthorized model action Insecure integrated component
- 61. Un-sanitized plugins output lead to data exfiltration https://embracethered.com/blog/posts/2023/chatgpt-cross-plugin-request-forgery-and-prompt-injection./ Classic risks
- 62. Application Modules Application Model Plugin Users External Sources Application access management Model plugin user control
- 63. User consent and controls in Gemini
- 64. Application Modules Application Model Plugin Users External Sources Denial of ML service
- 66. Implement DDOS mitigation techniques including rate limiting https://openreview.net/pdf?id=Q42f0dfjECO Controls
- 67. Application Modules Application Model Plugin Users External Sources Unauthorized model action Denial of ML service Model reverse engineering Insecure integrated component
- 68. Application Modules Application Model Plugin Users External Sources Adversarial training and testing User consents and controls Model plugin permissions Model plugin user control Application access management
- 70. Ensure that AI systems operate securely, ethically, and are in compliance throughout their entire lifecycle
- 71. Application Modules Application Model Plugin Users External Sources Insecure integrated component
- 72. Application Modules Application Model Plugin Data Modules Data Sources Users External Sources Model Modules Model Input Handling Model Model Output Handling System Modules Model and Frameworks Code Model and Data Storage Model Serving Data Filtering and Processing Training, Tuning and Evaluation Insecure code
- 74. Code review Application Modules Application Model Plugin Data Modules Data Sources Users External Sources Model Modules Model Input Handling Model Model Output Handling System Modules Model and Frameworks Code Model and Data Storage Model Serving Data Filtering and Processing Training, Tuning and Evaluation
- 75. Require code review to reduce security bugs introduction and mitigate insider risk code tampering Controls
- 76. https://security.googleblog.com/2023/10/googles-reward-criteria-for-reporting.htm l Establish a bug bounty to help test your AI systems https://www.landh.tech/blog/20240304-google-hack-50000/ Controls
- 77. Securing AI requires implementation of controls across the stack Implementation of classical controls and AI specific, novel defense is critical to secure AI workflows AI Risks are a combination of classical issues and novel AI specific threats Takeaways
- 78. Improve security by adding additional controls Review your AI workflows risk and controls to understand your posture Apply Today In the next 6 month
- 79. Today AI workflows security Thanks for attending Previously at RSA AI cybersecurity capabilities Get the slides online