Continuous Security Testing
- 1. © 2016 by RayLai, Confidential and Proprietary© 2016 by RayLai, Confidential and Proprietary Continuous Security Testing Ray Lai Dec 7, 2016 (Version 0.2)
- 2. © 2016 by RayLai, Confidential and Proprietary Synopsis This session addresses the technology challenges of continuous security testing to “deliver securely,” and discusses best practices and tooling based on first hand experience in both enterprise and startup environment.
- 3. © 2016 by RayLai, Confidential and Proprietary The Problem How to complete security testing whenever a build is initiated Develop Security testDeploy Impact US$10,000 / bug fixed US$4M data breach (2016) Damage to brand
- 4. © 2016 by RayLai, Confidential and Proprietary Security Testing: Now and Then People • Professional/consultants Technology • Commercial tools Process • Central governance • Testing before SW release Before/Now Desired State People • Engineering community Technology • Open source tools • UI and API security testing Process • Self-serve • Continuous: at build/deploy time
- 5. © 2016 by RayLai, Confidential and Proprietary How
- 6. © 2016 by RayLai, Confidential and Proprietary What is “Continuous Security Testing”? Continuous Security Testing = New Tools?
- 7. © 2016 by RayLai, Confidential and Proprietary Continuous Security Testing: Point of View App Source Codes Objectives • Efficiency • Feedback loop for developers Security Metrics Automated Security Testing Web App Vulnerability Scanning Threat Modeling Secure Build Build PipelineInput Output Stakeholders • DevOps • Developers • Quality engineers • Security • Static code analysis • Vuln scanning • App assessment • Pen test • Risk assessment • Compliance • Operations review • Defect reports • Threat modeling Challenges:false positives/noise,long build time
- 8. © 2016 by RayLai, Confidential and Proprietary Continuous Security Testing: Paradigm Shift Testing approach • Network vs app security • Front-end vs backend; integrated test Tools • Enterprise (integrated), On- premise Execution • Testing separated from build pipeline Old New Testing approach • Testing how apps and systems are connected Tools • Cloud-based, per pay use • Single-purpose, open source Execution • DevOps: Test in build phase
- 9. © 2016 by RayLai, Confidential and Proprietary How to Deliver Securely? Exercise 1 You have just completed functional and security testing for your flagship app and platform. However, a security researcher informs you that they can reset user password via SMS, and take over a user’s account. Options 1. Extend test coverage (resources) 2. Enforce pen test before release (professionals) 3. Educate engineers (training) How would you do differently to deliver securely?
- 10. © 2016 by RayLai, Confidential and Proprietary How to Deliver Securely? Exercise 2 A security researcher reports a Cross-site Scripting (XSS) issue in your portal, with XSS examples in existing static Web pages. Your dev team argues that XSS on static Web pages is not harmful, and any fix will take them at least 1 month. Options 1. Validate if this is a fire drill (risk management, education) 2. Fix any XSS defect (resources) 3. Enforce pen test before release (professionals) How would you do differently to deliver securely?
- 11. © 2016 by RayLai, Confidential and Proprietary How to Deliver Securely? Exercise 3 Amazon AWS notifies your security team that your standalone test VM (AMI) has been compromised. While your app team argues that it is an isolated instance, AWS finds traces that it was used to access other AMI instances without permission, and also becomes a DDOS agent. Options 1. Introduce security controls and security testing for cloud apps (technology, resources) 2. Enforce pen test before release (professionals) 3. Raise awareness (education) How would you do differently to deliver securely?
- 12. © 2016 by RayLai, Confidential and Proprietary How to Deliver Securely? Recommendation (Past) Continuous Security Testing Exercise 1 (password reset) Threat modeling Test for trusted boundaries (connected security) In build pipeline, spin an app instance to test for end-to-end flow. Exercise 2 (XSS) Resolve any XSS defects Raise awareness Include UI testing for XSS in automated test suite (build process). Exercise 3 (compromised AMI) Cloud security testing – production pen test Cloud security controls In build pipeline, spin an AMI instance to test AWS security policies and controls. This is what I would do differently...
- 13. © 2016 by RayLai, Confidential and Proprietary Recommendation • Continuous security testing is a journey (various maturity levels) – Identify your low hanging fruits and priorities – Building community is an art (e.g. contents, outcome and appropriate engagement) • Be open mind about changes in execution approach – DevOps -> build pipeline for security – Gradual shift from commercial, integrated tools to open source tools • Be prepared for the next big wave – Emerging security challenges in IoT and block chain technologies
- 14. © 2016 by RayLai, Confidential and Proprietary Appendix: Best Practices
- 15. © 2016 by RayLai, Confidential and Proprietary What to Prioritize https://techcrunch.com/2015/01/22/security-for-startups-in-10-steps/ Prioritize Security culture Secure platform Your Website is front-door Secure coding Security controls Plan for failure Physical security Be open with public Email with multi-factor auth Threats
- 16. © 2016 by RayLai, Confidential and Proprietary Which Tools to Use brakerman Static Code Analysis / Vulnerability Scanners Cloud / Infrastructure Security Data SecurityConnected Systems WhatWeb Web Scanner w3af OSSEC Arachni findBugs FlawFinder PMD
- 17. © 2016 by RayLai, Confidential and Proprietary What to Test Major Security Vulnerabilities Static Code Analysis Web App Scan Pen Test OWASP Top 10 Partial Partial Partial HeartBleed-type SSL Issues No No Partial DOS, e.g. due to memory leakage Partial (Memory leakage) Partial Partial DDOS, e.g. bot attack No No Partial Cloud hosting security issues No Limited Yes NoSQL database security issues No No Partial IoT security issues No Varies Varies Blockchain security issues No No Varies
- 18. © 2016 by RayLai, Confidential and Proprietary What to Test (cont’d) OWASP Top 10 Static Code Analysis Web App Scan Pen Test A1 Injection X X X A2 Broken Auth and Session Management X X A3 Cross-site Scripting (XSS) Partial X X A4 Insecure Direct Object References Partial X A5 Security Misconfiguration X X A6 Sensitive Data Exposure Partial X A7 Missing Function Level Access Control X Partial A8 Cross-Site Request Forgery (CSRF) X X A9 Using Components with Known Vulnerabilities Partial A10 Unvalidated Redirects and Forwards Partial X
- 19. © 2016 by RayLai, Confidential and Proprietary How to Execute (Build Pipeline for Security) Web App Scan Deploy to Production Security Assessment Deploy to Production Build Pipeline Penetration Test Runtime Container Security Test Client Web App Scan BeforeNew Separate, serial events Continuous integration process to create temporary containers for security testing
- 20. © 2016 by RayLai, Confidential and Proprietary Security Build Pipeline (Example) App Container Jenkins Server App Container Docker Swarm App Container Security Scanner (Docker) Shared Data Container / Data Volume AppsData Pre-requisites • Apps deployed in docker containers • App data (synthetic or scrubbed production data) in shared data containers • Parallelize testing tasks in Jenkins Jenkins Setup • Create docker swarm (apps and data) • Launch docker swarm • Create security test client (security scanner) • Launch security test client • Post test results in shared data container or data volume • Shut down all docker containers Gauntlt, ClamAV, Brakerman, OWASP ZAP
- 21. © 2016 by RayLai, Confidential and Proprietary What to Look For: Web App Security Testing Session Management • HTTPOnly and SECURE flags (cookies) UI and App • XSS, CSRF Data • Injection vulnerabilities Traditional Approach What’s New API • API token robustness; playback and Man-in-the- middle attacks Data theft • Data privacy issues (in transit or at rest) NoSQL database • Access control issues
- 22. © 2016 by RayLai, Confidential and Proprietary What to Look For: Platform and Cloud Platform Infrastructure • Network security, e.g. port scan, mis-configuration • Default passwords and settings Identity management • SSO; access control Compliance • Security certification, e.g. PCI Traditional Approach What’s New Platform Stack • Timely patches • Core security, e.g. Heartbleed bug • DDOS Data security • Storage security, e.g. S3 Compliance • Public cloud (e.g. AWS) security policies and controls
- 23. © 2016 by RayLai, Confidential and Proprietary How to Measure Success Defect Density Threat Model Custom Risk Scores OWASP Top 10 WhiteHat WSI Security Index Cenzic HARM Scores HP Fortify Scores Internal Metrics Financial management app has a defect density of 15.23 (5M LOC) 25 attack vectors Biz risks: 5.2 Tech risks: 7.9 Tech risks reflect # of total vulnerabilities from scanning/testing Industry Example: For benchmarking
- 24. © 2016 by RayLai, Confidential and Proprietary How Do I Get Trained Traditional Channels What’s New Security Certification • CEH, CISSP, etc. Conferences • RSA, OWASP AppSec, BlackHat, DEFCON, etc. Product training • E.g. Fortify Online Resources for the Impatient • Youtube • Udemy, Coursera, etc. Community • Meetups or user groups, e.g. OWASP