Create Agile confidence for better application security

Editor's Notes

• #4: Last time we talked about network intrusion, risks and vulnerabilities. As part 2 of 3 webinars, today we’ll discuss Agile development methodology, the goals of security testing, and best practices for integrating security and compliance testing in the Agile environment. And finally, we’ll leave you with an action plan.
• #5: The primary principles of Agile development are adaptive methodologies and integrated development teams. In an Agile environment, developers work in short build cycles, or sprints, to focus on a few key features for each release. Feedback from customers (internal or external) and other stakeholders is constant and welcomed. This allows Agile teams to respond quickly to changing requirements, making sure that they deliver the right features and functionality, faster. Traditional development teams are separate groups made of exclusively of developers. Agile teams are more inclusive, and include all stakeholders. Product owners, architects, developers and QA work together to produce a series of deployable releases. Smaller functional releases makes testing quicker, allowing faster feedback and response. Working together ensures that all requirements are known to all parties, and negotiations over functionality and delivery dates can be negotiated quickly, resulting in faster time to market. We run Agile here at Rogue Wave, with our Scrum teams on two-week sprints for all our product development.
• #6: Agile has become popular because it offers several advantages over the waterfall methodologies. Unlike waterfall development, with fixed requirements, longer development cycles, and 1-2 releases each year, Agile development is adaptive, with frequent, functional releases. In Agile, changing requirements are a feature, not a bug. Agile teams are integrated, with product owners, architects, developers and testing professionals working together. In traditional development shops, there is a separation of duties. Security and compliance testing are independent of development. They conduct their tests, analyze the results, and provide development with bug lists, typically late in the development process With shorter release cycles and integrated teams, Agile allows organizations to adapt to changing market needs quickly, and deliver usable software faster. While you may think Agile is best suited for smaller projects, such as websites and GUI, most companies are testing or applying Agile in their most critical – and complex - application development efforts. it is increasingly used in larger, more complicated projects. Surprisingly, we’ve also seen Agile in mixed hardware and software projects.
• #7: So when we examine security as it often happens during development, the functions are separate. In a traditional development environment, security, compliance, and development are separate, autonomous groups. The argument for this is the principal of Separation of Duties – in this case making sure that independence exists between development and security. Development builds a release, tests it for functionality, then passes it to security for testing. Security tools have traditionally been used only by security personnel, typically later in the development lifecycle. Tools include static analysis of source code, dynamic analysis of running applications, and scanning for vulnerable open source components. At each phase of the Secure Development Lifecycle, there are a number of best practices. Security requirements, threat modeling, and several other activities help organizations avoid problems later in the secure software development lifecycle.
• #8: The problem with this, in particular as it applies to an Agile environment, is that the testing happens late in the development process, independent of the development team. For example, Traditional static testing requires a compilable application, complete with all dependencies, usually only possible after significant development efforts.. Dynamic analysis, or pen testing, requires a finished application in a test environment, complete with data. By definition this will be very late in the development process. There have been many studies on the costs associated with delaying security testing, but the numbers remain fairly consistent. The later in the process a bug is identified, the more costly it is to remediate; up to 150X as much as fixing the bug during the requirements or design phase. It’s easy to understand why. Not only is it likely that more code refactoring will be required once an application is “fully baked”, but organizational costs come into play, including triage, prioritization, research, recoding, and retesting to ensure the fix didn’t introduce other problems. In short, costs are higher and releases are delayed. We recently had Jeffrey Hammond of Forrester present at an event, and he said “"It takes 18 months to deploy a new release of a project/product/app where a single line of code is changed.” Finally, we’ve seen this model result in conflicts between security and development teams. The development teams often feel that security has little involvement during the build process, to only parachute in late in the process, run code through their magic boxes, and produce a long list of bugs – with lots of false positives – just when the product is getting to its release date.
• #9: In an Agile environment, release cycles may be measured in days rather than weeks, making testing for security and compliance more challenging. Agile requires frequent testing and rapid, continuous feedback. Shipping code to a separate group for testing, and receiving results days later, will break the Agile model. To be successful in an Agile environment, compliance and security testing and feedback must be integrated with the rest of the Agile team. Note, with Agile, “Release to Market” doesn’t always mean an external release, Potentially shippable increment, or PSI, and minimum viable product (MVP) are two terms used to describe what may or may not be released to customers. When we examine the process, testing is brought in throughout the development lifecycle, rather than waiting until the development is complete. For this reason, testers are typically part of the Agile teams, and testing user stories are built into the backlog from the outset of the sprint, or iteration.
• #10: The first step to deploying an integrated security model is to identify the goals. In an Agile environment, the teams have responsibility for requirements, design, building, and testing applications. Security needs to be part of this, and participate at each phase of the SDL. There is not a single model that works for all Agile teams, so we need to understand the objectives, risk tolerance, and needs of each Agile team Integration with Agile requires receiving information in a timely manner, so teams can react, adjust, and respond while not breaking the schedule Continuous integration and Agile allow teams to improve over time – the same should be true for security practices in Agile Finally, we need to maintain the integrity of the separation of duties principle – an Agile environment does not have to mean no oversight
• #11: Best practices for integrating security and compliance testing in the Agile environment are designed to meet the fundamental requirements of both the Agile methodology and the legitimate requirements of security and compliance groups. These make possible the advantages of Agile development, and state of the art testing tools and methodologies. They include: Integrate security and compliance testing Flexible deployments Context for remediation Continuous improvement Reporting for all stakeholders Let’s look at each of these in depth.
• #12: Best practices for integrating security and compliance testing in the Agile environment are designed to meet the fundamental requirements of both the Agile methodology and the legitimate requirements of security and compliance groups. These make possible the advantages of Agile development, and state of the art testing tools and methodologies. They include: Integrate security and compliance testing Flexible deployments Context for remediation Continuous improvement Reporting for all stakeholders We will now address each practice separately.
• #13: #1 – Integrate security and compliance testing In an Agile environment, the teams have responsibility for requirements, design, building and testing applications. The integrated teams have the authority to determine when all requirements are met. Security should be included in this as well. Rather than outsource security and compliance testing to an outside group, security and compliance need to adapt to the same rules as everyone else, and work within the Agile team. This provides the teams with the responsibility and the tools to deliver software quickly that meets the needs of all stakeholders., and to fix bugs during each sprint. With security personnel working as part of the team, they can test and verify code on a schedule that works best for the team. The relative risk associated with any bug can be discussed and prioritized, and compensating controls can be evaluated for effectiveness without delay. By integrating security testing into the Agile team, faster feedback is both possible and necessary. Identifying critical security bugs earlier allows faster, less costly remediation, and ultimately a better end result delivered in a timely manner.
• #14: Different teams will choose to integrate security and compliance testing in different ways. In the words of the Agile Manifesto, this means “Individuals and interactions over processes and tools” Some will follow the more traditional model, and run tests separately at the end of each sprint. This is the way many groups start out, as it’s simpler to integrate security in this manner. Security personnel run the tests, but do so as part of the Agile team. For groups working towards continuous integration, integrating testing into the build server allows automation of testing for security and compliance. Results are consistent, and those that are considered non-critical can be suppressed in the results so that they need not be revisited with each build. The most cost-effective approach, and one that is getting increased attention from forward-looking teams, is moving testing to the desktop by integrating with the IDE. This provides developers with immediate feedback and the opportunity to self-correct.
• #16: #2 – Enforce standards as they relate to each project To determine the appropriate level of security testing required, we first need to understand the criticality of the application. An application that manages credit cards or healthcare information, which is also web-facing, poses different potential risks to an organization, and therefore requires different scrutiny than an internal tool. Compliance requirements may also come into play. Understand that the level of testing you perform, and the rule sets you deploy, should match those requirements. Once we understand the nature and criticality of the application, we can choose which security and compliance rule sets we use in testing. For example, web applications still exhibit a lot of SQL injection and cross-site scripting errors. Developers also make use of increasing amounts of open source software, some of which (especially older versions) include known security vulnerabilities. OWASP ranks these as three of their top 10 web application vulnerabilities, and PCI Data Security Standards require that you prove that you have tested for the top 10 vulnerabilities at a minimum. The rules you deploy in testing will also vary with the programming language and frameworks you deploy. Buffer overflow rules are important in C and C++ applications, Spring and Struts can require discrete rules to ensure secure usage, and many large organizations use custom frameworks to consistency. Custom frameworks may require custom rules to ensure secure use. In short, make sure you are testing for the types of vulnerabilities that are important to each application.
• #17: Compliance rule sets are increasingly important to many organizations. While some regulatory bodies provide high level requirements, others, such as PCI, are very specific. In most cases, testing for the issues in the OWASP Top 10 or SANS Top 25 will meet these requirements. Specific rule sets OWASP Top 10 SANS Top 25 It is also best to have auditable reports that document your efforts. A report that describes the rules tested and results obtained, over time, provides evidence that you are following the requirements and remediating security issues appropriately.
• #19: #3 – Provide Agile teams with appropriate context for remediating bugs One of the biggest problems development teams have with security testing is the ambiguity of the results. Even putting aside false positives, which can be overwhelming with some testing tools, developers need information on the root cause of the identified vulnerabilities to make changes as efficiently as possible. In an Agile environment, with a focus on speed, this is even more important. Testing tools need to provide context about reported issues to allow faster triage and remediation. The obvious first step is to use tools that allow you to prioritize results based on the severity and type of error. Eliminating informational results and, depending on the environment, low severity results, allows Agile teams to quickly narrow in on bugs that can be fixed in minutes, versus those that require help from architects and security specialists. This context also makes a case for testing at the desktop, as at that point in time, the developer writing code has the most context for addressing bugs and security vulnerabilities. Like any backlog issue, these errors can be triaged and prioritized. For example, a simple SQL injection error might be solved in minutes by changing a concatenated string to a parameterized statement. In contrast, an architectural error, such as improper authentication, may require more planning for a fix. The developers need to know what they should work on first to have the greatest impact on security, and maintain sprint momentum.
• #20: The ability to provide developers with a “root cause” for a reported issue can be invaluable in minimizing code changes, and accelerating remediation. Take the example of input validation errors. Most tools report these when the data is used (at “the sink”, v. “the source”). Since user input may be used in multiple ways, a single input may manifest itself in dozens of sinks, and therefore dozens of reported errors. Each of those “sink issues” can be remediated individually, but this is time consuming. The appropriate remediation for this is to validate the input at the source, as this will automatically correct all of the sink errors. In other words, if you fix the problem in one spot, it will resolve all of the uses of that data. Look for tools that can trace, or track data interprocedurally to identify the root cause, and remediation becomes much more effective.
• #21: Best practices for integrating security and compliance testing in the Agile environment are designed to meet the fundamental requirements of both the Agile methodology and the legitimate requirements of security and compliance groups. These make possible the advantages of Agile development, and state of the art testing tools and methodologies. They include: Integrate security and compliance testing Flexible deployments Context for remediation Continuous improvement Reporting for all stakeholders We will now address each practice separately.
• #22: Best Practice #4 – Continuous Improvement Continuous improvement fits well with the Agile methodology, which is built around brief, repeatable processes. Helping Agile teams avoid, rather than fix security issues should be a high priority. Rather than forcing developers to look up remediation advice on the web, or from internal coding standards, push that information directly to the IDE where it can be easily used. Studies show that people learn through repetition. The graph represents what is known as the “forgetting curve”. The black line in the graph shows memory retention from a single class. In terms of security training, this means that holding a secure coding training event can be helpful, but if the information is not reinforced quickly and consistently, over 90% of the knowledge from that class can vanish with the first week! If brief reminders are provided, as shown by the yellow line in the graph, knowledge retention improves dramatically, until it ultimately becomes part of a student’s long term memory. Pushing security testing and remediation guidance to the IDE also provides developers with near real-time feedback, improving their ability to recognize risky coding structures and self-correct.
• #23: Forcing developers to rely of Google for guidance is not a good practice. Not only are we reliant on the information that is available, and what Google believes to be relevant, you’re at the mercy of how the searcher asked the question. Results and methods will vary, making code maintenance more difficult. Instead, try to standardize coding and remediation standards. Look for tools that provide “best practices” for detecting, avoiding and remediating common bugs for each language you use. Over time, many organizations will customize and augment these to deploy their own coding standards that are specific to the language and frameworks used by the development teams. By pushing this to the IDE, organizations can accelerate remediation and provide continuous developer education.
• #24: Best practices for integrating security and compliance testing in the Agile environment are designed to meet the fundamental requirements of both the Agile methodology and the legitimate requirements of security and compliance groups. These make possible the advantages of Agile development, and state of the art testing tools and methodologies. They include: Integrate security and compliance testing Flexible deployments Context for remediation Continuous improvement Reporting for all stakeholders We will now address each practice separately.
• #25: Best Practice #5 – Enterprise Reporting While we advocate that Agile teams “own” testing for security and compliance, the principle of separation of duties must be honored. Separation of Duties provides checks and balances required for audits and provides other stakeholders with the information they require to do their jobs. When moving to integrated testing, remember the different “views” of the data required by other functional areas. Development will require roll-up reports to identify trends by type of bug, team, and perhaps individual users. This helps track the effectiveness of the teams, and help to pinpoint ongoing training needs. Security will require reporting to understand and quantify the risks associated with each application, and determine when intervention is required. Compliance needs information to submit with regulatory audits, and provide evidence that appropriate testing has taken place, and… Legal/Governance must ensure compliance with various open source licensing models.
• #26: Getting started with integrated testing isn’t hard, but also requires planning to ensure a smooth process Provide Agile teams with the tools and responsibility for security and compliance testing, and include members from these disciplines in the Agile team. This will allow you to align everyone’s priorities and needs before you begin implementation. Start small and think big. Select a project for your pilot and determine the security and compliance standards for it. Make sure your process is flexible enough to accommodate critical security projects as well as non-critical ones. Consider the best way to integrate the testing activities. As stated, many start with integration into the build server, but integrating at the desktop provides additional efficiency and contribute to continuous improvement Build or purchase coding and remediation standards, and put them in a location that is easily accessed by developers, ideally in the IDE so workflow is not interrupted. This accelerates bug fixes and minimizes the risk of “development via Google searches” Finally, Close the loop – make sure all stakeholders can access the information they require independently. This addresses the separation of duties principal and simplifies internal and external auditing.
• #27: This concludes the presentation, and we hope you found it helpful. For more information on Rogue Wave Klocwork, and its IDE-based security testing and remediation technology, a free trial is available at klocwork.com/freetrial For more information on open source security and compliance testing, please visit openlogic.com Thank you