Secure application deployment in the age of continuous delivery

Editor's Notes

• #4: Image: http://morguefile.com/p/209940
• #5: http://www.istockphoto.com/photo/computer-crime-concept-gm516607038-89059287?st=9174601 Source: http://www.verizonenterprise.com/verizon-insights-lab/dbir/2016/ Every year since 2008, Verizon have published a report on the attempted data breaches occurring within their data centers. For 2015, they found close to 90% of them had either a financial or espionage component to them. This report is well worth the read, and there are a few key findings in this report we should all be aware of. Costs of data breaches are heavily skewed towards legal consultation and forensics, and not to the public components of credit monitoring or lawsuits Despite some vulnerabilities having been public for years, there remain vulnerable components in use Some of those components simply may not have a patch forthcoming for a variety of reasons.
• #6: Despite years of organizations spending energy protecting against attacks, it remains up to the attacker to define what’s valuable. Consider the case of ransomware. A police department in the town next to where I live was subjected to a raonsomeware attack. For roughly 500 USD in bitcoin, the attackers would decrypt the booking and evidence records they had just crypto locked. As an attacker, they likely had no knowledge of who they had attacked or what they had locked up. What mattered was the ransom, and that they had a police organization’s files didn’t factor into the equation.
• #7: https://www.cesg.gov.uk/guidance/open-source-software-%E2%80%93-exploring-risk-good-practice-guide-38
• #8: Let’s take a little bit of time and look at how an attack is created. Potential attackers have a number of tools at their disposal, and use a number of different tactics. In this case, the attacker wishes to create an attack on a given component. In order to be effective, they have two primary models. First they can actively contribute code in a highly active area of the component with an objective of planting a back door of some form. The hope being that their code will fail to be recognized as suspect given how quickly the area of code is evolving. Second they can look for areas of code which are stable, and the longer they’ve bene stable, the better. The reason for this is simple, old code is likely written by someone who isn’t with the project any longer, or perhaps doesn’t recall all assumptions present at the time the code was written. After all, its been long understood that even with the best developers, assumptions change and old code doesn’t keep up. The goal in both cases being to create an attack against the component, so they test, and fail, and iterate against the component until they’re successful or move on. Assuming they’re successful, they create a deployment tool and document the tool for others. Of course, given the publicity received by some recent vulnerabilities, a little PR goes a long way. Now there are responsible researchers who follow a similar workflow, and they legitimately attempt to work with component creators to disclose vulnerabilities. They too will publish results, but are less interested in creating the an attack beyond a proof of concept. http://www.istockphoto.com/photo/person-in-hooded-sweater-using-a-laptop-on-wooden-table-gm464503138-58544934?st=cf78f31 http://www.istockphoto.com/photo/cloud-computing-gm518556682-90104967
• #10: https://www.cesg.gov.uk/guidance/open-source-software-%E2%80%93-exploring-risk-good-practice-guide-38 If you’re using commercial software, the vendor is responsible for best practice deployment guidance, the notification of any security vulnerabilities and ultimately patches and workarounds for disclosed vulnerabilities. This is part of the deliverable they provide in return for their license fee. If you’re using open source software, that process becomes partly your responsibility. To illustrate the level of information you have to work with, let’s look at a media-wiki maintenance release from December 2015. “various special pages resulted in fata errors” – this clearly is something which needs resolution, but which pages? How do you test? “1.24.6 marks the end of support for 1.24.x” – this is good to know, but I hope it was published elsewhere. “However, 1.24.5 had issues (along with other versions) so it was thought fair to fix them” – This is a good thing, but can we expect this treatment in the future? From the title, we also have a fix for 1.23.x, but what other versions?
• #13: There is one thing we should all notice from this data: The vulnerable code was present for years until discovery. What may not be known, is that these vulnerabilities were found by researchers, not analysis tools.
• #14: Source: https://info.blackducksoftware.com/OpenSourceSA_LP.html
• #27: Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow On July 13 2015, the bug report associated with what would ultimately become CVE-2015-7547 was created. https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html https://sourceware.org/bugzilla/show_bug.cgi?id=18665 https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT) http://cve.mitre.org/cve/cna.html https://openclipart.org/detail/200681/primary-patch
• #28: Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow The report indicated that a traditional buffer management issue was present. Specifically it said “this change causes the thisanssizp pointer variable used in the recvfrom function on line 1282 to use the wrong size if a new buffer is created after the thisanssizp address has been changed at line 1257” and indicated that the result would be “The program will crash if the calculated size of the buffer used is 0. The recvfrom function will not crash, but any further accesses to the buffer where the bytes read was 0 from the recvfrom function will crash the program. ” https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html https://sourceware.org/bugzilla/show_bug.cgi?id=18665 https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT) http://cve.mitre.org/cve/cna.html https://openclipart.org/detail/200681/primary-patch
• #29: Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow On Feb 16, 2016, a CVE assignment was made to the bug list, and it further indicated the problem was introduced in May of 2008 in version 2.9. https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html https://sourceware.org/bugzilla/show_bug.cgi?id=18665 https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT) http://cve.mitre.org/cve/cna.html https://openclipart.org/detail/200681/primary-patch
• #30: Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow It wasn’t until two days later that the NVD (national vulnerability database) was updated to reflect the vulnerability. This meant that a staggered awareness situation ensued. When disclosures are staggered, there is an increased potential that a bad actor can take advantage of the situation. It’s important to note that this NVD entry came through US-CERT. One important thing to notice is that unlike the bug report, the overview contains significantly more actionable information (e.g. indicating that DNS is impacted “a crafted DNS response that triggers a call to the getaddrinfo function with the AF_UNSPEC or AF_INET6 address family, related to performing "dual A/AAAA DNS queries" and the libnss_dns.so.2 NSS module” https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html https://sourceware.org/bugzilla/show_bug.cgi?id=18665 https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT) http://cve.mitre.org/cve/cna.html https://openclipart.org/detail/200681/primary-patch
• #31: Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow Of course all most data center operators really want is to secure their environment, so the question really becomes “how to do that”. For an indication of how that plays out, we’ll look at VMware’s response as an example. The first thing most vendors do is create some form of security advisory, which in this case is “VMware Knowledge Base article 2144032”. In it they list the then current information about their exposure to the vulnerability. Often times, it’s minimal, but as they investigate, details are fleshed out. On February 22nd, they amended the KB article and created a security advisory “VMware Security Advisory VMSA-2016-0002” which included details on a patch for ESXi 5.5 and then updated a day later for a patch in ESXi 6.0. It’s not the least bit uncommon for some subordinate products, including older versions, to take longer to patch and on March 29th updates were announced to cover “vCenter Server Appliance (VCSA), 5.0 U3f, 5.1 U3c, and 5.5 U3c” Once you’ve a patch, then you can start working on resolving the issue in your infrastructure. https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html https://sourceware.org/bugzilla/show_bug.cgi?id=18665 https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT) http://cve.mitre.org/cve/cna.html https://openclipart.org/detail/200681/primary-patch https://www.youtube.com/watch?v=hkryI6eapOA http://blogs.vmware.com/security/2016/02/vmware-products-and-cve-2015-7547-glibc-getaddrinfo-security-issue.html
• #32: Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow From a timeline perspective, we’re looking at close to eight years from time of bug to vulnerability resolution. For much of that time, the impact was unknown. Starting in July of 2015, the scope of the issue started to be known, but close to seven months were required for investigation, triage, development of a fix and public disclosure to occur. Once the bug was known, the risk of exploitation increased slightly until that disclosure, but following the disclosure risk goes up dramatically. That’s one reason why knowing what’s running in your environment is so important. The overall goal being to reduce the time between disclosure and fix. https://sourceware.org/ml/libc-alpha/2016-02/msg00416.html https://sourceware.org/bugzilla/show_bug.cgi?id=18665 https://security.googleblog.com/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7547 (published via US-CERT) http://cve.mitre.org/cve/cna.html https://openclipart.org/detail/200681/primary-patch https://www.youtube.com/watch?v=hkryI6eapOA
• #33: Source: Future of Open Source 2016 survey: https://www.blackducksoftware.com/2016-future-of-open-source
• #37: http://www.istockphoto.com/photo/strength-in-unity-gm514713440-88219133?st=af7fa36
• #38: Docker Container Security Scanner https://info.blackducksoftware.com/Security-Scan.html 14 Day Free Trial to Black Duck Hub https://info.blackducksoftware.com/Demo.html Red Hat Atomic Host Integration (Requires Black Duck Hub) atomic scan --scanner blackduck [container]