Justin collins - Practical Static Analysis for continuous application delivery
1 like288 views
- The document discusses using static analysis for continuous application security. It provides examples of how to write custom static analysis rules and checks for various programming languages like Ruby and JavaScript to detect unsafe calls like delete_survey without a user ID parameter. - It demonstrates how to parse code into an abstract syntax tree using libraries like Esprima and RubyParser, then walk the trees to find problematic code patterns. Existing tools like Bandit and Brakeman are also shown for writing custom rules and checks. - The key steps are to start small by identifying a single security issue, tailor the solution to your environment, and automate enforcement through continuous integration or other automated processes.
![grep/ack
grep -R "delete_survey([^,)]+)" *
ack "delete_survey([^,)]+)" --type=python
28](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-28-320.jpg)
![Bash
git checkout $@ --quiet
files=$(git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2)
grep "delete_survey([^,)]+)" $files
30](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-30-320.jpg)
![Bash
git checkout $@ --quiet
files=$(git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2)
grep "delete_survey([^,)]+)" $files
32](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-32-320.jpg)
![Create a Rule
class CheckSurvey < Rule
def run(file_name, code)
if code =~ /delete_survey([^,)]+)/
warn file_name, "delete_survey without user ID"
end
end
end
35](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-35-320.jpg)
![Base Rule Class
class Rule
@rules = []
@warnings = []
def self.inherited klass
@rules << klass
end
def self.run_rules files
files.each do |name|
code = File.read(name)
@rules.each do |r|
r.new.run(name, code)
end
end
end
def self.warnings
@warnings
end
def warn file_name, msg
Rule.warnings << [file_name, msg]
end
end 36](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-36-320.jpg)
![Code to Run It
system "git checkout #{ARGV[0]}"
files = `git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2`
Rule.run_rules(files.split)
Rule.warnings.each do |file_name, message|
puts "#{message} in #{file_name}"
end
if Rule.warnings.any?
exit 1
end
37](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-37-320.jpg)
![False Negatives
delete_survey(input.split(",")[0])
40](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-40-320.jpg)
![Python (Astroid)
Module()
body = [
Discard()
value =
CallFunc()
func =
Name(delete_survey)
args = [
Name(survey_id)
]
starargs =
kwargs =
]
AstroidBuilder().string_build("delete_survey(survey_id)"
)
45](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-45-320.jpg)
![JavaScript (Esprima)
{
"type": "Program",
"body": [
{
"type": "ExpressionStatement",
"expression": {
"type": "CallExpression",
"callee": {
"type": "Identifier",
"name": "delete_survey"
},
"arguments": [
{
"type": "Identifier",
"name": "survey_id"
}
]
}
}
],
"sourceType": "script"
}
esprima.parse("delete_survey(survey_id)")
46](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-46-320.jpg)
![Brakeman Custom Check
require 'brakeman/checks/base_check'
class Brakeman::CheckDeleteSurvey < Brakeman::BaseCheck
Brakeman::Checks.add self
Brakeman::WarningCodes::Codes[:get_survey] = 2001
@description = "Finds delete_survey calls without a user_id"
def run_check
@tracker.find_call(target: false, method: :delete_survey).each do |result|
next if duplicate? result
add_result result
if result[:call].second_arg.nil?
warn :result => result,
:warning_type => "Direct Object Reference",
:warning_code => :get_survey,
:message => "Use of get_survey without user_id",
:confidence => CONFIDENCE[:high]
end
end
end
end 52](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-52-320.jpg)
![JavaScript (Esprima)
{
"type": "Program",
"body": [
{
"type": "ExpressionStatement",
"expression": {
"type": "CallExpression",
"callee": {
"type": "Identifier",
"name": "delete_survey"
},
"arguments": [
{
"type": "Identifier",
"name": "survey_id"
}
]
}
}
],
"sourceType": "script"
}
esprima.parse("delete_survey(survey_id)")
57](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-57-320.jpg)
![Walking Esprima AST
var check_get_survey = function(ast) {
if(ast.type == "CallExpression" &&
ast.callee.type == "Identifier" &&
ast.callee.name == "delete_survey" &&
ast.arguments.length == 1) {
console.log("delete_survey called without user_id at line ",
ast.loc.start)
}
}
var walk = function(ast) {
if(Array.isArray(ast))
{
ast.forEach(walk);
}
else if(ast.type) {
check_get_survey(ast)
for (key in ast) {
walk(ast[key])
}
}
}
var esprima = require('esprima');
walk(esprima.parse('delete_survey(survey_id)', { loc: true }))
58](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-58-320.jpg)
![Walking RubyParser AST
require 'ruby_parser'
require 'sexp_processor'
class FindGetSurvey < SexpInterpreter
def process_call exp
if exp[1].nil? and
exp[2] == :delete_survey and
exp[4].nil?
puts "delete_survey called without user_id at line #{exp.line}"
end
end
end
ast = RubyParser.new.parse "delete_survey(survey_id)"
FindGetSurvey.new.process ast
61](https://image.slidesharecdn.com/justincollins-devseccon2016-161031170716/85/Justin-collins-Practical-Static-Analysis-for-continuous-application-delivery-61-320.jpg)
Justin collins - Practical Static Analysis for continuous application delivery
- 1. Join the conversation #devseccon By Justin Collins (@presidentbeef) Practical Static Analysis for Continuous Application Security
- 3. @presidentbeef Static analysis security tool for Ruby on Rails 3
- 5. Continuous Security Automated, always-on processes for detecting potential security vulnerabilities 5
- 6. Static Analysis Information determined about a program without actually running the code 6
- 7. Practical Static Analysis Things you can reasonably implement after this workshop 7
- 8. Why Static Analysis? Fast Run anytime Easy to automate Points directly at suspect code 8
- 9. Tool Cycle Run Tool Wait Interpret Results Fix Issues Repeat 9
- 11. 1 - Identify a Problem Repeated incidents with the same root cause Opt-in security, e.g. in APIs Unsafe calls/settings no one should use, ever 11
- 12. 2 - Identify a Solution Write a safer library Make security opt-out Detect unsafe calls/settings/API usage 12
- 13. 3 - Enforce the Solution Write tests Use static analysis Use dynamic analysis 13
- 14. 4 - Automate Enforcement Part of continuous integration Part of code review Standalone, continuously-running process Local scripts/hooks 14
- 17. Code Review 17
- 23. 1 - Identify a Problem delete_survey(survey_id) 23
- 24. 2 - Identify a Solution delete_survey(survey_id, user_id) 24
- 25. 3 - Enforce the Solution Use static analysis to check for use 25
- 27. Regular Expressions • grep • ack • ag • …or build your own 27
- 28. grep/ack grep -R "delete_survey([^,)]+)" * ack "delete_survey([^,)]+)" --type=python 28
- 30. Bash git checkout $@ --quiet files=$(git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2) grep "delete_survey([^,)]+)" $files 30
- 31. git diff --name-status A README.md M lib/blah.py M lib/a.rb M lib/version.rb D test/new_stuff.yml 31
- 32. Bash git checkout $@ --quiet files=$(git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2) grep "delete_survey([^,)]+)" $files 32
- 34. Changed Files Rules file1 - warning X file2 - warning Y file3 - warning X ... Multiple Rules 34
- 35. Create a Rule class CheckSurvey < Rule def run(file_name, code) if code =~ /delete_survey([^,)]+)/ warn file_name, "delete_survey without user ID" end end end 35
- 36. Base Rule Class class Rule @rules = [] @warnings = [] def self.inherited klass @rules << klass end def self.run_rules files files.each do |name| code = File.read(name) @rules.each do |r| r.new.run(name, code) end end end def self.warnings @warnings end def warn file_name, msg Rule.warnings << [file_name, msg] end end 36
- 37. Code to Run It system "git checkout #{ARGV[0]}" files = `git diff --name-status master HEAD | grep -E "^(A|M)" | cut -f 2` Rule.run_rules(files.split) Rule.warnings.each do |file_name, message| puts "#{message} in #{file_name}" end if Rule.warnings.any? exit 1 end 37
- 39. False Positives # Remember not to use delete_survey(survey_id)! function delete_survey(node) {...} 39
- 43. S-Expressions (call nil "delete_survey" (local "survey_id")) delete_survey(survey_id) 43
- 44. Ruby (RubyParser) s(:call, nil, :delete_survey, s(:call, nil, :survey_id)) RubyParser.new.parse("delete_survey(survey_id") 44
- 45. Python (Astroid) Module() body = [ Discard() value = CallFunc() func = Name(delete_survey) args = [ Name(survey_id) ] starargs = kwargs = ] AstroidBuilder().string_build("delete_survey(survey_id)" ) 45
- 46. JavaScript (Esprima) { "type": "Program", "body": [ { "type": "ExpressionStatement", "expression": { "type": "CallExpression", "callee": { "type": "Identifier", "name": "delete_survey" }, "arguments": [ { "type": "Identifier", "name": "survey_id" } ] } } ], "sourceType": "script" } esprima.parse("delete_survey(survey_id)") 46
- 49. Bandit Custom Rule import bandit from bandit.core import test_properties as test @test.checks('Call') @test.test_id('B350') def unsafe_get_survey(context): if (context.call_function_name_qual == 'delete_survey' and context.call_args_count < 2): return bandit.Issue( severity=bandit.HIGH, confidence=bandit.HIGH, text="Use of delete_survey without user ID." ) 49
- 51. brakeman.org 51
- 52. Brakeman Custom Check require 'brakeman/checks/base_check' class Brakeman::CheckDeleteSurvey < Brakeman::BaseCheck Brakeman::Checks.add self Brakeman::WarningCodes::Codes[:get_survey] = 2001 @description = "Finds delete_survey calls without a user_id" def run_check @tracker.find_call(target: false, method: :delete_survey).each do |result| next if duplicate? result add_result result if result[:call].second_arg.nil? warn :result => result, :warning_type => "Direct Object Reference", :warning_code => :get_survey, :message => "Use of get_survey without user_id", :confidence => CONFIDENCE[:high] end end end end 52
- 53. Brakeman Custom Check brakeman --add-checks-path custom_checks/ 53
- 57. JavaScript (Esprima) { "type": "Program", "body": [ { "type": "ExpressionStatement", "expression": { "type": "CallExpression", "callee": { "type": "Identifier", "name": "delete_survey" }, "arguments": [ { "type": "Identifier", "name": "survey_id" } ] } } ], "sourceType": "script" } esprima.parse("delete_survey(survey_id)") 57
- 58. Walking Esprima AST var check_get_survey = function(ast) { if(ast.type == "CallExpression" && ast.callee.type == "Identifier" && ast.callee.name == "delete_survey" && ast.arguments.length == 1) { console.log("delete_survey called without user_id at line ", ast.loc.start) } } var walk = function(ast) { if(Array.isArray(ast)) { ast.forEach(walk); } else if(ast.type) { check_get_survey(ast) for (key in ast) { walk(ast[key]) } } } var esprima = require('esprima'); walk(esprima.parse('delete_survey(survey_id)', { loc: true })) 58
- 60. Ruby (RubyParser) s(:call, nil, :delete_survey, s(:call, nil, :survey_id)) RubyParser.new.parse("get_survey(survey_id") 60
- 61. Walking RubyParser AST require 'ruby_parser' require 'sexp_processor' class FindGetSurvey < SexpInterpreter def process_call exp if exp[1].nil? and exp[2] == :delete_survey and exp[4].nil? puts "delete_survey called without user_id at line #{exp.line}" end end end ast = RubyParser.new.parse "delete_survey(survey_id)" FindGetSurvey.new.process ast 61
- 62. Summary Start small: identify single issue to solve Tailor solution to your environment Automate enforcement 62
- 63. Join the conversation #devseccon Thank You Source Code: https://github.com/presidentbeef/devseccon