![3. Stay green - Tools
<Q>: WTF??? On Chromium startup:
==25254== ERROR: AddressSanitizer: global-buffer-overflow on address
0x000011d3dde5 at pc 0x5ab21a bp 0x7fff00659450 sp 0x7fff00659428
READ of size 10 at 0x000011d3dde5 thread T0
#0 0x5ab219 in __interceptor_memcmp _asan_rtl_
#1 0xa1edc08 in fillInUnixFile .../sqlite/amalgamation/sqlite3.c:28654
#2 0xa1efe7c in unixOpen .../sqlite/amalgamation/sqlite3.c:29294
<A>:
Diff of /trunk/tools/build/scripts/slave/runtest.py:
+ # Avoid aggressive memcmp checks until http://crbug.com/178677 is fixed.
+ os.environ['ASAN_OPTIONS'] = 'strict_memcmp=0'
Hunting for living](https://image.slidesharecdn.com/44con-slides-140527194519-phpapp01/85/44-con-slides-20-320.jpg)
![Bad access information:
+0x000 alloc_stack : [62] 0x0f999970 Void
+0x0f8 alloc_stack_size : 0x3c '<'
+0x0fc alloc_tid : 0x14a8
+0x100 free_stack : [62] (null)
+0x1f8 free_stack_size : 0 ''
+0x1fc free_tid : 0
+0x200 error_type : 3 ( HEAP_BUFFER_OVERFLOW )
+0x204 access_mode : 0 ( ASAN_READ_ACCESS )
+0x208 access_size : 8
+0x20c shadow_info : [128] "06499E3F is 23 bytes
beyond 384-byte block [06499CA8,06499E28)."
+0x290 microseconds_since_free : 0
SyzyASan](https://image.slidesharecdn.com/44con-slides-140527194519-phpapp01/85/44-con-slides-28-320.jpg)
![ExceptionAddress: 564a0cd7 (chrome_..!WebCore::
WaveShaperDSPKernel::lazyInitializeOversampling+0x0...06)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000000
Parameter[1]: 27261fe4
Attempt to read from address 27261fe4
.
STACK_TEXT:
chrome_...!WebCore::WaveShaperDSPKernel::
lazyInitializeOversampling+0x6 [...
webkitsourcemoduleswebaudiowaveshaperdspkernel.cpp @ 53]
chrome_...!WebCore::WaveShaperProcessor::setOversample+0x29
.
APPLICATION_FAULT_INVALID_POINTER_READ_chrome!WebCore::
WaveShaperDSPKernel::lazyInitializeOversampling+6
Dump-analysis](https://image.slidesharecdn.com/44con-slides-140527194519-phpapp01/85/44-con-slides-31-320.jpg)
44 con slides
- 1. Browser Bug Hunting Memoirs of a last man standing Atte Kettunen (@attekett) OUSPG https://code.google.com/p/ouspg/
- 3. Started at OUSPG in summer 2011 First security bug from Chrome 2011-12 Since then ~100 Vulns ~60 Rewards 39 CVEs Atte Kettunen
- 4. Mozilla since 2004 - Sec-High/Critical $3,000 Google since 2010 - Typical security bugs $1,000-$3,133.7 - Possibility for bonus rewards ● PoC, exploit, awesomeness (Microsoft 2013 June 25 - July 25) Browser Bug Bounty Programs
- 5. Easy to get started - Lots of bugs o/ Helpful vendor security teams and supportive responses to first bug submissions Supportive (secretive/competitive) community of other bounty hunters Browser Bug Bounty Programs
- 6. ● Use-after-free ○ DOM ○ CSS ○ Rendering ● Buffer-overflow ○ Media formats ○ Parsers ○ Decoders ○ Coordinates Where the bugs are
- 7. AddressSanitizer - global-buffer-overflow - READ of size 2 #0 nsCharTraits<unsigned short>::length() #1 nsAString_internal::Assign() . Repro-file: <link rel="stylesheet" href="data:text/css;charset=utf-16, p#two%1%7Bbackground-color%65535A%3B%7D% 0D%0A"/> Some bug - CVE-2012-4185 - Firefox
- 8. AddressSanitizer - heap-use-after-free - READ of size 2 #0 WebCore::nextBreakablePosition() #1 ...::RenderBlock::LineBreaker::nextLineBreak() . Repro-file: <html><body> <ruby> <q style="column-gap:2;">a </ruby> <cite style="word-break: break-all;">a <q style="text-transform:uppercase;">a <sup style="text-overflow:ellipsis;"> </body></html> Some bug - Regression - Chrome
- 9. ==3213== ERROR: AddressSanitizer heap-buffer-overflow on address 0x7f50cd6ffcf8 at pc 0x7f50dd159dde bp 0x7fff3e0accd0 sp 0x7fff3e0accc8 READ of size 2 at 0x7f50cd6ffcf8 thread T0 #0 0x7f.de in WebCore::CSSParser::lex(void*) ???:0 #1 0x7f.78 in cssyyparse(void*) ???:0 #2 0x7f.40 in WebCore::CSSParser::parseDeclaration() . Repro-file: <a style=top:-1px> Some bug - Regression - Chrome
- 10. Three golden rules: Hunting for living
- 11. Three golden rules: 1. Stay green Hunting for living
- 12. Three golden rules: 1. Stay green - Features Hunting for living
- 13. Three golden rules: 1. Stay green - Features 2. Stay green - Competition Hunting for living
- 14. Three golden rules: 1. Stay green - Features 2. Stay green - Competition 3. Stay green - Tools Hunting for living
- 15. 1. Stay green - Features ● New features are published all the time ○ New code o/ ● Some changes are not highlighted ○ Minor updates to JavaScript API support etc. ● Old bugs fixed ○ New code o/ ● Old features can change ○ Prefixes disappear(-webkit,-moz), ○ Features can get disabled Hunting for living
- 16. 1. Stay Green - Features ● Firefox Aurora - Release note: "Partial support for Web Audio, targeted at web developers for testing" (May 17, 2013) Hunting for living
- 17. 2. Stay green - Competition ● Tools ○ Different approach -> Different bugs? ● Targets ○ Find new minefields ● Platforms ○ Different code on different platforms Hunting for living
- 18. 2. Stay green - Competition @cevans: "@j00ru has melted polar ice with his PDF fuzzing on 9k cores." Hunting for living
- 19. 3. Stay green - Tools ● Instrumentations ○ New instrumentation -> detect new issues ● Build environments ○ Broken builds @#!¤#... ● Fuzzers ○ New techniques Hunting for living
- 20. 3. Stay green - Tools <Q>: WTF??? On Chromium startup: ==25254== ERROR: AddressSanitizer: global-buffer-overflow on address 0x000011d3dde5 at pc 0x5ab21a bp 0x7fff00659450 sp 0x7fff00659428 READ of size 10 at 0x000011d3dde5 thread T0 #0 0x5ab219 in __interceptor_memcmp _asan_rtl_ #1 0xa1edc08 in fillInUnixFile .../sqlite/amalgamation/sqlite3.c:28654 #2 0xa1efe7c in unixOpen .../sqlite/amalgamation/sqlite3.c:29294 <A>: Diff of /trunk/tools/build/scripts/slave/runtest.py: + # Avoid aggressive memcmp checks until http://crbug.com/178677 is fixed. + os.environ['ASAN_OPTIONS'] = 'strict_memcmp=0' Hunting for living
- 21. ● Instrumentation ● Fuzzers ● Hardware/Infrastructure Tools
- 22. ● Clang compiler plugin ● Adds instrumentation to check memory access at runtime ● Similar to Valgrind ● Only 2x slowdown ● Created at Google ● Used by Google & Mozilla ● Linux & OS X ● http://www.chromium.org/developers/testing/addresssanitizer AddressSanitizer
- 23. ● Awesome with use-after-frees ● Very good for buffer-overflows and out of bounds access ● Good but confused with type confusions AddressSanitizer
- 24. ==6==ERROR: AddressSanitizer: heap-use-after-free on address 0x6070000268d0 at pc 0x7f845771029f bp 0x7fff...2a0 sp 0x7fffc7eea298 READ of size 8 at 0x6070000268d0 thread T0 (chrome) #0 0x7f845771029e (... /asan-linux-release-209136/chrome+0x96f229e) #1 0x7f84576aacea (... /asan-linux-release-209136/chrome+0x968ccea) #2 0x7f8451ce00f3 (... /asan-linux-release-209136/chrome+0x3cc20f3) . 0x6070000268d0 is located 64 bytes inside of 72-byte region [0x607000026890,0x6070000268d8) freed by thread T19 (AudioOutputDevi) here: #0 0x7f844f58e101 (... /asan-linux-release-209136/chrome+0x1570101) #1 0x7f845887b5ec (... /asan-linux-release-209136/chrome+0xa85d5ec) . AddressSanitizer
- 25. ==6==ERROR: AddressSanitizer: heap-use-after-free on address 0x6070000268d0 at pc 0x7f845771029f bp 0x7fff...2a0 sp 0x7fffc7eea298 READ of size 8 at 0x6070000268d0 thread T0 (chrome) #0 0x7f845771029e in WebCore::WaveShaperDSPKernel:: lazyInitializeOversampling(...) .../WebKit/Source/wtf/OwnPtr.h:138 #1 0x7.a in WebCore::WaveShaperProcessor::setOversample(...) ... /WebKit/Source/modules/webaudio/WaveShaperProcessor.cpp:70 . 0x6070000268d0 is located 64 bytes inside of 72-byte region [0x607000026890,0x6070000268d8) freed by thread T19 (AudioOutputDevi) here: #0 0x7.1 in operator delete(void*) _asan_rtl_ #1 0x7.c in WebCore::AudioDSPKernelProcessor::uninitialize() src/third_party/WebKit/Source/wtf/OwnPtrCommon.h:47 . AddressSanitizer
- 26. ● Used to instrument binaries ● Redirects heap-related calls to own run- time library ● Currently only heap-instrumentation ● Chrome/Chromium only atm. ● About 3x Slowdown ● Windows only ● https://code.google.com/p/sawbuck/wiki/SyzyASanDesignDocument SyzyASan
- 27. SyzyASAN error: heap-buffer-overflow on address 0x0379D1A7 (stack_id=0x44CB69D7) READ of size 8 at 0x0379D000 #0 0x000068ef23be in (unknown) #1 0x000068f387f4 in (unknown) #2 0x000068eeb486 in (unknown) #3 0x000068e8add7 in (unknown) . . . SyzyASan
- 28. Bad access information: +0x000 alloc_stack : [62] 0x0f999970 Void +0x0f8 alloc_stack_size : 0x3c '<' +0x0fc alloc_tid : 0x14a8 +0x100 free_stack : [62] (null) +0x1f8 free_stack_size : 0 '' +0x1fc free_tid : 0 +0x200 error_type : 3 ( HEAP_BUFFER_OVERFLOW ) +0x204 access_mode : 0 ( ASAN_READ_ACCESS ) +0x208 access_size : 8 +0x20c shadow_info : [128] "06499E3F is 23 bytes beyond 384-byte block [06499CA8,06499E28)." +0x290 microseconds_since_free : 0 SyzyASan
- 30. ● Heap allocation monitoring for Windows ● No feedback - Only crash :( ● “Works” on Chrome/Chromium ● env: CHROME_ALLOCATOR="winheap" ● Enable Chrome error reporting -> minidumps ● Firewall Chrome( No free 0-days for Google ;) ) ● Debugging tools x86 Page-Heap
- 31. ExceptionAddress: 564a0cd7 (chrome_..!WebCore:: WaveShaperDSPKernel::lazyInitializeOversampling+0x0...06) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000000 NumberParameters: 2 Parameter[0]: 00000000 Parameter[1]: 27261fe4 Attempt to read from address 27261fe4 . STACK_TEXT: chrome_...!WebCore::WaveShaperDSPKernel:: lazyInitializeOversampling+0x6 [... webkitsourcemoduleswebaudiowaveshaperdspkernel.cpp @ 53] chrome_...!WebCore::WaveShaperProcessor::setOversample+0x29 . APPLICATION_FAULT_INVALID_POINTER_READ_chrome!WebCore:: WaveShaperDSPKernel::lazyInitializeOversampling+6 Dump-analysis
- 32. ● Dumb fuzzing ○ Yes, still works ○ Yes, you can still find bugs with bit-flipping of image-files ● Smart fuzzing ○ Finds bugs fast but runs out of bugs faster. :( Fuzzers
- 33. Dumb fuzzing ● Radamsa || Surku o/ ○ https://code.google.com/p/ouspg/ ● Mutate old repros ( find ./src/ -type d -name *crashtest* | xargs ls; ) ● Collect winnings Fuzzers
- 34. Smart fuzzing ● W3C/MDN(/MSDN) ● Again stay green ● Most of the JavaScript APIs in browsers are really similar ● Some of the public tools have the logic in them already ● W3C spec + grep + sed = $$$ Fuzzers
- 35. Smart fuzzing WebAudio API - PannerNode - Specification interface PannerNode : AudioNode { void setPosition(double x, double y, double z); void setOrientation(double x, double y, double z); void setVelocity(double x, double y, double z); attribute double refDistance; attribute double maxDistance; attribute double rolloffFactor; }; Fuzzers
- 36. Smart fuzzing 2D Canvas API - Specification // rects void clearRect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h); void fillRect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h); // shadows attribute unrestricted double shadowOffsetX; // (default 0) attribute unrestricted double shadowOffsetY; // (default 0) attribute unrestricted double shadowBlur; // (default 0) Fuzzers
- 37. Individuals: ● Physical machines -> sysadmining <3 ● SSD <3 ● RAM++ ● Vicious cycle of Bug->Bounty->New HW->Bug->Bounty... Hardware/Infrastructure
- 38. 8x Dual Core CPU, 2GB ram, USB-stick, aka. Badgers OUSPG - 2011
- 39. University HW: Badgers 6x Quad core AMD A10, 16GB ram, SSD 6x Dual Dual Core AMD antique, 8GB, 10k rpm BYOD: 4x Quad core i7-3770K, 16GB ram, SSD And additional 30+ cores misc hardware with 133.7+ GB of ram and bunch of SSDs OUSPG - 2013
- 40. OUSPG - 2013
- 42. ClusterFuzz aka. CF ● Google fuzzing cluster ● 2012 - ○ 6000 Chrome instances ○ 50m+ test cases per day ○ Plans for quadrupling at that time ● ASAN, multiple fuzzers, minimization, regression ranges, verify fixes, dupes & dupes & dupes... Hardware/Infrastructure
- 43. “cluster-fuzz is a soulless bug hunting machine. It has no want or need for your gratitude. It lives only to feed on bugs.” ClusterFuzz
- 44. ● 12 machines running 24/7 ● ~50 cores, ~133.7GB of RAM ● approx. 20m test cases per day ● 19 file-formats ● git, scp, auto-update, auto-minimize ● Radamsa and ... My stuff
- 45. ● Browser fuzzer harness ● Written in JavaScript ( Node.js ) ● Linux, Windows, OS X ● Test case generators and instrumentations loaded as modules ● Uses WebSockets for test case injection to browser ● Stable - https://code.google.com/p/ouspg/downloads/list ● Trunkish - https://github.com/attekett/NodeFuzz NodeFuzz
- 46. Requirements: Google Chrome installed $ sudo apt-get install nodejs $ git clone https://github.com/attekett/NodeFuzz.git $ cd NodeFuzz $ npm install $ vim config.js #Optional $ node nodefuzz.js NodeFuzz - Setup - Ubuntu
- 47. ● Fairly new JS API (Chrome 2011, FF 2013) ● "The API has been designed to allow modular routing.(UAF) Basic audio operations are performed by audio nodes that are linked together to form an audio routing graphs.(UAF/BOF) Inside a same context, several sources are supported, with different kind of channel layout.(UAF/BOF) This modular design allows for great flexibility and for the creation of complex audio functions and of dynamic effects. (BOF)" - MDN NodeFuzz - module - WebAudio
- 48. Bugs found: ● Chrome - 4 UAF, 3 BOF ● Firefox - 1 UAF, 8 BOF NodeFuzz - module - WebAudio
- 49. CVE-2013-0879 - Chrome - BOF <script> try{var context= new webkitAudioContext()}catch(e){} try{var oscillator= context.createOscillator()}catch(e){} try{oscillator.start(0.701,0.7,0.7)}catch(e){} setInterval(function(){ try{oscillator.connect(context.destination);}catch(e){} },4) try{oscillator.stop(0.70)}catch(e){} </script> NodeFuzz - module - WebAudio
- 50. CVE-2013-2845 - Chrome - UAF <script> var Context0= new webkitAudioContext() var Analyser0=Context0.createAnalyser(); var WaveShaper0=Context0.createWaveShaper(); var Convolver3=Context0.createConvolver(); Analyser0.connect(WaveShaper0); WaveShaper0.connect(Context0.destination); Convolver3.connect(Analyser0); setInterval(function(){ Analyser0.disconnect(); },4) </script> NodeFuzz - module - WebAudio
- 51. DEMO!!! && Q&A