Making big data small
- 1. MAKING BIG DATA SMALL BARON SCHWARTZ STRATA CONFERENCE NYC 2013
- 2. ME @XAPRB ON TWITTER [email protected] LINKEDIN.COM/IN/XAPRB Optimization, Backups, Replication, and more 3rd Edition Covers Version 5.5 High Performance MySQL Baron Schwartz, Peter Zaitsev & Vadim Tkachenko
- 4. STORAGE TRANSFER / BANDWIDTH COMPUTE (CPU, RAM, I/O) TIME
- 5. DATA LIFECYCLE GATHER TRANSFER STORE RETRIEVE COMPUTE
- 6. THE MEGABYTE HOUR WHAT DOES 350 MILLISECONDS COST?
- 9. DO MORE WITH LESS
- 11. GENERATE KNOWLEDGE DATA INFORMATION KNOWLEDGE
- 12. SUCH KNOWLEDGE IS TOO WONDERFUL FOR ME; IT IS HIGH, I CANNOT ATTAIN IT. - PSALM 139
- 14. DOING MORE WITH LESS
- 17. DATA LIFECYCLE GATHER TRANSFER STORE RETRIEVE COMPUTE
- 18. THE RISK?
- 19. LESS DATA LESS COMPUTE TECHNIQUES LOSSINESS FEATURES
- 20. LESS DATA LESS COMPUTE TECHNIQUES LOSSINESS FEATURES
- 21. DECREASE REDUNDANCY LESS DATA
- 22. DELTA COMPRESSION LESS DATA
- 23. NULLS AND ZEROES LESS DATA
- 26. DISCARDING DUPLICATE SIGNALS LESS DATA
- 28. LESS DATA LESS COMPUTE TECHNIQUES LOSSINESS FEATURES
- 29. BLOOM FILTERS LESS COMPUTE
- 30. EXPONENTIALLY WEIGHTED MOVING AVERAGES LESS COMPUTE 400 300 200 100 0 1 2 3 4 5 400 300 200 100 0 1 2 3 4 5
- 32. LESS DATA LESS COMPUTE TECHNIQUES LOSSINESS FEATURES
- 33. MODELS LOSSINESS
- 34. LOSSINESS
- 35. LOSSINESS
- 36. LESS DATA LESS COMPUTE TECHNIQUES LOSSINESS FEATURES
- 40. SPARSE OUTLIER STORAGE FEATURES
- 41. LOST HAYSTACKS
- 42. WHAT ABOUT THE PIONEER ANOMALY?
- 43. SMALL DATA METHOD GATHER EXPLORE ISOLATE DISCARD APPLY SAMPLES VISUALIZE NEEDLES! HAYSTACK REVIEW
- 45. CONTACT / QUESTIONS @XAPRB ON TWITTER [email protected] LINKEDIN.COM/IN/XAPRB PLEASE RATE THIS TALK :-)
- 46. IMAGE CREDITS HTTP://WWW.FLICKR.COM/PHOTOS/PIPER/458847242/ HTTP://WWW.FLICKR.COM/PHOTOS/MIONG/4222013704/ HTTP://WWW.FLICKR.COM/PHOTOS/BENSTEVINSON/4908627827/ HTTP://EN.WIKIPEDIA.ORG/WIKI/FILE:BLOOM_FILTER.SVG HTTP://WWW.FLICKR.COM/PHOTOS/JINGLESLENOBEL/4915172257/ HTTP://WWW.FLICKR.COM/PHOTOS/MAUROPPP/429231880/ HTTP://WWW.FLICKR.COM/PHOTOS/ECSTATICIST/3979554514/ HTTP://WWW.FLICKR.COM/PHOTOS/TAMBAKO/9158311021/ HTTP://WWW.FLICKR.COM/PHOTOS/FUZZYLITTLEMANPEACH/4689679368/ HTTP://WWW.FLICKR.COM/PHOTOS/FWP/49514206/ HTTP://WWW.FLICKR.COM/PHOTOS/QTHOMASBOWER/3511960429/ HTTP://WWW.FLICKR.COM/PHOTOS/JIMNISTA/7239478006/ HTTP://WWW.FLICKR.COM/PHOTOS/FLOWERS-OF-THE-SEA/6971586987/ HTTP://WWW.FLICKR.COM/PHOTOS/OLDPATTERNS/5817264314/ HTTP://WWW.FLICKR.COM/PHOTOS/BREWBOOKS/184343329/ HTTP://WWW.FLICKR.COM/PHOTOS/PAREEERICA/4235666733/ HTTP://WWW.FLICKR.COM/PHOTOS/JDHANCOCK/5177358991/ HTTP://WWW.FLICKR.COM/PHOTOS/ROSEMARY/47396922/ HTTP://WWW.FLICKR.COM/PHOTOS/ECSTATICIST/1337749333/ HTTP://WWW.FLICKR.COM/PHOTOS/PINKSHERBET/233228813/ HTTP://BOST.OCKS.ORG/MIKE/SIMPLIFY/ HTTP://WWW.FLICKR.COM/PHOTOS/PSOUP216/7540279198/ HTTPS://GITHUB.COM/BRENDANGREGG/HEATMAP/WIKI HTTP://WWW.FLICKR.COM/PHOTOS/WALKINGSF/6221430560/ HTTP://WWW.FLICKR.COM/PHOTOS/WALKINGSF/7835295762/ HTTP://WWW.FLICKR.COM/PHOTOS/NASAMARSHALL/8615696355/ HTTP://WWW.FLICKR.COM/PHOTOS/BOOLEANSPLIT/8482641188/ HTTP://WWW.FLICKR.COM/PHOTOS/HTAKASHI/6861028238/ HTTP://WWW.FLICKR.COM/PHOTOS/ZDETO/4383899069/ JPL VIA HTTP://WWW.PLANETARY.ORG/EXPLORE/PROJECTS/PIONEER-ANOMALY-1/ PIONEER_ANOMALY_STAIRS_DATA_TAPES_3.HTML HTTP://EN.WIKIPEDIA.ORG/WIKI/FILE:PIONEER_10-11_SPACECRAFT.JPG