RISELab:Enabling Intelligent Real-Time Decisions
- 1. RISELab: Enabling Intelligent Real-time Decisions Ion Stoica February 8, 2017
- 2. Berkeley’s AMPLab (2011-2016) 2 Algorithms Machines People Goal: Nextgeneration of open source data analytics stack for industry & academia BerkeleyDataAnalytics Stack (BDAS)
- 3. Berkeley’s AMPLab (2011-2016) 3 Algorithms Machines People Goal: Nextgeneration of open source data analytics stack for industry & academia BerkeleyDataAnalytics Stack (BDAS) …
- 4. RISE: Real-time Intelligent Secure Execution
- 5. From batch data to advanced analytics AMPLab 5 From live data to real-time decisions RISELab
- 6. RISE Lab (2017-2022) 12 faculty across AI, systems, security, and architectures 11 Founding sponsors 6
- 7. Why? Data only as valuable as the decisions it enables 7
- 8. Why? What does this mean? • Faster decisions better than slower decisions • Decisions on fresh data better than decisions on stale data • Decisions on personalized data better than on aggregate data 8 Data only as valuable as the decisions it enables
- 9. Goal Real-time decisions on live data with strong security 9 decide in ms the current stateof theenvironment privacy, confidentiality, integrity
- 10. Typical decision system Decision System Query Decision Environment + sensors & actuators Observations, Feedback Preprocess Intermediate data Decision Engine
- 11. Decision Query Decision Engine Preprocess Intermediate data Environment + sensors & actuators Typical decision system Decision System Observations, Feedback Live Update latency (e.g., ~1 seconds)
- 12. Decision Query Decision Engine Preprocess Intermediate data Environment + sensors & actuators Typical decision system Decision System Observations, Feedback Live Update latency (e.g., ~1 seconds) Secure Real-time decision latency (e.g., ~10 ms)
- 13. Example of decision systems Decision System Query Decision Training Models (diff.tradeoffs complexity/ accuracy) Model Serving Feedback Observations, Feedback ML Pipeline (e.g., Clipper + Spark/Tensorflow) Decision System Obs. Action Update Policy Policy obs à action Query Policy Observations, Rewards Reinforcement Learning Systems (e.g., Ray) Pre- process Interm- ediate Data Decision Engine
- 14. What else do we want from decisions? Intelligent:complexdecisions in uncertain environments Robust: handle complexnoise, unforeseeninputs, failures Explainable:ability to explainnon-obvious decisions
- 15. Goal Develop open source platforms, tools, and algorithms for intelligent real-time decisions on live-data
- 16. Some Proposed Research Secure Real-time Decisions Stack (SRDS) • Open source platform to develop of RISE apps • Secure from ground up • Reinforcement Learning (RL) as one of key app patterns Learning control hierarchies: speeduplearning, training Shared learning: learn over confidential data 16
- 17. Secure Real-time Decisions Stack (SRDS) • Open source platform to develop of RISE apps • Secure from ground up • Reinforcement Learning (RL) as one of key app patterns Learning control hierarchies: speeduplearning, training Shared learning: learn over confidential data 17 Some Proposed Research
- 18. Secure Real-time Decision Stack (SRDS) scheduler object store RISE μkernel Ray Clipper … Ground(data contextservice) Time Machine
- 19. scheduler object store RISE μkernel Ray Clipper … Ground(data contextservice) Time Machine Minimalist executionengine: • Support both data flow and task-parallel execution models • High-throughput, low-latency: ~ 1M tasks/sec @ ms latency Secure Real-time Decision Stack (SRDS)
- 20. scheduler object store RISE μkernel Ray Clipper … Ground(data contextservice) Time Machine Central repositoryfor models,APIs to capture the context in whichdata gets used and produced Status:ongoing project with industry partners Secure Real-time Decision Stack (SRDS)
- 21. scheduler object store RISE μkernel Ray Clipper … Ground(data contextservice) Time Machine Replaying of apps at fine granularity • Simplify development, debugging • Robustness: replay against perturbed inputs • Explainability: identify inputs causing decision • Security: confirm vulnerabilities, test security patches, compliance auditing Secure Real-time Decision Stack (SRDS)
- 22. scheduler object store RISE μkernel Ray Clipper … Ground(data contextservice) Time Machine Dramatically simplify development of RISE applications • Apache Spark: improve latency and security • Clipper: model serving for Apache Spark, Scikit learn, etc • Ray: framework for RL applications Secure Real-time Decision Stack (SRDS)
- 23. ImprovingApache Spark Drizzle • Decrease latency of Structured Streaming and ML algorithms by ~10x • Techniques: group scheduling, shared variables 23
- 24. StreamingLatency:YCSB benchmark 24 0 500 1000 1500 2000 1 2 4 8 12 16 20 24 MedianEventLatency (ms) Throughput (Million events/s) Spark Drizzle Flink Drizzle-Opt Drizzle-Opt: Reduce-by on mapper side
- 25. StreamingLatency:YCSB benchmark 25 0 500 1000 1500 2000 1 2 4 8 12 16 20 24 MedianEventLatency (ms) Throughput (Million events/s) Spark Drizzle Flink Drizzle-Opt Drizzle-Opt: Reduce-by on mapper side
- 26. StreamingLatency:YCSB benchmark 26 0 500 1000 1500 2000 1 2 4 8 12 16 20 24 MedianEventLatency (ms) Throughput (Million events/s) Spark Drizzle Flink Drizzle-Opt Drizzle-Opt: Reduce-by on mapper side
- 27. StreamingLatency:YCSB benchmark 27 0 500 1000 1500 2000 1 2 4 8 12 16 20 24 MedianEventLatency (ms) Throughput (Million events/s) Spark Drizzle Flink Drizzle-Opt Drizzle-Opt: Reduce-by on mapper side
- 28. StreamingLatency:YCSB benchmark 28 0 500 1000 1500 2000 1 2 4 8 12 16 20 24 MedianEventLatency (ms) Throughput (Million events/s) Spark Drizzle Flink Drizzle-Opt Drizzle-Opt: Reduce-by on mapper side 15x
- 29. MLlib: SGD Performance 29 0 20 40 60 4 8 16 32 64 128 Time/iter(ms) Machines Spark Drizzle
- 30. MLlib: SGD Performance 30 0 20 40 60 4 8 16 32 64 128 Time/iter(ms) Machines Spark Drizzle 6x
- 31. ImprovingApache Spark Drizzle • Decrease latency of Structured Streaming and ML algorithms by ~10x • Techniques: group scheduling, shared variables • Some of these techniques will make their way to Apache Spark Opaque • Full data encryption, authentication, and verification (Intel’s SGX) • Oblivious mode: hide data access pattern • Support most SparkSQL functionality • See Wenting’s talk later 31
- 32. RISELab Already promising results Expect muchmore over the next five years! 32 Goal: Develop open source platforms,tools, and algorithms for intelligent real-timedecisions on live- data
- 33. Thank you