Event Stream Processing with Multiple Threads
1 like327 views
The document discusses event stream processing using multiple threads. It presents the problem of processing an input event stream through a computation P to produce an output event stream as a single task. The solution involves splitting the computation P into parts that can be processed in parallel by multiple threads to improve efficiency. The system is based on connecting simple computing units called processors into a pipeline. New thread-aware processors are introduced, including ones for non-blocking pushing of events, pull pipelines, and preemptive pulling. These allow existing queries to gain multi-threading capabilities with minimal changes. Experimental results on sample queries show speedups from 5% to 400% by adding just one thread-aware processor.
![Experimental Results
Turn around
→
A
/a/b
//character[status=Walker]/id/text()
→ p1
→
A
→ p2
→ →
→
→
/a/b
//character[status=Blocker]/id/text()
→
→
→
3
→
→
→
<?
→
→
→→
→
→
→
→
f1
f2
→
→ →
→
→
→
→](https://image.slidesharecdn.com/all-170915182810/85/Event-Stream-Processing-with-Multiple-Threads-88-320.jpg)
![M4
Multi-threaded
Experimental Results
Turn around
→
A
/a/b
//character[status=Walker]/id/text()
→ p1
→
A
→ p2
→ →
→
→
/a/b
//character[status=Blocker]/id/text()
→
→
→
3
→
→
→
<?
→
→
→→
→
→
→
→
f1
f2
→
→ →
→
→
→
→](https://image.slidesharecdn.com/all-170915182810/85/Event-Stream-Processing-with-Multiple-Threads-89-320.jpg)
Event Stream Processing with Multiple Threads
- 1. MULTIPLEMULTIPLEMULTIPLEMU THREADSTHREADSTHREADSTHR MULTIPLE THREADS EVENT STREAM PROCESSING with Sylvain Hallé Raphael Khoury Sébastien Gaboury Université du Québec a Chicoutimi
- 2. The Problem
- 3. input event stream The Problem
- 4. P computation input event stream The Problem
- 5. output event stream P computation input event stream The Problem
- 6. Find ways to split P into parts that can be processed in parallel 1 output event stream P computation input event stream The Problem
- 7. genericFind ways to split P into parts that can be processed in parallel 1 output event stream P computation input event stream The Problem
- 8. Input events arrive one by one2 genericFind ways to split P into parts that can be processed in parallel 1 output event stream P computation input event stream The Problem
- 9. Input events arrive one by one2 generic Output events are produced one by one3 Find ways to split P into parts that can be processed in parallel 1 output event stream P computation input event stream The Problem
- 10. The Problem
- 13. The System
- 14. The System
- 15. Based on the composition (piping) of simple computing units called processors The System
- 16. The System
- 17. → Fork f = new Fork(2); The System
- 18. → → → → Fork f = new Fork(2); FunctionProcessor sum = new FunctionProcessor( ); f The System
- 19. → → → + → Fork f = new Fork(2); FunctionProcessor sum = new FunctionProcessor(Addition instance); f The System
- 20. → → → + → Fork f = new Fork(2); FunctionProcessor sum = new FunctionProcessor(Addition.instance); CountDecimate decimate = new CountDecimate(n); f n The System
- 21. → → → + → → Fork f = new Fork(2); FunctionProcessor sum = new FunctionProcessor(Addition.instance); CountDecimate decimate = new CountDecimate(n); Connector.connect(fork, LEFT, sum, LEFT) f n The System
- 22. → → → + → → → Fork f = new Fork(2); FunctionProcessor sum = new FunctionProcessor(Addition.instance); CountDecimate decimate = new CountDecimate(n); Connector.connect(fork, LEFT, sum, LEFT) . connect(fork, RIGHT, decimate, INPUT) f n The System
- 23. → → → + → → → Fork f = new Fork(2); FunctionProcessor sum = new FunctionProcessor(Addition.instance); CountDecimate decimate = new CountDecimate(n); Connector.connect(fork, LEFT, sum, LEFT) . connect(fork, RIGHT, decimate, INPUT) . connect(decimate, OUTPUT, sum, RIGHT); f n The System
- 24. → → → + → → → Fork f = new Fork(2); FunctionProcessor sum = new FunctionProcessor(Addition.instance); CountDecimate decimate = new CountDecimate(n); Connector.connect(fork, LEFT, sum, LEFT) . connect(fork, RIGHT, decimate, INPUT) . connect(decimate, OUTPUT, sum, RIGHT); Pullable p = sum.getOutputPullable(OUTPUT); while (p.hasNext() != NextStatus.NO) { Object o = p.next(); . .. } f n The System
- 25. The System Function Applies a function to every input event Cumulative Computes the progressive "sum" of all input events Trim Removes the first n input events Decimate Outputs every n-th event Group Encloses a group of connected processors Fork Duplicates a stream into multiple copies Slice Splits a stram into multiple sub-streams Window Applies a function to a sliding window of n events f Σ.f n Filter A first, Boolean stream, decides if each event of a second stream should be output { f
- 26. The System X G F U 1 < Σ f ⊥ < Σ f ⊥ F G = = = = Σ f ? ♣ LTL PALETTE
- 27. Existing processors should be changed ! 4 not P P PPP P PPP The Problem
- 28. Existing processors should be changed ! 4 not P P PPP P PPP Enclose them within new, processors Solution: thread-aware P The Problem
- 29. Thread Manager Responsible for a pool of N threads M
- 30. Runnable r = ... ManagedThread t = m.tryNewThread(r); if (t != null) t.start(); else r.run(); Returns a new thread that can be started, or null if all N threads are busy If t is null, the desired processing must be done in the current thread Thread Manager Responsible for a pool of N threads M
- 45. ψ φ.pull φ W W.pull ψ.pull Pre-emptive Pulling TIME
- 46. ψ φ.pull φ W W.pull ψ.pull Pre-emptive Pulling TIME
- 62. ψ push(e1) φ.pull φ2φ1 φe1 e2 e3 S φ.pull push(e2) e'1 e'1 e'2 φ2 push(e3) e'3 Pull Pipeline TIME
- 63. ψ push(e1) φ.pull φ2φ1 φe1 e2 e3 S φ.pull push(e2) e'1 e'1 e'2 φ2 push(e3) e'3 Pull Pipeline TIME
- 70. Window ψ push(e) φ1.push(e) φn-1.push(e) . . . φ2.push(e) Blocking Push TIME
- 71. Window ψ push(e) φ1.push(e) φn-1.push(e) . . . collect e' shift copy φ2.push(e) Blocking Push TIME
- 72. Window ψ push(e) φ1.push(e) φn-1.push(e) . . . collect e' shift copy push(e') φ2.push(e) Blocking Push TIME
- 73. Window ψ push(e) φ1.push(e) φn-1.push(e) . . . collect e' shift copy push(e') φ2.push(e) Event e is pushed to the n copies of φ sequentially. Each call to push is blocking. Blocking Push TIME
- 76. Window ψ push(e) φ0.push(e) φ0 Non-blocking Push TIME
- 77. Window ψ push(e) φ0.push(e) φ1.push(e) φ0 φ1 Non-blocking Push TIME
- 78. Window ψ push(e) φ0.push(e) φn-1.push(e) φ1.push(e) φ0 φ1 φn-1. . . Non-blocking Push TIME
- 79. Window ψ push(e) φ0.push(e) φn-1.push(e) φ1.push(e) φ0 φ0.waitFor φ1 φn-1. . . Non-blocking Push TIME
- 80. Window ψ push(e) φ0.push(e) φn-1.push(e) φ1.push(e) φ0 φ0.waitFor φn-1.waitFor φ1 φn-1. . . Non-blocking Push TIME
- 81. Window ψ push(e) φ0.push(e) φn-1.push(e) collect e' shift copy φ1.push(e) φ0 φ0.waitFor φn-1.waitFor φ1 φn-1. . . Non-blocking Push TIME
- 82. Window ψ push(e) φ0.push(e) φn-1.push(e) collect e' shift copy push(e') φ1.push(e) φ0 φ0.waitFor φn-1.waitFor φ1 φn-1. . . Non-blocking Push TIME
- 83. Window ψ push(e) φ0.push(e) φn-1.push(e) collect e' shift copy push(e') φ1.push(e) φ0 φ0.waitFor φn-1.waitFor φ1 φn-1. . . Calls to push are non-blocking; each runs in a separate thread. Non-blocking Push TIME
- 84. 3 new thread-aware processors Non-blocking push Pull pipeline Pre-emptive pull Implemented in a separate independent from the rest of the code palette → → NP → PP → PE
- 85. Multi-threading in a Query → → { f → → + Σ.f 10 FunctionProcessor sum = new FunctionProcessor( new CumulativeFunction(Addition.instance)); WindowProcessor win = new WindowProcessor(sum, 10);
- 86. ThreadManager m = new ThreadManager(4); FunctionProcessor sum = new FunctionProcessor( new CumulativeFunction(Addition.instance)); NonBlockingProcessor nbp = new NonBlockingProcessor(sum, m); WindowProcessor win = new WindowProcessor(nbp, 10); → → { f 10 Multi-threading in a Query M→ → → → + Σ.f 4
- 87. Experimental Results Auction bidding Candidate selection Endless bashing Spontaneous Pingu creation Turn around 1.07 4.44 1.45 1.05 1.86 Benchmark (CRV 2016) Speedup A single thread-aware processor inserted in the query
- 88. Experimental Results Turn around → A /a/b //character[status=Walker]/id/text() → p1 → A → p2 → → → → /a/b //character[status=Blocker]/id/text() → → → 3 → → → <? → → →→ → → → → f1 f2 → → → → → → →
- 89. M4 Multi-threaded Experimental Results Turn around → A /a/b //character[status=Walker]/id/text() → p1 → A → p2 → → → → /a/b //character[status=Blocker]/id/text() → → → 3 → → → <? → → →→ → → → → f1 f2 → → → → → → →
- 90. Take-home Points Multi-threading capabilities added to BeepBeep 3 Existing processors do not need to be aware of the presence of threads Minimally intrusive: less than 5 lines to add multi- threading to part of a query 5% - 400% speedup on a small sample of queries Requires manual tuning and intuition Not all queries can gain from parallelism!
- 91. https://liflab.github.io/beepbeep-3 THE END THE END ...but for how long ? https://www.researchgate.net/publication/318337325