Lock-free algorithms for Kotlin Coroutines
15 likes2,695 views
The document discusses lock-free algorithms for Kotlin coroutines. It covers the implementation of a lock-free doubly linked list using single-word compare-and-swap operations. It also discusses how to build more complex atomic operations, like a multi-word compare-and-swap, to enable select expressions in Kotlin coroutines.
![Doubly linked list: remove (0)
S
N
1
N
P
S
P
H T
R1
CAS here
2
N
P
Operation Descriptor
A ref: ???
expectA: ???
updateA: Rem[???]
…
Outcome: UNDECIDED
Both not known in advance
Deterministic f(expectA)](https://image.slidesharecdn.com/2017sptcc-lock-freealgorithmsforkotlincoroutines-170705192032/85/Lock-free-algorithms-for-Kotlin-Coroutines-119-320.jpg)
![Doubly linked list: remove (1)
S
N
1
N
P
S
P
H T
R1
2
N
P
Operation Descriptor
A ref: ???
expectA: ???
updateA: Rem[???]
…
Outcome: UNDECIDED
DCSS Descriptor
affected node: #1
operation ref
old value: #2](https://image.slidesharecdn.com/2017sptcc-lock-freealgorithmsforkotlincoroutines-170705192032/85/Lock-free-algorithms-for-Kotlin-Coroutines-120-320.jpg)
![Doubly linked list: remove (2)
S
N
1
N
P
S
P
H T
R1
2
N
P
Operation Descriptor
A ref: ???
expectA: ???
updateA: Rem[???]
…
Outcome: UNDECIDED
It locks what node we are to removeCannot change w/o removal of #1
We don’t support PushLeft!!!
DCSS Descriptor
affected node: #1
operation ref
old value: #2](https://image.slidesharecdn.com/2017sptcc-lock-freealgorithmsforkotlincoroutines-170705192032/85/Lock-free-algorithms-for-Kotlin-Coroutines-121-320.jpg)
![Doubly linked list: remove (3)
S
N
1
N
P
S
P
H T
R1
2
N
P
Operation Descriptor
A ref: Node #1
expectA: Node #2
updateA: Rem[#2]
…
Outcome: UNDECIDED
desc is updated after successful DCSS
DCSS Descriptor
affected node: #1
operation ref
old value: #2](https://image.slidesharecdn.com/2017sptcc-lock-freealgorithmsforkotlincoroutines-170705192032/85/Lock-free-algorithms-for-Kotlin-Coroutines-122-320.jpg)
![Doubly linked list: remove (4)
S
N
1
N
P
S
P
H T
R1
2
N
P
Operation Descriptor
A ref: Node #1
expectA: Node #2
updateA: Rem[#2]
…
Outcome: UNDECIDED
Stays pointed until operation
outcome is decided](https://image.slidesharecdn.com/2017sptcc-lock-freealgorithmsforkotlincoroutines-170705192032/85/Lock-free-algorithms-for-Kotlin-Coroutines-123-320.jpg)
Lock-free algorithms for Kotlin Coroutines
- 2. Speaker: Roman Elizarov • 16+ years experience • Previously developed high-perf trading software @ Devexperts • Teach concurrent & distributed programming @ St. Petersburg ITMO University • Chief judge @ Northeastern European Region of ACM ICPC • Now work on Kotlin @ JetBrains
- 3. Agenda • Kotlin coroutines overview & motivation for lock- free algorithms • Lock-free doubly linked list • Lock-free multi-word compare-and-swap • Combining them to get more complex atomic operations (without STM)
- 4. Kotlin basic facts • Kotlin is a JVM language developed by JetBrains • General purpose and statically-typed • Object-oriented and functional paradigms • Open source under Apache 2.0 • Reached version 1.0 in 2016 • Compatibility commitment • Now at version 1.1 • Officially supported by Google on Android
- 5. Kotlin is … • Modern • Concise • Safe • Extensible • Pragmatic • Fun to work with!
- 9. Blocking threads Kotlin fun postItem(item: Item) { val token = requestToken() val post = submitPost(token, item) processPost(post) }
- 10. fun postItem(item: Item) { requestToken { token -> submitPost(token, item) { post -> processPost(post) } } } Callbacks Kotlin
- 11. fun postItem(item: Item) { requestToken() .thenCompose { token -> submitPost(token, item) } .thenAccept { post -> processPost(post) } } Futures/Promises/Rx Kotlin
- 12. Coroutines Kotlin fun postItem(item: Item) { launch(CommonPool) { val token = requestToken() val post = submitPost(token, item) processPost(post) } }
- 13. CSP & Actor models • A style of programming for modern systems • Lots of concurrent tasks / jobs • Waiting most of the time • Communicating all the time Share data by communicating
- 14. Kotlin coroutines primitives • Jobs/Deferreds (futures) • join/await • Channels • send & receive • synchronous & buffered channels • Select/alternatives • Atomically wait on multiple events • Cancellation • Parent-child hierarchies
- 15. Implementation challenges • Coroutines are like light-weight threads • All the low-level scheduling & communication mechanisms have to scale to lots of coroutines
- 17. Building blocks • Single-word CAS (that’s all we have on JVM) • Automatic memory management (GC) • Practical lock-free algorithms • Lock-Free and Practical Doubly Linked List-Based Deques Using Single-Word Compare-and-Swap by Sundell and Tsigas • A Practical Multi-Word Compare-and-Swap Operation by Timothy L. Harris, Keir Fraser and Ian A. Pratt.
- 18. Doubly linked list S N 1 N P S P H T sentinel sentinel Use same node in practice next links form logical list contents prev links are auxiliary
- 24. Doubly linked list (remove 1) S N 1 N P S P H T Mark removed node’s next link Use wrapper object for mark in practice Cache wrappers in pointed-to nodes CAS Retry remove on CAS failure 1 2 Don’t use AtomicMarkableReference
- 30. Helping
- 36. Concurrent insert (4) S N 1 N P S P H T 2 N P 3 N P reinit & repeat I2 I3
- 49. Concurrent remove & insert (3) S N 1 N P S P H T 2 N P detect wrong prev (t.prev.next -- removed) do “correct prev”R1 R1 I2
- 56. Concurrent remove & insert (2) S N 1 N P S P H T 2 N P R1 R1 I2 will succeed marking on remove retry
- 60. Takeaways • A kind of algo you need a paper for • Hard to improve w/o writing another paper • Good news: stress tests uncover most impl bugs • Bad news: when stress test fails, you up to long hours • More bad news: hard to find bugs that violate lock- freedomness of algorithm
- 61. Summary: what we can do • Insert items (at the end of the queue) • Remove items (at the front of the queue) • Traverse the list • Remove items at arbitrary locations • In O(1)
- 62. Linearizability • Insert last • Linearizes at CAS of next • Remove first / arbitrary • Success – at CAS of next • Fail – at read of head.next
- 63. More about algorithm • Sundell & Tsigas algo supports deque operations • Can PushLeft & PopRight • PopLeft is simple – read head.next & remove • But cannot linearize them all at cas points • PushLeft, PushRight, PopRight - Ok • PopLeft linearizes at head.next read (!!!)
- 64. Summary of impl notes • Use GC (drop all memory management details) • Merge head & tail into a single sentinel node • Empty list is just one object (prev & next onto itself) • One item += one object • Reuse “remove mark” objects • One-element lists reuse of ptrs to sentinel all the time • Encapsulate! S N P 1 N P Q
- 67. Basic mods (1) • Insert item conditionally on prev tail value S N 1 N P S P H T 2 N P check & bailout before CAS
- 68. Basic mods (2) • Remove head conditionally on prev head value S N 1 N P S P H T R1 check & bailout before CAS
- 69. Practical use-case: synchronous channels val channel = Channel<Int>() // coroutine #1 for (x in 1..5) { channel.send(x * x) } // coroutine #2 repeat(5) { println(channel.receive()) } 1 2 3
- 70. Senders wait Sender #1H Sender #2 Sender #3 T More senders Incoming receivers Receiver removes first if it is a sender node Sender inserts last if it is not a receiver node
- 71. Receivers wait Receiver #1H Receiver #2 Receiver #3 T More receivers Incoming senders Sender removes first if it is a receiver node Receiver inserts last if it is not a sender node
- 72. Send function sketch fun send(element: T) { while (true) { // try to add sender, unless prev is receiver if (enqueueSend(element)) break // try to remove first receiver val receiver = removeFirstReceiver() if (receiver != null) { receiver.resume(element) // resume receiver break } } } 1 2 3 4
- 73. Channel use-case recap • Uses insert/remove ops conditional on tail/head node • Can abort (cancel) wait to receive/send at any time by using remove • Full removal -- no garbage is left • Pretty efficient in practice • One item lists – one “garbage” object
- 75. Use-case: select expression val channel1 = Channel<Int>() val channel2 = Channel<Int>() select { channel1.onReceive { e -> ... } channel2.onReceive { e -> ... } }
- 76. Impl summary: register (1) Select status: NS Channel1 Queue Channel2 Queue 1. Not selected 2. Selected
- 83. DCSS spec in pseudo-code A B fun <A,B> dcss( a: Ref<A>, expectA: A, updateA: A, b: Ref<B>, expectB: B) = atomic { if (a.value == expectA && b.value == expectB) { a.value = updateA } } 1 2 3 4
- 84. DCSS: init descriptor DCSS Descriptor (a, expectA, updateA, b, expectB) A BexpectA expectB updateA
- 85. DCSS: prepare DCSS Descriptor (a, expectA, updateA, b, expectB) A B CAS ptr to descriptor if a.value == expectA expectA expectB updateA
- 86. DCSS: read b.value DCSS Descriptor (a, expectA, updateA, b, expectB) A B CAS ptr to descriptor if a.value == expectA expectA expectB updateA
- 90. DCSS: States Init A: ??? (desc created) A: desc A was expectA prep ok A: ??? A was !expectA prep fail one tread 1 2 A: updateA B was expectB success A: expectA B was !expectB 4 5 fail Any other thread encountering descriptor helps complete Originator cannot learn what was the outcome Lock-free algorithm without loops! 3
- 91. Caveats • A & B locations must be totally ordered • or risk stack-overflow while helping • One way to look at it: Restricted DCSS (RDCSS)
- 92. DCSS Mod: learn outcome A B fun <A,B> dcssMod( a: Ref<A>, expectA: A, updateA: A, b: Ref<B>, expectB: B): Boolean = atomic { if (a.value == expectA && b.value == expectB) { a.value = updateA true } else false }
- 93. DCSS Mod: init descriptor DCSS Descriptor (a, expectA, updateA, b, expectB) A BexpectA expectB updateA Outcome: UNDECIDED Consensus
- 100. CASN spec in pseudo-code A B fun <A,B> cas2( a: Ref<A>, expectA: A, updateA: A, b: Ref<B>, expectB: B, updateB: B): Boolean = atomic { if (a.value == expectA && b.value == expectB) { a.value = updateA b.value = updateB true } else false } 1 2 3 4 5 For two words, for simplicity
- 101. CASN: init descriptor DCSS Descriptor (a, expectA, updateA, b, expectB, updateB) A BexpectA expectB updateA Outcome: UNDECIDED updateB
- 107. CASN: States A: ??? B: ??? O: UND A: desc B: ??? O: UND A: desc B: desc O: UND A: updateA B: desc O: SUCC Init A: updateA B: updateB O: SUCC 1 2 3 5 A: desc B: desc O: SUCC 4 6 A: ??? B: ??? O: FAIL A != expectA A: desc B: ??? O: FAIL B != expectB one tread A: expectA B: ??? O: FAIL 7 8 9 DCSS Prevents from going back in this SM descriptor is known to other (helping) threads
- 110. Trivial example: Treiber stack 1TOP 2 3 New node CAS expect update
- 116. Doubly linked list: insert (3) S N 1 N P S P H T 2 N P Operation Descriptor desc is updated after successful DCSS A ref: Node #1 expectA: Sentinel updateA: Node #2 … Outcome: UNDECIDED DCSS Descriptor affected node: #1 operation ref
- 122. Doubly linked list: remove (3) S N 1 N P S P H T R1 2 N P Operation Descriptor A ref: Node #1 expectA: Node #2 updateA: Rem[#2] … Outcome: UNDECIDED desc is updated after successful DCSS DCSS Descriptor affected node: #1 operation ref old value: #2
- 124. Closing notes • All we care about is CAS that linearizes operation • Subsequent updates are helper moves • Invoke regular help/correct functions • Perfect algorithm to combine with optional Hardware Transactional Memory (HTM)
- 126. References • Kotlin language • http://kotlinlang.org • Kotlin coroutines support library • http://github.com/kotlin/kotlinx.coroutines