3 years with Clojure
35 likes31,617 views
This document summarizes a presentation about the speaker's 3 years of experience using Clojure. Some key points include: the speaker wanted an operationally sane environment with good tooling and performance on the JVM; Clojure provided a productive, concise language with an excellent concurrency story and stability; Clojure's consistent, well-designed core and easy upgrades have made it a fun and motivating language to work with.
![Clojure: baseline
performance
(let [d (java.util.Date.)]
d)](https://image.slidesharecdn.com/3yearswithclojure-140522114903-phpapp02/85/3-years-with-Clojure-59-320.jpg)
![Clojure: baseline
performance
(let [d (java.util.Date.)]
d)](https://image.slidesharecdn.com/3yearswithclojure-140522114903-phpapp02/85/3-years-with-Clojure-60-320.jpg)
3 years with Clojure
- 1. Life of an immutant 3 years with Clojure
- 2. About me • @michaelklishin • github.com/michaelklishin
- 4. Setting expectations • This is an experience report
- 5. Setting expectations • This is an experience report • Not a language primer
- 6. Setting expectations • This is an experience report • Not a language primer • Assumes no familiarity except basic facts
- 7. Setting expectations • This is an experience report • Not a language primer • Assumes no familiarity except basic facts • ~200 slides.This is gonna be fast.
- 8. Background
- 10. Background • Data processing • Scheduling systems
- 11. Background • Data processing • Scheduling systems • Specialized Web crawlers
- 12. Background • Data processing • Scheduling systems • Specialized Web crawlers • Billing, accounting
- 13. Background • Data processing • Scheduling systems • Specialized Web crawlers • Billing, accounting • Infrastructure automation
- 14. Wanted
- 15. Wanted • An environment that does not suck
- 16. Wanted • An environment that does not suck is operationally sane
- 17. Wanted • An environment that does not suck is operationally sane • A runtime that has decent tooling (profilers, concurrency-related metrics, GC metrics)
- 18. Wanted • An environment that does not suck is operationally sane • A runtime that has decent tooling (profilers, concurrency-related metrics, GC metrics) • A runtime that yields good baseline performance
- 19. Wanted • An environment that does not suck is operationally sane • A runtime that has decent tooling (profilers, concurrency-related metrics, GC metrics) • A runtime that yields good baseline performance • Painless upgrade paths
- 21. Wanted • Community • Benefit from existing libraries and tools
- 22. Wanted • Community • Benefit from existing libraries and tools • Concurrency and parallelism
- 23. Wanted • Community • Benefit from existing libraries and tools • Concurrency and parallelism • A productive, concise language
- 24. Wanted JVM
- 25. JVM
- 26. JVM • One of the most mature runtimes you can find
- 27. JVM • One of the most mature runtimes you can find • Excellent tooling
- 28. JVM • One of the most mature runtimes you can find • Excellent tooling (VisualVM,YourKit, jstack, jhat,VM/+XX flags, BTrace, …)
- 29. JVM • One of the most mature runtimes you can find • Excellent tooling • Excellent baseline performance
- 30. JVM • One of the most mature runtimes you can find • Excellent tooling • Excellent baseline performance • Multilingual: Clojure, Java, JRuby, Scala, JS implementations
- 32. Clojure
- 33. Clojure • Very high level, concise, expressive
- 34. Clojure • Very high level, concise, expressive • Excellent concurrency/parallelism story
- 35. Clojure • Very high level, concise, expressive • Excellent concurrency/parallelism story • Very good baseline performance
- 36. Clojure • Very high level, concise, expressive • Excellent concurrency/parallelism story • Very good baseline performance • Consistent, well designed
- 37. Clojure • Very high level, concise, expressive • Excellent concurrency/parallelism story • Very good baseline performance • Consistent, well designed • Decent tooling
- 38. Clojure • Very high level, concise, expressive • Excellent concurrency/parallelism story • Very good baseline performance • Consistent, well designed • Decent tooling • Fun to work with (motivation)
- 39. Clojure • No bored academics, built by a practicioner
- 41. Clojure: concurrency/ parallelism story • Immutable data structures by default
- 42. Clojure: concurrency/ parallelism story • Immutable data structures by default • Identity/values separation
- 43. Clojure: concurrency/ parallelism story • Immutable data structures by default • Identity/values separation • Concurrency-aware reference types
- 44. Clojure: concurrency/ parallelism story • Immutable data structures by default • Identity/values separation • Concurrency-aware reference types • java.util.concurrent
- 45. Clojure: concurrency/ parallelism story When I find myself in times of trouble Prof. Doug Lea comes to me Coding lines of wisdom j.u.c.
- 46. Clojure: concurrency/ parallelism story • Immutable data structures by default • Identity/values separation • Concurrency-aware reference types • java.util.concurrent • Reactor,Akka, Disruptor, RxJava…
- 47. Clojure: concurrency/ parallelism story • Async I/O: NIO, Netty,Vert.x…
- 48. Clojure: concurrency/ parallelism story • You have choice
- 49. Clojure: concurrency/ parallelism story • You have choice • Runtime parallelism (on JVM, CLR)
- 50. Clojure: concurrency/ parallelism story • You have choice • Runtime parallelism (on JVM, CLR) • Hundreds, thousands of threads at the same time
- 53. Clojure: baseline performance • What matters the most is what bytecode the compiler produces
- 55. Clojure: baseline performance • What matters the most is what bytecode the compiler produces • Java interop bytecode is typically the same as produced by javac
- 56. Clojure: baseline performance • What matters the most is what bytecode the compiler produces • Java interop bytecode is typically the same as produced by javac • Watch out for reflective calls, primitive boxing
- 57. Clojure: baseline performance • Optional type hints
- 58. Clojure: baseline performance • Optional type hints • Type inference in simple cases
- 59. Clojure: baseline performance (let [d (java.util.Date.)] d)
- 60. Clojure: baseline performance (let [d (java.util.Date.)] d)
- 61. Clojure: baseline performance • Optional type hints • Type inference in simple cases
- 62. Clojure: baseline performance • Optional type hints • Type inference in simple cases • If JVM can optimize your bytecode, things run screaming fast
- 63. Clojure: baseline performance • Optional type hints • Type inference in simple cases • If JVM can optimize your bytecode, things run screaming fast • From 2% (Java lib wrappers) to 200% (naïve numerics heavy code, tight loops) penalty compared to Java
- 64. Clojure: baseline performance • Immutable data structures have O(log32 n) access complexity
- 65. Clojure: baseline performance • Immutable data structures have O(log32 n) access complexity • Immutable data structures have associated GC tax
- 66. Clojure: baseline performance • Immutable data structures have O(log32 n) access complexity • Immutable data structures have associated GC tax • Focus on exploiting parallelism over squeezing µs from a single thread
- 67. Clojure: baseline performance • Poor startup time, excellent execution performance
- 68. Clojure: baseline performance • Poor startup time, excellent execution performance • Anecdotal evidence: 10 — 50 times throughput improvements compared to Ruby in real apps
- 71. Clojure: stability • Have been using 1.3 since alpha1
- 72. Clojure: stability • Have been using 1.3 since alpha1 • Hit by 4 Clojure bugs in ~3 years
- 73. Clojure: stability • Have been using 1.3 since alpha1 • Hit by 4 Clojure bugs in ~3 years • 1 bug is an edge case that is not worth fixing (I did something really stupid)
- 74. Clojure: stability • Have been using 1.3 since alpha1 • Hit by 4 Clojure bugs in ~3 years • 1 bug is an edge case that is not worth fixing (I did something really stupid) • Alphas more solid than GA releases of almost all other languages I’ve worked with
- 75. Clojure: stability • Hit by 4 Clojure bugs in ~3 years • 1 bug is an edge case that is not worth fixing (I did something really stupid) • Alphas more solid than GA releases of almost all other languages I’ve worked with • Others (e.g.ThoughtWorks) report the same experience
- 76. Clojure: stability • Upgraded 28 apps & libraries from 1.3 to 1.4. Just 1 change due to a tricky 3rd party library.The rest was a drop-in replacement.
- 77. Clojure: stability • Upgraded 28 apps & libraries from 1.3 to 1.4. Just 1 change due to a tricky 3rd party library.The rest was a drop-in replacement. • When upgrading is this easy, people do it happily
- 78. Clojure: stability • Upgraded 28 apps & libraries from 1.3 to 1.4. Just 1 change due to a tricky 3rd party library.The rest was a drop-in replacement. • When upgrading is this easy, people do it happily • No library binary compatibility hell some other JVM languages suffer from
- 79. Clojure: stability • Many active developers are conservative with versioning & calling things “done”
- 80. Clojure: stability • Many active developers are conservative with versioning & calling things “done” • Fits JVM ecosystem maturity really well
- 81. Clojure: stability • The real reason: Clojure is small and was designed to be simple
- 82. Clojure: stability • No monkey patching (there are features that give you all the good parts of teh)
- 84. Clojure: consistency • Collections/sequences library is very unified
- 85. Clojure: consistency • Collections/sequences library is very unified • Easy to implement your own data types that work just like core Clojure types
- 86. Clojure: consistency • Collections/sequences library is very unified • Easy to implement your own data types that work just like core Clojure types • Almost no “bad baggage” in the core
- 87. Clojure: consistency • Collections/sequences library is very unified • Easy to implement your own data types that work just like core Clojure types • Almost no “bad baggage” in the core • The language was actually designed
- 89. Clojure: deployment • Pretty standard JVM deployment options
- 90. Clojure: deployment • Pretty standard JVM deployment options • Compile everything AOT into a single jar (“überjar”), including dependencies, assets, etc
- 91. Clojure: deployment • Pretty standard JVM deployment options • Compile everything AOT into a single jar (“überjar”), including dependencies, assets, etc • A tiny shell script + start-stop-daemon
- 92. Clojure: deployment • Pretty standard JVM deployment options • Compile everything AOT into a single jar (“überjar”), including dependencies, assets, etc • A tiny shell script + start-stop-daemon • Chef or Capistrano
- 94. Clojure: deployment • CloudFoundry • Heroku
- 95. Clojure: deployment • CloudFoundry • Heroku • Immutant/JBoss AS
- 96. Clojure: deployment • CloudFoundry • Heroku • Immutant/JBoss AS • OpenShift
- 98. Clojure: operations • VisualVM, jstack, jheap, …
- 99. Clojure: operations • VisualVM, jstack, jheap, … • nREPL
- 100. Clojure: operations • VisualVM, jstack, jheap, … • nREPL • Remote REPL over HTTP(S)
- 101. Clojure: operations • VisualVM, jstack, jheap, … • nREPL • Remote REPL over HTTP(S) • Not quite Erlang but still impressive
- 102. Clojure: operations • VisualGC gives real insight into what is going on with object allocation, tenuring, death rate
- 103. Clojure: operations • VisualGC gives real insight into what is going on with object allocation, tenuring, death rate • JVM DTrace support (Solaris, SmartOS, FreeBSD, OS X)
- 104. Clojure: operations • VisualGC gives real insight into what is going on with object allocation, tenuring, death rate • JVM DTrace support (Solaris, SmartOS, FreeBSD, OS X) • BTrace
- 105. Clojure: community
- 106. Clojure: community • Full of intelligent engineers who don’t give two shits about Hacker News
- 107. Clojure: community • Full of intelligent engineers who don’t give two shits about Hacker News • Over 10 conferences to date
- 108. Clojure: community • Full of intelligent engineers who don’t give two shits about Hacker News • Over 10 conferences to date • ~10K people on the mailing list
- 109. Clojure: community • Full of intelligent engineers who don’t give two shits about Hacker News • Over 10 conferences to date • ~10K people on the mailing list • Users from small and not-so-small startups to Fortune 50 corporations in finance, biotech, retail
- 110. Clojure: fun to work with
- 111. Clojure: fun to work with • Profoundly changes the way you think
- 112. Clojure: changes the way you think
- 113. Clojure: changes the way you think • About managing your program state
- 114. Clojure: changes the way you think • About managing your program state • About the value of experimenting (REPL driven development)
- 115. Clojure: changes the way you think • About managing your program state • About the value of experimenting (REPL driven development) • About metaprogramming
- 116. Clojure: changes the way you think • About managing your program state • About the value of experimenting (REPL driven development) • About metaprogramming • About isolating side effects
- 117. Clojure: changes the way you think
- 118. Clojure: changes the way you think • Program = pure core + parts that communicate with the outside world (network, disk I/O)
- 119. Clojure: changes the way you think • Program = pure core + parts that communicate with the outside world (network, disk I/O) • Pure parts are ridiculously easy to reason about, test, experiment with and reuse
- 120. Clojure: changes the way you think • Program = pure core + parts that communicate with the outside world (network, disk I/O) • Pure parts are ridiculously easy to reason about, test, experiment with and reuse • Typical Rails app: 95% of methods have side effects
- 121. Clojure: changes the way you think • Typical Clojure app: 40% of functions have side effects
- 122. Clojure: changes the way you think • Typical Clojure app: 40% of functions have side effects • Anecdotal evidence: 3-4 times less time spent on tests
- 123. Clojure: changes the way you think • Typical Clojure app: 40% of functions have side effects • Anecdotal evidence: 3-4 times less time spent on tests • Moral of the story: make more of your code not have side effects
- 124. Clojure: fun to work with • Profoundly changes the way you think
- 125. Clojure: fun to work with • Profoundly changes the way you think • Motivates you to learn
- 126. Clojure: fun to work with • Profoundly changes the way you think • Motivates you to learn • Easy things are easy, hard things are possible
- 127. Clojure: fun to work with • Profoundly changes the way you think • Motivates you to learn • Easy things are easy, hard things are possible • Wheels are already invented in clojure.core or JDK
- 128. Clojure: fun to work with • Stable: fewer distractions, get in the flow
- 129. Clojure: fun to work with • Stable: fewer distractions, get in the flow • REPL driven development keeps you in the flow for longer periods
- 130. Clojure: fun to work with • Stable: fewer distractions, get in the flow • REPL driven development keeps you in the flow for longer periods • Polymorphism done right <3
- 131. Clojure: fun to work with “Clojure demands that you raise your game and rewards you greatly for it…”
- 132. Clojure: fun to work with • The community is full of smart and down to earth people.
- 134. Clojure: use cases
- 135. Clojure: use cases • Data processing
- 136. Clojure: use cases • Data processing • Stream processing
- 137. Clojure: use cases • Data processing • Stream processing • Micro-services
- 138. Clojure: use cases • Data processing • Stream processing • Micro-services • Machine learning
- 139. Clojure: use cases • Data processing • Stream processing • Micro-services • Machine learning • Web development
- 141. Clojure: misc
- 142. Clojure: misc • Leiningen (leiningen.org) is excellent
- 143. Clojure: misc • Leiningen (leiningen.org) is excellent • Deps management, build tool, custom tasks are just Clojure functions
- 144. Clojure: misc • Leiningen (leiningen.org) is excellent • Deps management, build tool, custom tasks are just Clojure functions • Packages your code + all deps into a single jar for deployment
- 145. Clojure: misc • Leiningen (leiningen.org) is excellent • Deps management, build tool, custom tasks are just Clojure functions • Packages your code + all deps into a single jar for deployment • Makes it trivial to develop against multiple Clojure versions
- 147. Clojure: misc • clojars.org • Deploy via SSH (SCP) or HTTPS
- 148. Clojure: misc • clojars.org • Deploy via SSH (SCP) or HTTPS • Very easy to deploy snapshot releases
- 149. Clojure: misc • clojars.org • Deploy via SSH (SCP) or HTTPS • Very easy to deploy snapshot releases • Clojars and Leiningen are maintained by the same core team
- 150. Clojure: misc • travis-ci.org support
- 151. Clojure: misc • travis-ci.org support • Test against multiple JDKs
- 152. Clojure: ecosystem • Feels a bit like Ruby circa 2009
- 153. Clojure: ecosystem • Feels a bit like Ruby circa 2009 • But with the entire JVM ecosystem at your reach
- 154. Clojure: ecosystem • Feels a bit like Ruby circa 2009 • But with the entire JVM ecosystem at your reach • And Leiningen + clojars.org
- 155. Clojure: ecosystem • Feels a bit like Ruby circa 2009 • But with the entire JVM ecosystem at your reach • And Leiningen + clojars.org • Community ~ doubles every year in size
- 156. Clojure: ecosystem • Feels a bit like Ruby circa 2009 • But with the entire JVM ecosystem at your reach • And Leiningen + clojars.org • Community ~ doubles every year in size • Google for “The State of Clojure” survey
- 157. Clojure: ecosystem • ~35% of people come from Java, ~35% from Ruby or Python
- 158. Clojure: ecosystem • ~35% of people come from Java, ~35% from Ruby or Python • Libraries, not frameworks
- 159. Clojure: ecosystem • ~35% of people come from Java, ~35% from Ruby or Python • Libraries, not frameworks • Decent libraries just for almost every problem
- 160. Clojure: ecosystem • ~35% of people come from Java, ~35% from Ruby or Python • Libraries, not frameworks • Decent libraries just for almost every problem • 15 books, 2 outdated (cover 1.2)
- 161. Clojure: ecosystem • ~35% of people come from Java, ~35% from Ruby or Python • Libraries, not frameworks • Decent libraries just for almost every problem • 15 books, 2 outdated (cover 1.2) • clojure-doc.org, tryclj.com, …
- 162. Clojure: ecosystem
- 164. Clojure: what sucks
- 165. Clojure: what sucks • clojure.org
- 166. Clojure: what sucks • clojure.org • Documentation (guides, the ref is mostly OK, books are great)
- 167. Clojure: what sucks • clojure.org • Documentation (guides, the ref is mostly OK, books are great) • Compiler messages can be cryptic
- 168. Clojure: what sucks • clojure.org • Documentation (guides, the ref is mostly OK, books are great) • Compiler messages can be cryptic • JVM startup time + code compilation on each run (without REPL or tools such as Drip)
- 169. Clojure: what sucks • Paper contributor agreement
- 171. ClojureWerkz
- 172. ClojureWerkz “A growing collection of open source Clojure libraries…”
- 174. ClojureWerkz • Modern targets (Clojure 1.5+, JDK 6)
- 175. ClojureWerkz • Modern targets (Clojure 1.5+, JDK 6) • Feature rich
- 176. ClojureWerkz • Modern targets (Clojure 1.5+, JDK 6) • Feature rich • Well documented
- 177. ClojureWerkz • Modern targets (Clojure 1.5+, JDK 6) • Feature rich • Well documented • Friendly license (EPL, the same as Clojure)
- 178. ClojureWerkz • Modern targets (Clojure 1.5+, JDK 6) • Feature rich • Well documented • Friendly license (EPL, the same as Clojure) • Tested against multiple Clojure versions, 3 JDKs on travis-ci.org
- 180. ClojureWerkz • Projects reuse a lot of Java libraries
- 181. ClojureWerkz • Projects reuse a lot of Java libraries • Which are often officially supported (e.g. RabbitMQ, ElasticSearch, MongoDB, Riak, Reactor)
- 182. “ClojureWerkz stuff is really improving my Clojure experience, which is rapidly becoming my language of choice…”
- 185. Clojure: the takeaway • A lot of merit of its own
- 186. Clojure: the takeaway • A lot of merit of its own • Great stability
- 187. Clojure: the takeaway • A lot of merit of its own • Great stability • Hosted (symbiotic) language is a great thing
- 188. Clojure: the takeaway • A lot of merit of its own • Great stability • Hosted (symbiotic) language is a great thing • Don’t fear the JVM
- 189. Clojure: the takeaway • A lot of merit of its own • Great stability • Hosted (symbiotic) language is a great thing • Don’t fear the JVM • Immutability is essential for sane concurrent programming
- 190. Clojure: the takeaway • Clojure greatly rewards those who choose to use it
- 192. The real reason?
- 194. The real reason? ???
- 196. Thank you • @michaelklishin • github.com/michaelklishin • clojurewerkz.org • [email protected]