Doing data science with clojure
0 likes206 views
The document outlines a perspective on data science using Clojure, highlighting Simon Belak's experience as a core developer at Metabase, a leading open-source BI tool. It emphasizes the importance of speed and structure manipulation in analytics, advocating for efficient data-driven decision-making without compromising workflow. Additionally, it mentions the use of tools like huri.plot and Clojure's strengths in concurrent programming and data abstraction.
Doing data science with clojure
- 1. [email protected] @sbelak Doing data science with Clojure
- 2. My perspective • Core developer at Metabase, an open source BI/analytics tool. 3rd largest BI tool in the world. 21k+ companies use us daily, including N26, Revolut, Swisscom, Confluent, … • Built analytics department at GoOpti from the ground up • Helped 20+ companies become data-driven
- 3. Goal: answer 80% of questions stemming from data in <20min
- 4. The analytics chasm 2 min 20 min project Ideal. Almost real-time. Can be done during brainstorming without disrupting the flow. :( Added to roadmapSqueeze in somewhere in the day
- 5. Doing our job better is often a matter of speed and quantity
- 7. Clojure at a glance • (lisp (running-on :JVM)) • Functional, dynamic, immutable • Excellent concurrency and state-management primitives • Unparalleled data manipulation
- 8. Affordances of our tools shape how we approach problems
- 9. Data frames considered harmful • Data frame (=table) conflates representation and abstraction • Clojure excels in structure manipulation/encoding
- 10. Composability is key to quick iterating • Curried versions where possible • ->> and partial friendly • Side benefit: consistent API
- 11. “This is possibly Clojure’s most important property: the syntax expresses the code’s semantic layers. An experienced reader of Clojure can skip over most of the code and have a lossless understanding of its high- level intent.” — Z. Tellman, Elements of Clojure
- 12. Live programming
- 13. Catching errors early more context easier debugging faster iterating
- 14. The power of sharing runtime
- 15. clojure.spec
- 17. The ecosystem
- 18. What about ML? Farm it out! (to sklearn)
- 19. Mini compilers targeting a specific library in another language
- 20. huri.plot • DSL that compiles to ggplot2 • Targets Gorilla REPL • bar chart, scatter plot, line chart, box & violin plot, heatmap, histogram
- 22. Takeouts • Speed-of-answer matters • You don’t have to reinvent every wheel in Clojure • Clojure is fantastic at structure manipulation, play to its strengths • Blurring the line between environment and work is a powerful idea