Hyperloglog Lightning Talk
Download as PPTX, PDF•0 likes•149 views
The document discusses the Hyperloglog algorithm for approximately counting unique elements in a data stream using less space than accurate counting. It describes how Redis implements Hyperloglog in an efficient way to count billions of uniques using only 12kb of memory. Developers are advised to use an existing Hyperloglog library like Redis rather than implementing their own solution.
Hyperloglog Lightning Talk
- 1. Lightning Talk: Hyperloglog with Node.js and Redis SIMON PRICKETT, REDIS UNIVERSITY. TWITTER: @SIMON_PRICKETT WEB: HTTPS://SIMONPRICKETT.DEV
- 2. Hyperloglog? • An algorithm for the "count distinct" problem. • What's the approximate size (cardinality) of a set? • Optimizes space efficiency over absolute accuracy.
- 3. Counting Uniques with a Set – Accurate, but...
- 4. Counting Uniques with the Hyperloglog Algorithm Benefits • Similar interface to a Set • Same performance (time complexity) as using a Set • Much more space efficient than a Set • Can't retrieve items, unlike a Set Trade-Offs • Absolute accuracy • Can't retrieve items, unlike a Set • Not built in, need an implementation...
- 5. Hyperloglog Implementation Add Function Count Function TL;DR Don't make your own, use a library or other implementation!
- 6. Redis Hyperloglogs • Fast, open source • Uses maximum 12kb • Count up to 2^64 uniques (18,446,744,073,709,552,000) • Persists to Redis key/value store • Support for most programming languages • Count estimates have 0.81% standard error
- 7. Approximating Uniques with Redis Hyperloglog
- 8. Resources: Redis University on YouTube