Beyond JSON - An Introduction to FlatBuffers
0 likes13,680 views
This document provides an introduction and overview of FlatBuffers, a cross-platform library for efficient serialization of structured data. It discusses why data serialization is needed, important criteria for persisting and sending data, comparisons of JSON and FlatBuffers, and demonstrates FlatBuffers features like the IDL, size and performance benefits, partial reads, and support for data versioning.
Beyond JSON - An Introduction to FlatBuffers
- 1. Maxim Zaks Beyond JSON - An Introduction to FlatBuffers
- 2. Agenda • About me • The Why and How of Data Serialisation • Deep dive into FlatBuffers
- 3. RESI News - Nachrichten, die Spaß machen
- 6. VS.
- 7. Why do we need data serialisation? • Persisting State • Machine to Machine Communication • Representation of Configuration
- 8. How can we persist data on mobile? • Custom binary representation • Language provided binary serialisation • Text based representation • CSV, XML, JSON,YAML • Embedded SQL or NoSQL DB • Binary cross platform serialisation library • FlatBuffers, Protocol Buffers, Cap’n Proto, SBE, Apache Thrift
- 9. Important criteria for persisting data • Size on disk • Speed of read & write / partial read / memory consumption • Human readable and writable • Support of OO language type system • Data versioning / evolution / migration
- 10. VS.
- 11. JSON • Size on disk 👍👎 • Speed of read & write / partial read / memory consumption 👎 • Human readable and writable 👍 👍 • Support of OO language type system 👍👎 • Data versioning / evolution / migration 👍👎
- 12. FlatBuffers • Size on disk 👍👎 • Speed of read & write / partial read / memory consumption 👍 👍 • Human readable and writable 👍👎 • Support of OO language type system 👍 👍 • Data versioning / evolution / migration 👍 👍
- 13. JSON FlatBuffers size on disk 👍👎 👍👎 read & write performance 👎 👍👍 human readable & writable 👍👍 👍👎 typed API 👍👎 👍👍 multi version support 👍👎 👍👍
- 14. But what about sending data?
- 15. Important criteria for sending data • Size on disk wire • Speed of read & write / partial read / memory consumption • Human readable and writable • Support of OO language type system • Data versioning / evolution / migration
- 16. Enough theory let’s do some practice
- 17. FlatBuffers IDL
- 20. FlatBuffers IDL
- 21. FlatBuffers IDL
- 22. FlatBuffers IDL
- 23. FlatBuffers IDL
- 30. What does it all mean in practice?
- 31. Size on disk / wire
- 32. Speed of read, write and memory allocation
- 35. Partial Read
- 36. • over 3M cities • ≈200 MB • real time search • in memory • from file 10x slower
- 37. Human readable & writable is a manner of tooling • ./flatc -b person.fbs person.json • ./flatc -t person.fbs -- person.bin --raw- binary --strict-json
- 38. Support of OO Language type system
- 40. Data is backwards & forwards compatible
- 41. More information • https://google.github.io/flatbuffers/ • https://github.com/mzaks/ FlatBuffersSchemaEditor • https://github.com/mzaks/FlatBuffersSwift • https://twitter.com/iceX33
- 42. Questions?
- 43. Thank you!