HTAP Queries

Editor's Notes

• #3: Anyone heard of HTAP before? Big Data
• #4: They freak out when you try to run batch queries. Finicky about offload windows They Typically have huge backlogs Emerging pattern. Atomic All operations in a transaction succeed or every operation is rolled back. Consistent On the completion of a transaction, the database is structurally sound. Isolated Transactions do not contend with one another. Contentious access to data is moderated by the database so that transactions appear to run sequentially. Durable The results of applying a transaction are permanent, even in the presence of failures. asic Availability The database appears to work most of the time. Soft-state Stores don’t have to be write-consistent, nor do different replicas have to be mutually consistent all the time. Eventual consistency Stores exhibit consistency at some later point (e.g., lazily at read time).
• #5: NOT A NEW CONCEPT
• #6: The scary santa claus!
• #8: We need both but we didn’t realize that until recently. Late Binding
• #9: It started as an epic battle! Society was divided…
• #10: The metro and the uber We need both but we didn’t realize that until recently. Late Binding
• #11: Issues in CAP theorem BASE is not ACID CQRS is well suited to edge cases, not many instances where it worked for the norms. Other issues in distributed systems remain or have become more specialised problems Two approaches to overcome operational copmexlity: Multimodel databases or tightly-integrated middleware over multiple single-model data stores Caveat: In order for a custom data model to support concurrent updates, the database must be able to synchronize updates across multiple keys. ACID transactions, if they are sufficiently performant, allow such synchronization.[12] JSON documents, graphs, and relational tables can all be implemented in a manner that inherits the horizontal scalability and fault-tolerance of the underlying data store.
• #13: Caveat: In order for a custom data model to support concurrent updates, the database must be able to synchronize updates across multiple keys. ACID transactions, if they are sufficiently performant, allow such synchronization.[12] JSON documents, graphs, and relational tables can all be implemented in a manner that inherits the horizontal scalability and fault-tolerance of the underlying data store.
• #15: OLAP systems are great are low volume transactions but typically complex and long running queries with larger working sets Differentiated between adhoc queries vs operational queries. Massively parallel processing engines came out Spins off like Vertica, Greenplum, Netezza and Teradata all were based on PostgreSQL at some point. Architectural drivers Reliability Short queries but very frequent Transaction Support is critical (ACID) Security Typically lots of ad-hoc writes Not great for complex and bulk reads Resource contention Indexes and other data structures. Data is typically in 3NF kind of models Although with PostgreSQL, we know this is not true. JSON, etc.
• #16: OLAP systems are great are low volume transactions but typically complex and long running queries with larger working sets Differentiated between adhoc queries vs operational queries. Massively parallel processing engines came out Spins off like Vertica, Greenplum, Netezza and Teradata all were based on PostgreSQL at some point. Architectural drivers Reliability Short queries but very frequent Transaction Support is critical (ACID) Security Typically lots of ad-hoc writes Not great for complex and bulk reads Resource contention Indexes and other data structures. Data is typically in 3NF kind of models Although with PostgreSQL, we know this is not true. JSON, etc.
• #17: OLAP systems are great are low volume transactions but typically complex and long running queries with larger working sets Differentiated between adhoc queries vs operational queries. Massively parallel processing engines came out Spins off like Vertica, Greenplum, Netezza and Teradata all were based on PostgreSQL at some point. Architectural drivers Reliability Short queries but very frequent Transaction Support is critical (ACID) Security Typically lots of ad-hoc writes Not great for complex and bulk reads Resource contention Indexes and other data structures. Data is typically in 3NF kind of models Although with PostgreSQL, we know this is not true. JSON, etc.
• #20: OLAP systems are great are low volume transactions but typically complex and long running queries with larger working sets Differentiated between adhoc queries vs operational queries. Massively parallel processing engines came out Spins off like Vertica, Greenplum, Netezza and Teradata all were based on PostgreSQL at some point. Architectural drivers Reliability Short queries but very frequent Transaction Support is critical (ACID) Security Typically lots of ad-hoc writes Not great for complex and bulk reads Resource contention Indexes and other data structures. Data is typically in 3NF kind of models Although with PostgreSQL, we know this is not true. JSON, etc.