Webinar - DataStax Enterprise 5.1: 3X the operational analytics speed, help for multi-tenant SaaS apps, & other shiny things
Download as PPTX, PDF3 likes1,015 views
Datastax Enterprise 5.1 introduces enhancements like 3x faster operational analytics, improved multi-tenancy support, and advanced replication capabilities. Key updates include DSE analytics performance improvements, a new file system (DSEFS) compatible with HDFS, and features for DSE search and graph databases. Additional tools such as OpsCenter 6.1 and Studio 2.0 offer enhanced metrics, UI improvements, and better support for developers.
Webinar - DataStax Enterprise 5.1: 3X the operational analytics speed, help for multi-tenant SaaS apps, & other shiny things
- 1. DataStax Enterprise 5.1: 3X the operational analytics speed, help for multi-tenant SaaS apps, & other shiny things Gehrig Kunz, Product Marketing David Gilardi, Technical Evangelist
- 2. • What is a DataStax Enterprise? • Explore what’s new in: • DataStax Enterprise 5.1 • Core • Analytics • Search • Graph • *Live demo* • OpsCenter 6.1 • Studio 2.0 Hey. Thanks for joining us. David @sonicdmg Gehrig @gehrigkunz (#BearDown)
- 3. Powering cloud applications Personalization Customer 360 Recommendation Fraud Detection Inventory Management Identity Management Security Supply Chain
- 4. Cloud application characteristics Real-Time DistributedAlways-OnContextual Scalable
- 5. Easy to build, effortless to scale • DataStax Enterprise 5.1 • Core • Analytics • Search • Graph • DataStax OpsCenter 6.1 • DataStax Studio 2.0 • DSE Drivers
- 6. Easy to build, effortless to scale • DataStax Enterprise 5.1 • Core • Analytics • Search • Graph • DataStax OpsCenter 6.1 • DataStax Studio 2.0 • DSE Drivers
- 9. DSE Core Powered by the best distribution of Apache Cassandra™ • Multi-tenancy made easy • Improvements to Advanced Replication • Continuous paging (DSE Analytics) • Production certified Cassandra 3.11.1 Apache Cassandra, Spark and Solr are trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.
- 10. Best distro of Apache Cassandra™ A look back at 5.0 introductions to DSE Core: • Introduced advanced replication • Tiered storage • Multi-instance
- 11. Row Level Access Control (RLAC) • Part of DSE Advanced Security. • Secures data in tables at the row level. • Handled via CQL. • Enables multi-tenancy capabilities on Cassandra tables.
- 12. RLAC FTW CREATE TABLE microservices.shoppingcart_by_tx ( timestamp UUID, cart UUID, amount decimal, store text, tx_id UUID PRIMARY_KEY((store, tx_id)) ); RESTRICT ROWS on microservices.shoppingcart_by_tx USING store; CREATE ROLE bobcoapp; GRANT SELECT ON ‘bobco’ IN microservices.shoppingcart_by_tx TO bobcoapp;
- 13. Advanced replication advanced-er • Lower overhead and improved performance. • Hub to Spoke replication; multi-directional replication between clusters now supported. • Support for multi-datacenter edge clusters.
- 14. Advanced replication for retail
- 15. Advanced replication for IoT
- 16. DSE Analytics ● 3x operational analytics performance improvements ● DataStax Enterprise File System (DSEFS) • HDFS minus the bad • Example use: PDFs and their metadata ● Separate Spark-only cluster • Adding/removing Spark nodes is cheap • Good for scaling up/down for specific needs • Example use: weekly analytics reporting ● Spark UI reachable from any node ● Production certified Apache Spark™ 2.0 • Support for SparkR.
- 17. What is DSEFS? • A new file system for DSE that is... • HDFS compatible • Distributed • Fault tolerant with no single point of failure • Scale-out Benefits of DSEFS over HDFS Benefits of DSEFS over Cassandra No namenode No overhead from compactions or commit log writes No secondary namenode No data density ramifications No zookeeper No overhead from delete operations; provides immediate deletes No single point of failure No JVM overhead for file system data
- 18. To the core Analytics Table Scan 1 ––– Prior to continuous paging –––
- 19. To the core Analytics Table Scan 1 Scan 2 ––– Without continuous paging –––
- 20. To the core Analytics Table Scan 1 Scan 2 ––– With continuous paging –––
- 21. The result – 3xResponseTime (Seconds) 0 50 100 150 200 250 300 Test 1 Test 2 Test 3 Test 4 Read performance (Lower is better) OSS C*/Spark DSE 5.1
- 22. DSE Search • Increased performance for Indexing; new data now made available for search much faster. • Enhanced CQL support for search index management; makes using search easier than ever. • Production certified Apache Solr 6.0 CREATE SEARCH INDEX IF NOT EXISTS ON songs WITH COLUMNS id,val1; DROP SEARCH INDEX ON songs WITH OPTIONS { deleteResources : true };
- 23. DSE Graph The first scale-out, real-time graph database • First on the market to enable out-of-the-box graph search. • Improved graph analytics: • With Spark GraphFrames, known as ‘DSEGraphFrames’ • Introducing Fluent API with 2 flavors: • Explicit = if you’re coming from DSE drivers • Implicit = if you’re coming from Tinkerpop world
- 24. Graph-based Search Type ahead, fuzzy, and spell-check searches now available on graph data
- 25. Easy to build, effortless to scale • DataStax Enterprise 5.1 • Core • Analytics • Search • Graph • DataStax OpsCenter 6.1 • DataStax Studio 2.0 • DSE Drivers
- 26. OpsCenter 6.1 improvements Live demo! Oooooooooooo
- 27. OpsCenter 6.1 improvements • Full support for DSE 5.1 • Updated repair section • Improved resiliency • Granular control to ignore keyspaces and/or tables • Improved UI to track repair progress • Enhanced metrics and alerts • Graph • Datacenter latency • Datacenter level backup and improved support for Amazon S3 • Backup and restore times decreased • Backup/restore SASI indexes • SASI index definitions automatically backed up on tables backed up, index definitions are rebuilt on restore • Backward compatibility of agents • OpsCenter can be upgraded ahead of agents from 6.1 moving forward
- 29. Improved UI
- 30. Enhanced metrics for DSE Graph and Datacenter Latency
- 31. Datacenter level backup and improved support for Amazon S3
- 32. Backward compatibility of agents from 6.1 moving forward
- 33. Show you stuff
- 35. Some developer love - Studio ● Studio 2.0 ○ New graph visuals ■Vertex size by property value ■Color by label or property value ■Shapes & Icons ■Improved cluster segregation and reduced overlap ○ Gremlin scan steps not using indexes highlighted in profiler ○ CQL Support ■Traditionally “Devcenter” functionality ■CQL ■Schema views ■Tracing ■Consistency level settings ■Schema aware content assist “intellisense magic”
- 36. Vertex size by property value
- 37. Color by label or property value
- 38. Shapes & Icons
- 39. Improved cluster segregation and reduced overlap
- 40. Improved cluster segregation and reduced overlap
- 42. • OpsCenter 6.1 • Timestamps in human readable format instead of epoch • Repair • Better logging • Added “pause” between repairs to avoid aggressive cyclic repairs • Lifecycle Manager • Concurrent inter-cluster jobs • More hardening • Schema viewer support for SASI indexes, MVs, UDTs, UDFs, UDAs Honorable mentions (1/3)
- 43. • Graph • API support for edit distance queries • Type ahead • Spell check • Support for multiple vertices & edges and commit as a single operation/transaction • Support for units in Geo.distance • Time and Date types added Honorable mentions (2/3)
- 44. • Studio 2.0 • Syntax highlighting • Code completion • Schema-aware completion • Validations • Snippets • Dockable schema viewer • Detailed view • CQL execution configurations • Support for charts • DSE Drivers • Unified authentication • Ability to run queries as another user • DateRangeField support in search Honorable mentions (3/3)
- 45. April 4th!
- 46. Release notes – http://docs.datastax.com/en/dse/5.1/dse- dev/datastax_enterprise/RNdse.html How to upgrade – http://docs.datastax.com/en/dse/5.1/upgrade/upgrade/dat astax_enterprise/upgdDSE51.html Graph example data sets – Things to check out All downloads available April 4th academy.datastax.com/slack https://github.com/datastax/graph-examples
- 47. Thank you
Editor's Notes
- #5: Now what we mean by cloud applications, is these few characteristics. Apps today have evolved to be contextual, personalized for us, always-on can never go down with real-time responsiveness. We’re serving up a global application that has numerous endpoints or users around the world and it needs to be scalable to support all of this. So where do we begin with that data layer?
- #14: To enable replication, edit the dse.yaml file. And enable Change Data Capture in cassandra.yaml. DSE Advanced Replication stores all of its settings within an automatically generated CQL tables. To configure replication, load values directly to the tables, or use the dse advrep command line tool. Multi-dc edge = cross microservice hub
- #15: Example use: Microservices analytics but the other major use case we see is replicating individual microservices into a consolidated cluster for cross-microservices view/analysis. It fits in with Customer 360 (in the case where the microservices are customer-related/focused).
- #16: Example use: Internet of Things The benefits have nothing to do with intermittent connections, in this case.
- #17: The "Separate Spark-only cluster" is all about options. A customer can deploy a separate cluster for Spark - good because segregated resources (lower contention), and ease of scaling up/down (like you say). I would shy away from saying this is a preferred mode or not, just another deployment mode that a number of folks have wanted.
- #19: The point of continuous paging is that it pipelines the pages expecting that you will ask for the next page. In "normal" paging, a single page is returned to the client application, and the server frees up all the resources. This allows the server to not have to hold on to expensive memory, etc, and is more efficient - unless you want more pages. When the client asks for the second page, the server has to spin up a bunch of data structures and find where it had left off from the last page, get the data, and then free those resources again. If you are going to fetch a bunch of pages, this statelessness is expensive. But with Continuous Paging the server keeps all those data structures in memory for a while anticipating that you will ask for the next page. It does eat up RAM, but at the benefit of more efficiently getting the next page. There is a timeout that if you haven't fetched the next page in a while (5 seconds default), then it will free the resources. If you picture and talk track is something like that, then :thumbsup:.
- #22: Lower is better.
- #28: Resiliency - previous, if encountered > 100 errors repair service would shutdown, in 6.1 it continues on and warns users, no more repair service shutdown “Generally speaking, when encountering any obstacles 6.1 RS will try to pause and resume or pause and restart, but it should never just stop”
- #30: Improved resiliency Resiliency - previous, if encountered > 100 errors repair service would shutdown, in 6.1 it continues on and warns users, no more repair service shutdown “Generally speaking, when encountering any obstacles 6.1 RS will try to pause and resume or pause and restart, but it should never just stop” Granular control to ignore keyspaces and/or tables Improved UI to track repair progress
- #32: S3 Backup and restore speeds decreased
- #33: OpsCenter can be upgraded ahead of agents from 6.1 moving forward
- #34: Repair screen Graph and datacenter metrics Datacenter level backup Backward compatibility of agents
- #36: Neat example with consistency level ONE compared to ALL for “select * from users” Tracing really illustrates the difference Easily demonstrate schema view for both graph and CQL Schema aware will constrain results to object looking at...makes you think SO MUCH LESS than before
- #38: Community detection Label Duration property w/linear option
- #42: Vertex size - Shapes and icons Color by label and property Graph profiler not using index CQL schema aware Tracing and replication factor Schema views