MapR-DB – The First In-Hadoop Document Database
Download as PPTX, PDF3 likes2,461 views
MapR-DB offers a NoSQL solution designed for big data, addressing issues prevalent in traditional RDBMS systems and other NoSQL databases. It emphasizes tighter Hadoop integration, high performance with low latency, and minimal administration for global deployments. The platform supports real-time analytics, automatic data management, and granular security, catering to diverse applications across complex datasets.
MapR-DB – The First In-Hadoop Document Database
- 1. © 2015 MapR Technologies 1© 2015 MapR Technologies MapR-DB: New Options For Creating Breakthrough Next Gen Apps with NoSQL And Hadoop
- 2. © 2015 MapR Technologies 2 NoSQL Was Designed For Big Data • RDBMSs has been the default choice for applications – But face cost/time challenges for rapidly growing, varying data sets • NoSQL was designed for big data – E.g., User transaction data, sensor data, IoT data, time series data, etc. RDBMS NoSQL
- 3. © 2015 MapR Technologies 3 Known NoSQL Database Challenges Today With Other NoSQL Databases • Data loss • Data inconsistency • Long maintenance downtime (e.g., compactions, anti-entropy) • Coarse grained access controls X • Cluster/silo sprawl – Maintenance pains – Complexity, more error prone • Constant data movement between database and analytics cluster – Excessive bandwidth utilization – Delays in accessing data • Modeling of complex data – Longer app development cycles – Higher chance of coding errors • Multiple databases for multiple kinds of applications
- 4. © 2015 MapR Technologies 4 Requirements to Resolve Today’s Challenges • Tighter Hadoop integration – Reduce cluster sprawl – Reduce data movement – Enable real-time analytics on live data – Lower administrative overhead • Flexible JSON data model • Automatic optimizations – Less maintenance downtime – Consistent high performance • Fine grained access controls – More than simply table/document level • Globally consistent deployment capability Hadoop NoSQL Data Platform
- 5. © 2015 MapR Technologies 5 MapR-DB Architectural Principles Dramatically Simpler, High-Performance at Global Scale • Self-healing from HW and SW failures – Replicated state and data for instant recovery – Automated re-replication of data • High performance and low latency – Integrated system with fewer software layers – Single hop to data – No compactions, low i/o amplification (patented secret sauce) • Minimal administration – Single namespace for files and tables (and streams going forward) – Built-in data management & protection – Automatic splits and merges as data grows and shrinks • Global low-latency replication for disaster recovery
- 6. © 2015 MapR Technologies 6 Built-into Hadoop = Real-time Hadoop NoSQL Churn Analysis Offers Fraud Detection Customer Profiles Log files IoT Data Batch Copies Analytical Operational MapR Distribution Churn Analysis Offers Fraud Detection Customer Profiles Log files IoT Data Analytical + Operational Analytics as it happens, no cross-cluster copying Hadoop MapR-DB Non-MapR: • Batch-only • Cluster sprawl With MapR: • Real-time data access • Multi-use-case platform Revenue Optimization Predictive Analytics Sentiment analysis Click streams Call logs Social media
- 7. © 2015 MapR Technologies 7 Real-Time Integration with Other Systems MapR-DB replication engine is extensible for integration with any external systems MapR-DB Streaming Real-Time Reliable Transport Storm Elasticsearch Remote MapR-DB Tables Future
- 8. © 2015 MapR Technologies 8 Designed For Global deployments Multi-master (aka, active/active) replication Active Read/Write End Users • Faster data access – minimize network latency on global data with local clusters • Reduced risk of data loss – real-time, bi-directional replication for synchronized data across active clusters • Application failover – upon any cluster failure, applications continue via redirection to another cluster
- 9. © 2015 MapR Technologies 9 Real-Time Analytics With Hadoop Distributed clusters close to the end users, with real-time analytics at central cluster MapR-DB cluster (London) MapR-DB cluster (New York) MapR-DB cluster (Singapore) MapR-DB/Hadoop cluster Hadoop analytics Operational and analytical workloads combined in a single cluster in in a single datacenter Operationally efficient, consolidated MapR cluster Database operations Hadoop analytics Active Read/Write End Users
- 10. © 2015 MapR Technologies 10 Granular Security Use Access Control Expressions (ACEs) to set granular permissions. Example: user:mary | (group:admins & group:VP) & user:!bob
- 11. © 2015 MapR Technologies 11 Open Source OJAI API for JSON-Based Applications on Hadoop Open JSON Application Interface (OJAI) Databases Other Systems MapR-DB MapR-Client {JSON} File Systems
- 12. © 2015 MapR Technologies 12 Single Cluster Data Lake Capabilities Paste your MapR distribution for Hadoop diagram from Part A, (slide 2) here MapR-DB MapR-FS MapR Data Platform Distribution including Apache Hadoop MapR-DB: relational, time series, structured data MapR-FS: emails, blogs, tweets, log files, unstructured data Agile, self- service data exploration ETL into operational reporting formats (e.g., Parquet) Multi-tenancy: job/data placement control, volumes Access controls: file, table, column, column family, doc, sub-doc levels Sources RELATIONAL, SAAS, MAINFRAME DOCUMENTS, EMAILS LOG FILES, CLICKSTREAM SENSORS BLOGS, TWEETS, LINK DATA DATA WAREHOUSES, DATA MARTS Auditing: compliance, analyze user accesses Snapshots: track data lineage and history Table Replication: global multi-master, business continuity
- 13. © 2015 MapR Technologies 13 Q&A @mapr maprtech @mapr.com Engage with us! MapR maprtech mapr-technologies
Editor's Notes
- #3: What is NoSQL used for (one slide) Applications for non-relational data, rapidly growing data sets, time series data, consolidating disparate data sets
- #4: Typical pain points (one slide) Versus RDBMS –scaling challenges (leading to loss of performance and higher costs), data modeling challenges Versus existing NoSQL –data integrity/reliability, high maintenance, inability to handle 24x7 environments, limited security capabilities
- #5: What we think needs to be resolved (one slide) Hadoop integration “big data capabilities” – predictive analytics, anomaly detection, large-scale processing Enterprise-grade reliability Reduced cluster sprawl, real-time access to data, reduced data movement, reduced administration on disparate technologies Automatic optimizations – no compactions, garbage collection, anti-entropy, complex HA configurations Security (access controls) at granular level