Chapter1: NoSQL: It’s about making intelligent choices
Download as PPTX, PDF0 likes2,631 views
The document provides an overview of NoSQL databases, describing their features, business drivers, and various case studies demonstrating their applications. NoSQL systems enable rapid data processing, flexibility with data formats, and scalability without the constraints of traditional relational database management systems (RDBMS). Key benefits include improved handling of big data, real-time queries, and adaptability to changing business needs.
Chapter1: NoSQL: It’s about making intelligent choices
- 1. Basic Concepts No SQL Databases 1 Chapter1 : Making Sense of NoSQL
- 2. Outline • What is NoSQL? • NoSQL business Drivers • NoSQL case studies 2 Chapter1 : Making Sense of NoSQL
- 3. What is NoSQL? NoSQL is a set of concepts that allows the rapid and efficient processing of datasets with a focus on performance, reliability, and agility. The definition is broad definition and it does not exclude SQL or RDBMS. 3 Chapter1 : Making Sense of NoSQL
- 4. What are the goal features? • It’s more than rows in tables: NoSQL systems store and retrieve data from many formats: key-value stores, graph databases, column-family (Bigtable) stores, document stores, and even rows in tables. • It’s free of joins: NoSQL systems allow you to extract your data using simple interfaces without joins. • It’s schema-free: NoSQL systems allow you to drag-and-drop your data into a folder and then query it without creating an entity-relational model. 4 Chapter1 : Making Sense of NoSQL
- 5. What are the goal features? • It works on many processors: NoSQL systems allow you to store your database on multiple processors and maintain high- speed performance. • It uses shared-nothing commodity computers: Most (but not all) NoSQL systems leverage low-cost commodity processors that have separate RAM and disk. • It supports linear scalability: When you add more processors, you get a consistent increase in performance. • It’s innovative: NoSQL offers options to a single way of storing, retrieving, and manipulating data. NoSQL supporters (also known as NoSQLers) have an inclusive attitude about NoSQL and recognize SQL solutions as viable options. To the NoSQL community, NoSQL means “Not only SQL.” 5 Chapter1 : Making Sense of NoSQL
- 6. What NoSQL is Not? • It’s not about the SQL language: The definition of NoSQL isn’t an application that uses a language other than SQL. SQL as well as other query languages are used with NoSQL databases. • It’s not only open source: Although many NoSQL systems have an open source model, commercial products use NOSQL concepts as well as open source initiatives. You can still have an innovative approach to problem solving with a commercial product. • It’s not only big data: Many, but not all, NoSQL applications are driven by the inability of a current application to efficiently scale when big data is an issue. Though volume and velocity are important, NoSQL also focuses on variability and agility. 6 Chapter1 : Making Sense of NoSQL
- 7. What NoSQL is Not? • It’s not about cloud computing: Many NoSQL systems reside in the cloud to take advantage of its ability to rapidly scale when the situation dictates. NoSQL systems can run in the cloud as well as in your corporate data center. • It’s not about a clever use of RAM and SSD: Many NoSQL systems focus on the efficient use of RAM or solid state disks to increase performance. Though this is important, NoSQL systems can run on standard hardware. • It’s not an elite group of products: NoSQL isn’t an exclusive club with a few products. There are no membership dues or tests required to join. To be considered a NoSQLer, you only need to convince others that you have innovative solutions to their business problems. 7 Chapter1 : Making Sense of NoSQL
- 8. NoSQL on Google Trends 8 Chapter1 : Making Sense of NoSQL
- 9. Database Architecture Patterns 9 Chapter1 : Making Sense of NoSQL
- 10. NoSQL data store 10 Chapter1 : Making Sense of NoSQL
- 11. NoSQL Business Drivers - Volume • look for at alternatives to their current RDBMSs is a need to query big data using clusters of commodity processors. • The ability to increase processing speed was no longer an option. • The need to scale out (also known as horizontal scaling), rather than scale up (faster processors), moved organizations from serial to parallel processing. 11 • The data problems are split into separate paths and sent to separate processors to divide and conquer the work. Chapter1 : Making Sense of NoSQL
- 12. NoSQL Business Drivers - Velocity • Though big data problems are a consideration for many organizations moving away from RDBMSs, the ability of a single processor system to rapidly read and write data is also key. • Many single-processor RDBMSs are unable to keep up with the demands of real-time inserts and online queries to the database made by public-facing websites. • RDBMSs frequently index many columns of every new row, a process which decreases system performance. 12 • When single-processor RDBMSs are used as a back end to a web store front, the random bursts in web traffic slow down response for everyone, and tuning these systems can be costly when both high read and write throughput is desired. Chapter1 : Making Sense of NoSQL
- 13. NoSQL Business Drivers - Variability • Companies that want to capture and report on exception data struggle when attempting to use rigid database schema structures imposed by RDBMSs. • For example, if a business unit wants to capture a few custom fields for a particular customer, all customer rows within the database need to store this information even though it doesn’t apply. 13 • Adding new columns to an RDBMS requires the system be shut down and ALTER TABLE commands to be run. • When a database is large, this process can impact system availability, costing time and money. Chapter1 : Making Sense of NoSQL
- 14. NoSQL Business Drivers - Agility • The most complex part of building applications using RDBMSs is the process of putting data into and getting data out of the database. If your data has nested and repeated subgroups of data structures, you need to include an object-relational mapping layer. 14 • The responsibility of this layer is to generate the correct combination of INSERT, UPDATE, DELETE, and SELECT SQL statements to move object data to and from the RDBMS persistence layer. This process isn’t simple and is associated with the largest barrier to rapid change when developing new or modifying existing applications. Chapter1 : Making Sense of NoSQL
- 15. NoSQL Business Drivers - Agility • Generally, object-relational mapping requires experienced software developers such as Java Hibernate (or Nhibernate for .Net systems). Even a small change requests can cause slowdowns in development and testing schedules. • Now that you’re familiar with these drivers, you can look at your organization to see how NoSQL solutions might impact these drivers in a positive way to help your business meet the changing demands of today’s competitive marketplace. 15 Chapter1 : Making Sense of NoSQL
- 16. NoSQL Case Studies • LiveJournal’s Memcache • Google’s MapReduce • Google’s Bigtable • Amazon’s Dynamo • MarkLogic 16 Chapter1 : Making Sense of NoSQL
- 17. LiveJournal’s Memcache • LiveJourna is a blogging system. • The most precious resource is the RAM in each web server. • The number of visitors using the site continued to increase. • They had to add more web servers, each with its own separate RAM. • The engineers found ways to keep the results of the most frequently used database queries in RAM, avoiding the expensive cost of rerunning the same SQL queries on their database. • But each web server had its own copy of the query in RAM! 17 Chapter1 : Making Sense of NoSQL
- 18. LiveJournal’s Memcache (Cont.) • To create a distinct “signature” of every SQL query. This signature or hash was a short string that represented a SQL SELECT statement. • By sending a small message between web servers, any web server could ask the other servers if they had a copy of the SQL result already executed. • The concept of large pools of shared-memory servers shared and standardized in the communications protocol between the web front ends (called the memcached protocol). 18 Chapter1 : Making Sense of NoSQL
- 19. Google’s MapReduce • Google shared their process for transforming large volumes of web data content into search indexes using low-cost commodity CPUs. • The initial stages of the transformation are called the map operation. They’re responsible for data extraction, transformation, and filtering of data. • The second layer, the reduce function, is where the results are sorted, combined, and summarized to produce the final result. 19 Chapter1 : Making Sense of NoSQL
- 20. Google’s Bigtable • The motivation behind Bigtable was the need to store results from the web crawlers that extract HTML pages, images, sounds, videos, and other media from the internet. • The resulting dataset was so large that it couldn’t fit into a single relational database. • The solution was neither a full relational database nor a filesystem, but what they called a “distributed storage system” that worked with structured data. • Creating one large table that stored all the data they needed. • Developers didn’t need to worry about the physical location of the data they manipulated. • A table with a billion rows and a million columns. 20 Chapter1 : Making Sense of NoSQL
- 21. Amazon’s Dynamo • The business motivation behind Dynamo was Amazon’s need to create a highly reliable web storefront that supported transactions from around the world 24 hours a day, 7 days a week, without interruption. • In its initial offerings, Amazon used a relational database to support its shopping cart and checkout system with unlimited licenses for RDBMS software. • Amazon found that because key value stores had a simple interface, it was easier to replicate the data and more reliable, extensible, and able to support their 24/7 business model. 21 Chapter1 : Making Sense of NoSQL
- 22. MarkLogic • MarkLogic is company managing large collections of XML documents (contained markup). • MarkLogic defined two types of nodes in a cluster: query and document nodes. • Query nodes receive query requests and coordinate all activities associated with executing a query. • Document nodes contain XML documents and are responsible for executing queries on the documents in the local filesystem. • Query requests are sent to a query node, which distributes queries to each remote server that contains indexed XML documents. • All document matches are returned to the query node. When all document nodes have responded, the query result is then returned. 22 Chapter1 : Making Sense of NoSQL
- 23. MarkLogic (Cont.) • The MarkLogic architecture, moving queries to documents rather than moving documents to the query server, allowed them to achieve linear scalability with petabytes of documents. • Since 2001, MarkLogic has matured into a general-purpose highly scalable document store with support for ACID transactions and fine-grained, role-based access control. • Initially, the primary language of MarkLogic developers was XQuery paired with REST; newer versions support Java as well as other language interfaces. 23 Chapter1 : Making Sense of NoSQL
- 24. 24 Questions? Introduction to Pervasive ComputingChapter1 : Making Sense of NoSQL