Databricks @ Strata SJ
Download as PPTX, PDF•6 likes•1,616 views
Databricks Cloud simplifies the big data process by providing a unified, cloud-hosted platform powered by Spark, making it easier to manage data pipelines and clusters. It offers instant deployment of managed Spark clusters, an interactive notebook environment, and tools for real-time collaboration and visualization. Founded by the creators of Spark, Databricks aims to streamline the big data journey, minimizing costs and complexities associated with traditional tools.
Databricks @ Strata SJ
- 1. Making Big Data Simple with Databricks Cloud February 2015
- 2. Your company starts a Big Data initiative You have data, you have questions, now what?
- 3. You have to navigate your data pipeline Get a cluster up and running Build a production pipeline Import and explore data
- 4. Problem: the journey is complex and costly Expensive to build and hard to manage Disparate and difficult tools Months of re- engineering to deploy Get a cluster up and running Build a production pipeline Import and explore data
- 5. The Solution: Databricks Cloud A unified, cloud-hosted data platform powered by Spark
- 6. From Challenges to Solutions Clusters are expensive to build and hard to manage Tools are disparate and difficult to integrate and use Long deployment cycles Hosted clusters in the Cloud Unified Interactive Workspace Instant deployment
- 7. What is Databricks Cloud Managed Spark Clusters in the Cloud Notebook Environment 3rd Party Applications Production Pipeline Scheduler
- 8. Managed Spark Clusters in the Cloud No lead time No DevOps Instant Spark clusters The best place to run Spark No expensive hardware
- 9. Explore and visualize data interactively Program in Python, SQL, and Scala Collaborate on-line in real-time Notebook environment
- 10. WYSIWYG builder Interactive plots One-click publishing Custom dashboards
- 11. Schedule production Spark jobs without complicated scripts Production pipeline scheduler
- 12. Real-time Decisions Learn, work and collaborate in a single, easy to use environment Zero Management Focus on your data, not operationsInstant Results From insights to products in just hours Open Platform Use your favorite tools to leverage a growing ecosystem. Why Databricks Cloud
- 13. About Databricks Founded by the creators of Spark in 2013 Largest organization contributing to Spark End-to-end hosted platform, Databricks Cloud
- 14. Get Started Now! Sign up for Databricks Cloud: https://databricks.com/registration
Editor's Notes
- #7: We do this by addressing every one of the three challenges: 1) To alleviate the need to set up and maintain clusters we provide a hosted solution, which makes it very easy to instantiate and manage clusters. 2) To obviate the need to deal with a zoo of tools, we are leveraging Apache Spark which integrates the functionalities of many of the existing big data tools and systems. 3) Finally, to dramatically simplify the use of these tools, we are introducing a web Workspace which allows users to interactively query and visualize the data.
- #9: Notebooks provide interactive query processing and visualization, and support on-line collaboration, thus allowing multiple users conduct jointly data explorations. Currently, notebooks allows users to query and analyze data using python, sql, and scala.
- #10: Notebooks provide interactive query processing and visualization, and support on-line collaboration, thus allowing multiple users conduct jointly data explorations. Currently, notebooks allows users to query and analyze data using python, sql, and scala.
- #11: Once you create one or more notebooks, you can take the most interesting results from these notebooks and create sophisticated dashboards. You can do this through a simple and intuitive dashboard builder, and then publish a dashboard with the click of a click. The dashboards are interactive in that every plot can depend on one or more variables. When these variables are updated, the query behind each plot is automatically re-executed, and the plot is regenerated.
- #12: Finally, the Workspace includes a job launcher that enables you to run arbitrary spark jobs programmatically. For example you can automatically schedule jobs to run when their input changes or periodically.