Experience SQL Server 2017: The Modern Data Platform
- 3. End-to-end mobile BI on any device Choice of platform and language Most secure over the last 7 years 0 20 40 60 80 100 120 140 160 180 200 Vulnerabilities(2010-2016) A fraction of the cost Self-serviceBIperuser Only commercial DB with AI built-in Microsoft Tableau Oracle $120 $480 $2,230 Industry-leading performance 1/10 Most consistent data platform #1 TPC-H performance 1TB, 10TB, 30TB #1 TPC-E performance #1 price/performance T-SQL Java C/C++ C#/VB.NET PHP Node.js Python Ruby R R and Python + in-memory at massive scale S Q L S E R V E R 2 0 1 7 I N D U S T R Y - L E A D I N G P E R F O R M A N C E A N D S E C U R I T Y N O W O N L I N U X A N D D O C K E R Private cloud Public cloud + T-SQL In-memory across all workloads 1/10th the cost of Oracle
- 4. F L E X I B L E , R E L I A B L E D ATA M A N A G E M E N T SQL Server on the platform of your choice Support for RedHat Enterprise Linux (RHEL), Ubuntu, and SUSE Enterprise Linux (SLES) Linux and Windows Docker containers Windows Server / Windows 10 Choice of platform and language
- 5. Windows Linux Developer, Express, Web, Standard, Enterprise Database Engine Integration Services Analysis Services, Reporting Services, MDS, DQS Maximum number of cores Unlimited Unlimited Maximum memory utilized per instance 12 TB 12 TB Maximum database size 524 PB 524 PB Basic OLTP (Basic In-Memory OLTP, Basic operational analytics) Advanced OLTP (Advanced In-Memory OLTP, Advanced operational analytics) Basic high availability (2-node single database failover, non-readable secondary) Advanced HA (Always On - multi-node, multi-db failover, readable secondaries) Security Basic security (Basic auditing, Row-level security, Data masking, Always Encrypted) Advanced security (Transparent Data Encryption) HADR Always On Availability Groups Failover Clustering Replication Data warehousing PolyBase2 Basic data warehousing/data marts (Basic In-Memory ColumnStore, Partitioning, Compression) Advanced data warehousing (Advanced In-Memory ColumnStore) Tools Windows ecosystem: Full-fidelity Management & Dev Tool (SSMS & SSDT), command line tools Linux/OSX/Windows ecosystem: Dev tools (VS Code), DB Admin GUI tool, command line tools Developer Programmability (T-SQL, CLR, Data Types, JSON) Windows Filesystem Integration - FileTable BI & Advanced Analytics Basic Corporate Business Intelligence (Multi-dimensional models, Basic tabular model) Basic “R” integration (Connectivity to R Open, Limited parallelism for ScaleR) Advanced “R” integration (Full parallelism for ScaleR) Hybrid cloud Stretch Database What’s in SQL Server On Linux?
- 6. SQL Server Linux Architecture LibOS (Win API and Kernel) Host Extension mapping to OS system calls (IO, Memory, CPU scheduling) SQLOS (SQLPAL) SQL PAL Everything else System Resource & Latency Sensitive Code Paths SQL Platform Abstraction Layer (SQLPAL) Linux Kernel SQLSERVRSQLAGENT ABI API Linux APIs (mmap, pthread_create, …) Linux Process (Ring 3) Ring 0 Based on Microsoft Research Drawbridge Project
- 7. Docker Containers: What and why? • Docker: Multi-platform container engine based on Linux and Windows Containers. • NOT virtualization • Image • lightweight, stand-alone, executable package that includes everything needed to run a piece of software, including the code, a runtime, libraries, environment variables, and config files • Container • runtime instance of an image—what the image becomes in memory when actually executed • Imagine a world of “database containers”
- 9. The World Leader in TPC-H and TPC-E Performance Benchmarks World’s First Enterprise-Class “Diskless Database” Adaptive Query Processing Query Store Wait Statistics Automatic Tuning result
- 10. Before Spill Spill After Spill detected and feedback generated
- 11. • dm_db_tuning_recommendations Detect • and system corrects Turn on Auto • to “last known good” Reverts back Perfect to help with parameter sniffing
- 13. M I S S I O N C R I T I C A L AVA I L A B I L I T Y O N A N Y P L AT F O R M Always On cross-platform capabilities HA and DR for Linux and Windows “Clusterless” Availability Groups Ultimate HA with OS-level redundancy and low-downtime migration Load balancing of readable secondaries
- 15. In-database Machine Learning Develop Train Deploy Consume Develop, explore and experiment in your favorite IDE Train models with sp_execute_external_ script and save the models in database Deploy your ML scripts with sp_execute_external_ script and predict using the models Make your app/reports intelligent by consuming predictions
- 16. SQL Server Machine Learning Services SQL Server 2016 • Extensibility Framework • R Support (3.2.2) • Microsoft R Server SQL Server 2017 • R Support (3.3.3) • Python Support (3.5.2) • Native Scoring using PREDICT • In-database Package Management Azure SQL DB • Native scoring using PREDICT • R Preview Coming Soon • Python Support in 2018
- 17. SQL Server 2017 – Graph Extensions • Graph – collection of node and edge tables • DDL Extensions – create node/edge tables • Properties associated with Node and Edge tables • All type of indexes are supported on node and edge tables. • Query Language Extensions – New built-in: MATCH, to • support pattern matching and traversals • Tooling and Eco-system
- 18. Session Recommendation Scenario Speaker Attendee Location Follows From Industry From Session Presents Attends Track Topic From From
- 19. -- Find the other sessions that these other users attended other_sessions AS ( SELECT at.name AS attendee_name, s.name AS session_name, COUNT(*) AS other_sessions_attended FROM Conference.Attendee_1 AS at JOIN Conference.SessionAttendee AS sa ON sa.AttendeeID = at.AttendeeID JOIN Conference.Sessions AS s ON s.SessionID = sa.SessionID JOIN OTHER_USR AS ou ON ou.attendeeid = at.attendeeid WHERE s.sessionid <> 101 GROUP BY at.name, s.name ) -- Recommend to the current user the top sessions from the -- list of sessions attended by other users SELECT TOP 10 s.name, COUNT(other_sessions_attended) FROM OTHER_SESSIONS AS os JOIN sessions AS s on s.name = OS.session_name GROUP BY s.name ORDER BY COUNT(other_sessions_attended) DESC; WITH Current_Usr AS ( SELECT AttendeeID = 6 ,SessionID = 101 -- Graph session ,AttendeeCount = 1 ) , -- Identify the other users who also attended the -- graph session Other_Usr AS ( SELECT at.attendeeID, s.sessionid, COUNT(*) AS Attended_by_others FROM Conference.Attendee_1 AS at JOIN Conference.SessionAttendee AS sa ON sa.AttendeeID = at.AttendeeID JOIN Conference.Sessions AS s ON s.SessionID = sa.SessionID JOIN Current_Usr AS cu ON cu.SessionID = sa.SessionID WHERE cu.AttendeeID <> sa.AttendeeID GROUP BY s.sessionid, at.attendeeid ) , Session Recommendations (“Before”)
- 20. SELECT TOP 10 RecommendedSessions.SessionName ,COUNT(*) FROM Sessions ,Attendee ,Attended AS AttendedThis ,Attended AS AttendedOther ,Sessions AS RecommendedSessions WHERE Session.Session_ID = 101 AND MATCH(RecommendedSessions<-(AttendedOther)-Attendee-(AttendedThis)->Sessions) AND (Sessions.SessionName <> RecommendedSessions.SessionName) AND Attendee.attendeeID <> 6 GROUP BY RecommendedSessions.SessionName ORDER BY COUNT(*) DESC; GO Session Recommendations with SQL Graph (“After”)
- 22. In-Memory OLTP Enhancements Columnstore Index Enhancements Resumable Online Index Rebuild SELECT INTO.. ON < Filegroup > New DMVs and enhancements Indirect checkpoint improvements And more…. Smart backup
- 23. Legacy SQL Server instance DMA: Assess and upgrade schema 1. Assess and identify issues 2. Fix issues 3. Upgrade database Data Migration Assistant SQL Server 2017 Database Experimentation Assistant (DEA) to compare perf
- 24. Migrating to SQL Server 2017 from other platforms Oracle SAP ASE DB2 Identify apps for migration Use migration tools and partners Deploy to production SQL Server Migration Assistant Global partner ecosystem AND SQL Server 2017 on Windows SQL Server 2017 on Linux OR
- 25. Oracle SQL SQL DB Azure Database Migration Service Accelerating your journey to the cloud • Streamline database migration to Azure SQL Database (PaaS) • Azure VM target coming • Managed service platform for migrating databases • Migrate SQL Server and third-party databases to Azure SQL Database
- 26. SQL Server on Linux and Docker Leader in TPC-H and TPC-E Performance1 Adaptive Query Processing and Automatic Tuning Availability Groups without Clusters Python and Native PREDICT for Machine Learning Services Graph Database A bunch of engine enhancements Integration Services enhancements Analysis Services enhancements
- 27. Experience SQL Server 2017: Start your journey here What’s New in SQL Server 2017 Download SQL Server 2017 Getting started with SQL Server on Linux Getting started with SQL Server on Docker SQL Server Blog ZDNet Review of SQL Server 2017
- 29. World’s First Scalable Persistent Memory Database Server An Enterprise-Class SQL Server “Diskless Database” HPE Scalable Persistent Memory on the HPE ProLiant DL380 Gen10 Microsoft SQL Server 2017 for Linux SQL Server “Diskless Database” SUSE SLE 12sp3 Operating System Persistent Memory OS technology Up to 45% performance improvement with Data Analytics workloads2 Up to 50% reduction in hardware costs1 Up to 85% reduction in rack space1 1 Based on a comparison between the hardware of the DL380 Gen10 server and the TPC-H @ 1,000GB hardware for the DL380 Gen9 server. See http://tpc.org/3327 2 Based on HPE internal testing on a TPC-H like benchmark, Oct. 2017. © Copyright 2017 Hewlett-Packard Enterprise Development LP. The information contained herein is subject to change without notice. Microsoft and SQL Server are U.S. registered trademarks of Microsoft Corporation. SUSE and SLE are registered trademarks of SUSE. TPC and TPC-H are trademarks of the Transaction Processing Performance Council. Publication no. a00007695enw, April 2017 Watch the video
- 30. What is a Graph Database? • Edges or relationships are first class entities in a Graph Database and can have attributes or properties associated with them. • A single edge can flexibly connect multiple nodes in a Graph Database. • You can express pattern matching and multi-hop navigation queries easily. • Supports OLTP and OLAP (analytics) just like SQL databases.
Editor's Notes
- #4: #1 price/performance in TPC-H non-clustered as of 9/1/2017 - http://www.tpc.org/3323 #1 TPC-H non-clustered benchmark as of 9/1/2017 - http://www.tpc.org/3323 #1 TPC-E performance as of 9/1/2017 - http://www.tpc.org/4075
- #5: Last but not least, customers need flexibility when it comes to the choice of platform, programming languages & data infrastructure to get from the most from their data. Why? In most IT environments, platforms, technologies and skills are as diverse as they have ever been, the data platform of the future needs to you to build intelligent applications on any data, any platform, any language on premises and in the cloud. SQL Server manages your data, across platforms, with any skills, on-premises & cloud Our goal is to meet you where you are with on any platform, anywhere with the tools and languages of your choice. SQL now has support for Windows, Linux & Docker Containers. It allows you to leverage the language of your choice for advanced analytics – R & Python.
- #12: Want to hear about a performance problem the next morning vs getting called in the middle of the night?
- #16: -SQL Server is a platform for operationalizing machine learning. -What is the Best practice to work against SQL Server with ML? -Use compute context and work against SQL Server all the way from feature engineering, experimentation/evaluation to operationalization of models.
- #18: One graph per database. Nodes and edges may have properties associated to them. Edge tables can be used to model many-to-many relationships.
- #24: [this slide contains animations] In assessments, Data Migration Assistant (DMA) automates the potentially overwhelming process of checking database schema and static objects for potential breaking changes from prior versions. DMA also offers performance and reliability recommendations on the target server. [click] The first phase is to use DMA to assess the legacy database and identify issues. [click] In the second phase, issues are fixed. The first and second phases are repeated until all issues are addressed. [click] Finally, the database is upgraded to SQL Server 2017. For more information, see: https://blogs.msdn.microsoft.com/datamigration/2016/08/26/data-migration-assistant-how-to-assess-your-on-premises-sql-server-instance/
- #25: Source: https://docs.microsoft.com/en-us/sql/ssma/sql-server-migration-assistant SAP ASE was formerly known as SAP Sybase ASE/Sybase.
- #26: Source: https://azure.microsoft.com/en-gb/campaigns/database-migration/ As organizations look to optimize their IT infrastructure so that they have more time and resources to focus on business transformation, Microsoft is committed to helping to accelerate these initiatives. Microsoft have announced that a new migration service is coming to Azure to streamline customers’ journey to the cloud. This service will streamline the tasks required to move existing competitive and SQL Server databases to Azure. Deployment options will include Azure SQL Database and SQL Server in Azure VM. Managed service platform for migrating databases. Azure SQL DB and managed instance as targets. Competitive DBs—Oracle and more. Meets enterprise nonfunctional requirements (NFRs)—compliance, security, costs, and so on. Talk about the technical details: Source ->Target. Secure. Feature parity with competitors. Zero data loss and near zero downtime migration with the Azure platform service.
- #27: 1 #1 non-clustered 1TB TPC-H (http://www.tpc.org/3331) , #1 non-clustered 10TB TPC-H (http://www.tpc.org/3329) , #1 non-clustered 30TB TPC-H (http://www.tpc.org/3321) , and #1 TPC-E (http://www.tpc.org/4081) as of November 20th, 2017.