Microsoft Database Options
Download as PPTX, PDF2 likes1,089 views
This document discusses data options in the cloud, including database choices like SQL Server, SQL Azure Database, and hosted SQL Server. It covers features of SQL Azure like scalability, high availability, and self-provisioning. Storage options like Windows Azure Storage blobs, tables, and queues are described. SQL Server 2008 R2 improvements in performance, manageability, business intelligence, and master data management are highlighted. Pricing models for SQL Azure and Windows Azure Storage are provided at the end.
Microsoft Database Options
- 1. Data Options in the CloudJoe [email protected]
- 2. Database ChoicesValue Props:Full h/w control – size/scale100% of API surface areaRoll-your-own HA/DR/scaleValue Props:100% of API surface areaRoll-your-own HA/DR/scaleDedicatedOn-premise SQL Server or other s/w on-premise Resource governance @ machine Security @ DB Server/OSValue Props:Auto HA, Fault-ToleranceFriction-free scaleSelf-provisioningSubset of API surface areaResourcesHosted Hosted SQL Server Resource governance @ VM Security @ DB Server/OSSQL Azure Database Virtual DB serverResource governance @ DB Security @ DB/Virtual ServerSharedLow“Friction”/ControlHighSQL Azure V1 targets scenarios that live in the lower left quadrant
- 3. SQL Server 2008 R2Focus AreasImprovements in Scalability and PerformanceEnhanced ManageabilitySelf Service Business IntelligenceMaster Data Management
- 4. Master Data ManagementSingle source of truth for Master DataThin client managementSafeguards of data integrityData versioningImport/Export capabilities
- 5. SQL Server VersionsNew with SQL 2008 R2Datacenter Edition (256 logical processor support)
- 6. Parallel Data Warehouse Edition (100+ TB support)Cloud DevelopmentWEB & CLOUDSthird party cloudweb applicationsdeveloper experienceuse existing skills and toolscomputestoragemanagementmanagementrelational dataconnectivityaccess controlON-PREMISESlob Applicationscomposite applications
- 7. Storage in the CloudWEB & CLOUDSthird party cloudweb applicationsdeveloper experienceuse existing skills and toolsmanagementmanagementconnectivityaccess controlcomputerelational datastorageON-PREMISESlob Applicationscomposite applications
- 8. Why Data in the CloudCapital CostsDBA (or lack thereof)High AvailabilityPerformanceScalability
- 9. Windows Azure StorageThe goal is to allow users and servicesAnywhere at anytime accessStore data for any length of time Scale to store any amount of dataBe confident that the data will not be lost Pay for only what they use/store
- 10. Windows Azure StorageStorageDurableScalable (capacity and throughput)Highly availableRich storage conceptsLarge user data items: blobsService state: tablesService communication: queuesSimple and familiar programming interfacesREST (HTTP and HTTPS).NET accessible
- 11. Fundamental data abstractionsBlobs – provide a simple interface for storing named files along with metadata for the fileTables – provide structured storage. a table is a set of entities, which contain a set of propertiesQueues – provide reliable storage and delivery of messages for an application
- 13. Blob Features and FunctionsStore large objects (up to 200GB)Associate metadata with blobmetadata is <name, value> pairs, up to 8KB per blobset/get with or separate from blob data bitsStandard REST InterfacePutBlobInserts a new blob or overwrites the existing blobGetBlobGet whole blob or a specific rangeDeleteBlob
- 14. Windows Azure TablesProvides structured storageMassively scalable tablesBillions of entities (rows) and TBs of dataAutomatically scales across servers as traffic growsHighly availableAnywhere at anytime access to your dataDurableData is replicated at least 3 timesFamiliar and easy to use programming interfacesADO.NET data services – .NET 3.5 SP1.NET classes and LINQREST - with any platform or language
- 15. Table Storage ConceptsentitytableaccountName =…Email = …usersName =…Email = …sallyPhoto ID =…Date =…photo indexPhoto ID =…Date =…
- 16. Table Data ModelTableA storage account can create many tablesTable name is scoped by accountData is stored in tablesA table is a set of entities (rows)An entity is a set of properties (columns)EntityTwo “key” properties that together are the unique ID of the entity in the tablePartitionKey – enables scalabilityRowKey – uniquely identifies the entity within the partition
- 17. Partition Deyand PartitionsEvery table has a partition keyIt is the first property (column) of your tableUsed to group entities in the table into partitionsA table partition All entities in a table with the same partition key valuePartition key is exposed in the programming model Allows application to control the granularity of the partitions and enable scalability
- 18. Partition ExampleTable partition – all entities in tablewith same partition key valueApplication controls granularity of partitionPartition 1Partition 2
- 19. Purpose of the Partition KeyEntity localityEntities in the same partition will be stored togetherEfficient querying and cache localityEntity group transactions Atomically perform multiple insert/update/delete over entities in same partition in a single transactionTable scalabilityWe monitor the usage patterns of partitionsAutomatically load balance partitionsEach partition can be served by a different storage nodeScale to meet the traffic needs of your table
- 20. Choosing a Partition KeyGranularity of entity group transactionsMake the partition key only as big as you need it for entity group transactionsSpread out load across partitionsMore partitions – makes it easier to automatically balance loadCurrently have one primary indexImportant to use a partition key that is common in your queriesIf partition key is part of queryFast access to retrieve entities within a single partitionIf partition key is not specified in a query Then every partition has to be scanned
- 21. Table Entities and PropertiesEach entity can have up to 255 propertiesMandatory properties for every entity in tablePartition keyRow key All entities have a system maintained versionNo fixed schema for rest of propertiesEach property is stored as a <name, typed value> pairNo schema stored for a table2 entities within the same table can have different propertiesProperties can be the standard .NET types String, binary, bool, DateTime, GUID, int, int64, and double
- 22. Table Programming ModelProvide familiar and easy to use interfacesLeverage your .NET expertiseTable entities are accessed as objects via ADO.NET Data Services – .NET 3.5 SP1LINQ – language Integrated queryRESTful access to table and entitiesInsert/update/delete entities over the tableQuery over tablesGet back a list of structured entities
- 23. Web + Worker Role PatternWeb roleWeb farm that handles request from the internetPush work items onto storage queueWorker roleProcess work item off storage queuePublic internetnmWeb roleWorker roleQLoad balancerCloud storage (tables, blobs, queues)
- 24. Windows Azure QueuesProvide reliable message deliverySimple, asynchronous work dispatchProgramming semantics ensure that a message can be processed at least onceQueues are highly available, durable and performance efficientAccess is provided via REST
- 25. Queue Storage ConceptsMessageQueueAccount128x128, http://…thumbnail jobs256x256, http://…sallyhttp://…photo processing jobshttp://…
- 26. Account, Queues and MessagesAn account can create many queuesQueue name is scoped by the accountAqueue contains messagesNo limit on number of messages stored in a queueA message is stored for at most a week in a queuehttp://<Account>.queue.core.windows.net/<QueueName>MessagesMessage size <= 8 KBTo store larger data, store data in blob/entity storage, and the blob/entity name in the message
- 27. queue programming APIqueuescreate/delete/clear queuesinspect queue lengthmessagesenqueue (queuename, message)dequeue (queueName, invisibility time T)returns the message with a messageIDmakes the message invisible for time Tdelete(queuename, messageID)
- 28. Queue Best PracticesMake message processing idempotentNeed to deal with failuresNo fixed order for dequeue messagesInvisible messages result in out of order processing Use the queue length to scale your workers
- 30. SQL AzureSQL Server Data Services false startFamiliar relational modelUses existing APIs & toolsFriction free provisioning and reduced managementBuilt for the cloud with availability and scaleClear feedback: “I want a SQL database in the cloud”
- 31. Service Provisioning ModelEach account has zero or more serversAzure wide, provisioned in a common portalBilling instrumentEach server has one or more databasesContains metadata about the databases and usageUnit of authenticationUnit of geo-locationGenerated DNS based nameEach database has standard SQL objectsUnit of consistencyUnit of multi-tenancyContains users, tables, views, indices, etc.Most granular unit of billingSKU’s Web edition -1 GBBusiness edition – 10 GB account server database
- 32. ArchitectureShared infrastructure at SQL database and belowRequest routing, security and isolationScalable HA technology provides the glueAutomatic replication and failoverProvisioning, metering and billing infrastructureMachine 5Machine 6Machine 4SQL InstanceSQL InstanceSQL InstanceSQL DBSQL DBSQL DBUserDB1UserDB2UserDB3UserDB4UserDB1UserDB2UserDB3UserDB4UserDB1UserDB2UserDB3UserDB4SDS Provisioning (databases, accounts, roles, …, Metering, and BillingScalability and Availability: Fabric, Failover, Replication, and Load balancingScalability and Availability: Fabric, Failover, Replication, and Load balancing
- 33. Sample of SQL compatibilityIn Scope for V1Tables, indexes and viewsStored ProceduresTriggersConstraintsTable variables, session temp tables (#t)…..Out of Scope for V1Distributed Transactions
- 35. CLR
- 36. Service Broker
- 37. Spatial
- 38. Physical server or catalog DDL and viewsLogical vs. Physical AdministrationCustomer can focus on logical administrationSchema creation and managementQuery optimizationSecurity management (logins, users, roles)Service handles physical managementAutomatically replicated with HA “out of the box”Transparent failover in case of failureLoad balancing of data to ensure SLADBA role places more focus on logical management
- 39. Programming ModelSmall data setsUse a single databaseSame model as on-premise SQL ServerLarge data setsPartition data across many databasesUse parallel fan-out queries to fetch the dataApplication code must be partition aware
- 41. Management ToolingSQL Server 2008 R2 Management StudioSQLCMDThird party/ communityadd-ons
- 42. Data Migration BCPSSISSQL Azure Migration Wizard
- 43. Customer Learning’s from TAP programUse SQL Azure to store metadata and BLOB storage for large filesHighly elastic load patterns in a cost effective way is an industry challengeCombination of different SKUs
- 44. Sync FrameworkSQL AzureLocal SQL DBSync ProcessSync ProcessLocal ComputerMSDN - http://msdn.microsoft.com/en-us/sync/default.aspx
- 45. DemoSQL Azure storage and Sync
- 46. PricingWindows Azure StorageBandwidth: $0.10 in / $0.15 out / GB$0.15/GB stored/monthSQL Azure StorageBandwidth: $0.10 in / $0.15 out / GBWeb edition (1GB): $9.99/monthBusiness edition (10GB): $99.99/month
- 47. Data Options in the [email protected]://www.joeshirey.com