SlideShare a Scribd company logo
Marco Parenzan
Senior Solution Architect @ beanTech
Microsoft Azure MVP
Community Lead 1nn0va // Pordenone
1nn0va After Hour
https://bit.ly/1nn0va-video
Linkedin: https://www.linkedin.com/in/marcoparenzan/
Marco Parenzan
Deploy Microsoft Azure Data Solutions
Da SQL Server a Azure SQL e oltre: le opportunità per i dati in
Azure
Dati relazionali in Azure
Dati non relazionali
Dati blob in Azure
Architetture a Messaggi, Comandi ed Eventi in Azure
Big Data in Azure
Gestire i dati dell'IoT in Azure
Servizi Cognitivi per i dati
Sicurezza
Agenda
Deploy Microsoft Azure Data Solutions
• Polyglot Persistence
• …non «spendere per spendere»…
• No more SQL...only! 
• Microservizi
• Ogni servizio porta con sé uno storage
• «Next,Next,Next,Next,...»
• Operations & DevOps
• Quanto è «dato» e quanto è «servizio»?
• Un confine sempre più labile
• Questo è tutto ciò che è il cloud
Alcuni temi
Deploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data Solutions
…musing…
https://azureperiodic.data3.com/
Deploy Microsoft Azure Data Solutions
Pricing Azure Storage Blobs Pricing | Microsoft Azure
Files and API
Events
External Tables
Azure Blobs
Storage
Account
IoT Hub
Devices
Events
Ingest
The batch point of view...
With no any specific request...
what is IoT all about?
Efficiency Anomalies
Batch Streaming
…musing…
https://news.microsoft.com/ignite-
march-2021-book-of-news/
https://techcommunity.microsoft.com/t5/azure-synapse-analytics/ignite-2021-announcements-
accelerate-time-to-insight-with-azure/ba-p/2912147
IoT
Central
IoT
Hub
Event
Hub
«Down the event stream»
…musing…
DEMO
• Azure Stream Analytics
• Azure Container Apps & co
• Apache Spark (and again Synapse)
Event Processing
…musing…
DEMO
• ML.NET
• Spark
• ADX (and again Synapse)
• Notebooks
• Azure Functions
• Cognitive Services
• Anomaly Detector
• Custom Vision
Anomaly Detection
…musing…
DEMO
• Azure love Blobs
• Developers love Blobs
• Developers love services that handles Blobs as first class citizens
• No relational
• No file system
Lesson learned
Deploy Microsoft Azure Data Solutions
Necessità: complex messages/entities, no mapping
{
"UserId": "123456",
"UserName": "Mario Rossi",
"EMail": "mario.rossi@somewhere.it",
"StoreInfo": {
"PreferredStore": {
"PreferredStoreId": "12876",
"PreferredStoreName": «Dovunque di Corso Vittorio"
}
}
}
{
"UserId": "123456",
"Reservations": [
{
"LastUpdate": "",
"CodiceArticolo": "",
"StoreId": "",
"AvailableParts": 4,
"ProductWarehouseHash": "638AB6D83H"
}
]
}
{
"CodiceArticolo": "AK2457",
"Descrizione": "Prodotto bello",
"GiacenzaAttuale": 100,
"Negozi": {
"CodiceNegozio": {
"GiacenzaAttuale": 20,
"LastUpdate": "2021-04-26T10:20:30"
}
},
"LastUpdate": "2021-04-26T10:20:30"
}
{
"CodiceArticolo": "AK2457",
"Descrizione": «Prodotto",
"Codici": {
"B3812": {
"Descrizione": "Prodotto bello",
"Usage": "blablabla"
}
},
"DistintaBase": [
{ "CodiceParte": "AK2457-P1" },
{ "CodiceParte": "AK2457-P2" }
],
"LastUpdate": "2021-04-26T10:20:30"
}
Column-family
Document
Graph
Turnkey global distribution
Elastic scale out
of storage & throughput
Guaranteed low latency at the 99th percentile
Comprehensive SLAs
Five well-defined consistency models
Table API
Key-value
Cosmos DB’s API for
MongoDB
Azure Cosmos Db
Modeling challenge #1: To embed or reference?
{
"menuID": 1,
"menuName": "Lunch menu",
"items": [
{"ID": 1, "ItemName": "hamburger", "ItemDescription":...}
{"ID": 2, "ItemName": "cheeseburger", "ItemDescription":...}
]
}
{
"menuID": 1,
"menuName": "Lunch menu",
"items": [
{"ID": 1}
{"ID": 2}
]
}
{"ID": 1, "ItemName": “hamburger", "ItemDescription":...}
{"ID": 2, "ItemName": “cheeseburger", "ItemDescription":...}
hash(City ID)
Beijing
Seattle
Paris
Redmon
d
Shanghai
Tianjin
Berlin
Austi
n
Range 1 Range 2 Range 5
RU’s divided among Partitions
Multi-Master – Read/Write in any region
Benefits
• Write scalability around the world
• Low latency (<10ms P99 for 1kb document)
writes around the world
• 99.999% High Availability around the world
• Well-defined consistency models
• Automatic conflict management
Common Change Feed Scenarios
Request Units
Each request consumes # of RU
Approx. 1 RU = 1 read of 1 KB document
Approx. 5 RU = 1 write of a 1KB document
Query: Depends on query & documents involved
GET
POST
PUT
Query
…
=
=
=
=
Operation Type # Requests per sec # RU's per Request RU's Needed
Write Single
Document 10,000 10 100,000
Top Query #1 700 100 70,000
Top Query #2 200 100 20,000
Top Query #3 100 100 10,000
Total RU/s 200,000 RU/s
Estimating Required RU’s
Storage Cost
Avg Record Size (KB) 1
Number of Records 100,000,000
Total Storage (GB) 100
Monthly Cost per GB $0.25
Expected Monthly Cost for Storage $25.00
Throughput Cost
Operation Type
Number of Requests per
sec Avg RU's per Request RU's Needed
Create 100 5 500
Read 400 1 400
Total RU/sec 900
Hourly Cost per 100 RU/sec $0.008
Monthly Cost per 100 RU/sec $6.00
Expected Monthly Cost for Throughput $54.00
Total Monthly Cost
[Total Monthly Cost] = [Monthly Cost for Storage] + [Monthly Cost for Throughput]
= $25 + $54
= $79 per month
* pricing may vary by region; for up-to-date pricing, see: https://azure.microsoft.com/pricing/details/cosmos-db/
Pricing Example
Service Bus
Architettura generica per User Story
• Microservizio responsabile del dominio
applicativo e della sua consistenza e
persistenza
• API di frontend per gestire le richieste
delle singole user stories specifiche verso
il microservizio
• Gestione in memory con Redis
• In caso di fail lettura da Cosmos
• In caso di update
• Aggiornamento di Redis
• Accodamento in serviceBus
• Batch aggiorna Cosmos per abbassare RU
API #1
Micro Service
Cosmos DB
REDIS
Caching
Update Batch
API #n
Read
Write
Command Read
Invoke
Invalidate Cache
Write
Invoke
Read
Command
…musing…
NO DEMO, SORRY!
• Azure love No SQL and Cosmos DB
• Developers love No SQL and Json
• Developer hates(?) Cosmos Db pricing?
• Developers loves (fuffa style ) microservices
• Everything has solution...
Lesson learned
Deploy Microsoft Azure Data Solutions
• È tutto Azure!
• Le dashboard, se si può, si fanno in Power BI (8€/utente/mese, ricordatevi,
anche solo per visualizzare)
• Governance dello sviluppo
• Premium Capacity
• Developers
• Deployment
BI? Power BI? Cosa c’entra con Azure?
• Developer maleducati
• Power BI developer MOLTO maleducati
Scenario
• PowerShell
• Azure DevOps
• Git non è pbix-friendly
• Ma è comunque un repo versionato
• E le pipeline sono un ottimo strumento di automation
Ops, Deployments
…musing…
DEMO
• DR of a DataWarehouse: big problem
• BIOps is not a practice
Lesson learned
Deploy Microsoft Azure Data Solutions
SQL virtual machines Managed instances
Azure SQL
Databases
• SQL Server surface area
(vast majority)
• Native virtual network
support
• Fully managed service
• SQL Server and OS server
access
• Expansive SQL and OS
version support
• Automated manageability
features for SQL Server
• Hyperscale storage (up to
100TB)
• Serverless compute
• Fully managed service
• Resource sharing between
multiple databases to
price optimize
• Simplified performance
management for multiple
databases
• Fully managed service
Azure SQL Database — Everything built-in
•Business continuity
High availability
Automated backups
Long term backup
retention
Geo-replication
•Scale
Advanced security
Automatic tuning
Built-in monitoring
Built-in intelligence
Azure SQL: multi-faceted service
On-demand flexible scale
Operate at the true rhythm of
your business
Fully managed & intelligent
Focus on your applications, not
your infrastructure
Cost-effective
Pay for performance. Period.
Adapts compute resources to the
workload without sacrificing
performance
Automatically pauses and resumes
Fully-managed and intelligent
database service
Built-in 99.99% availability
Pay only for compute resources you
consume, on a per-second basis
Further optimize costs with configurable
compute thresholds
Best for unpredictable and intermittent
workloads on single databases, such as:
Dev/test E-commerce
Line of Business
Deploy Microsoft Azure Data Solutions
COnclusions
Marco Parenzan
marcoparenzan/CodeGenerationForAzureWithDotNet (github.com)

More Related Content

What's hot (20)

PDF
Event driven workloads on Kubernetes with KEDA
Nilesh Gule
 
PDF
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
SeungYong Oh
 
PPTX
Azure DevOps - Minicurso Gratuito Azure na Prática #3
Renato Groff
 
PDF
Container orchestration k8s azure kubernetes services
Rajesh Kolla
 
PPTX
Serverless real use cases and best practices: Asavari Tayal, Microsoft, Serve...
iguazio
 
PPTX
AKS
girish goudar
 
POTX
Mobile 3: Launch Like a Boss!
MongoDB
 
PDF
Cncf event driven autoscaling with keda
JurajHantk
 
PPTX
Azure and deep learning
David Giard
 
PPTX
Windows Azure IaaS and Hybrid
Mike Martin
 
PPTX
Building a Real-Time IoT monitoring application with Azure
Davide Mauri
 
PDF
Virtualizing Apache Spark with Justin Murray
Databricks
 
PDF
[Serverless OpenHack Tokyo] Azure Serverless (English)
Naoki (Neo) SATO
 
PDF
Dok Talks #111 - Scheduled Scaling with Dask and Argo Workflows
DoKC
 
PDF
Kubernetes in Azure
Karl Ots
 
PPTX
Azure Service Fabric Mesh
Udaiappa Ramachandran
 
PPTX
Infrastructure as Code on Azure - NET Conf AR v2018
Victor Silva
 
PPTX
Improving CloudStack for operators
ShapeBlue
 
PPTX
OpenStack + Nano Server + Hyper-V + S2D
Alessandro Pilotti
 
PDF
Azure kubernetes service (aks) part 3
Nilesh Gule
 
Event driven workloads on Kubernetes with KEDA
Nilesh Gule
 
Kubernetes Forum Seoul 2019: Re-architecting Data Platform with Kubernetes
SeungYong Oh
 
Azure DevOps - Minicurso Gratuito Azure na Prática #3
Renato Groff
 
Container orchestration k8s azure kubernetes services
Rajesh Kolla
 
Serverless real use cases and best practices: Asavari Tayal, Microsoft, Serve...
iguazio
 
Mobile 3: Launch Like a Boss!
MongoDB
 
Cncf event driven autoscaling with keda
JurajHantk
 
Azure and deep learning
David Giard
 
Windows Azure IaaS and Hybrid
Mike Martin
 
Building a Real-Time IoT monitoring application with Azure
Davide Mauri
 
Virtualizing Apache Spark with Justin Murray
Databricks
 
[Serverless OpenHack Tokyo] Azure Serverless (English)
Naoki (Neo) SATO
 
Dok Talks #111 - Scheduled Scaling with Dask and Argo Workflows
DoKC
 
Kubernetes in Azure
Karl Ots
 
Azure Service Fabric Mesh
Udaiappa Ramachandran
 
Infrastructure as Code on Azure - NET Conf AR v2018
Victor Silva
 
Improving CloudStack for operators
ShapeBlue
 
OpenStack + Nano Server + Hyper-V + S2D
Alessandro Pilotti
 
Azure kubernetes service (aks) part 3
Nilesh Gule
 

Similar to Deploy Microsoft Azure Data Solutions (20)

PPTX
Azure PaaS databases
Gianluca Hotz
 
PPTX
Perchè un programmatore ama anche i database NoSQL
Marco Parenzan
 
PPTX
Session: Modern Data WareHouse
Karina Matos
 
PPTX
Cosmos db
Martino Bordin
 
PPTX
Tour de France Azure PaaS 3/7 Stocker des informations
Alex Danvy
 
PPTX
What's new with Azure Sql Database
Marco Parenzan
 
PPTX
Azure data platform overview
James Serra
 
PPTX
Innovations of .NET and Azure (Recaps of Build 2017 selected sessions)
Jeff Chu
 
PDF
Azure Data services
Rajesh Kolla
 
PPTX
Hurtownia danych jako serwis w chmurze, na przykładzie azure sql data warehou...
Alicja Folga
 
PPTX
Azure Comsos DB Use Cases
Nicholas Vossburg
 
PPTX
Azure Databases with IaaS
Kellyn Pot'Vin-Gorman
 
PPTX
Azure data platform overview
Alessandro Melchiori
 
PPTX
Sql Start! 2020 - SQL Server Lift & Shift su Azure
Marco Obinu
 
PDF
Customer Migration to Azure SQL Database_2024.pdf
George Walters
 
PPTX
Azure database services for PostgreSQL and MySQL
Amit Banerjee
 
PPTX
L'architettura di classe enterprise di nuova generazione - Massimo Brignoli
Data Driven Innovation
 
PDF
Microsoft Azure: Applications Migration Paths
Lorenzo Barbieri
 
PPTX
Azure CosmosDb - Where we are
Marco Parenzan
 
PPTX
Azure Database Services for MySQL PostgreSQL and MariaDB
Nicholas Vossburg
 
Azure PaaS databases
Gianluca Hotz
 
Perchè un programmatore ama anche i database NoSQL
Marco Parenzan
 
Session: Modern Data WareHouse
Karina Matos
 
Cosmos db
Martino Bordin
 
Tour de France Azure PaaS 3/7 Stocker des informations
Alex Danvy
 
What's new with Azure Sql Database
Marco Parenzan
 
Azure data platform overview
James Serra
 
Innovations of .NET and Azure (Recaps of Build 2017 selected sessions)
Jeff Chu
 
Azure Data services
Rajesh Kolla
 
Hurtownia danych jako serwis w chmurze, na przykładzie azure sql data warehou...
Alicja Folga
 
Azure Comsos DB Use Cases
Nicholas Vossburg
 
Azure Databases with IaaS
Kellyn Pot'Vin-Gorman
 
Azure data platform overview
Alessandro Melchiori
 
Sql Start! 2020 - SQL Server Lift & Shift su Azure
Marco Obinu
 
Customer Migration to Azure SQL Database_2024.pdf
George Walters
 
Azure database services for PostgreSQL and MySQL
Amit Banerjee
 
L'architettura di classe enterprise di nuova generazione - Massimo Brignoli
Data Driven Innovation
 
Microsoft Azure: Applications Migration Paths
Lorenzo Barbieri
 
Azure CosmosDb - Where we are
Marco Parenzan
 
Azure Database Services for MySQL PostgreSQL and MariaDB
Nicholas Vossburg
 
Ad

More from Marco Parenzan (20)

PPTX
Azure IoT Central per lo SCADA engineer
Marco Parenzan
 
PPTX
Azure Hybrid @ Home
Marco Parenzan
 
PPTX
Static abstract members nelle interfacce di C# 11 e dintorni di .NET 7.pptx
Marco Parenzan
 
PPTX
Azure Synapse Analytics for your IoT Solutions
Marco Parenzan
 
PPTX
Power BI Streaming Data Flow e Azure IoT Central
Marco Parenzan
 
PPTX
Power BI Streaming Data Flow e Azure IoT Central
Marco Parenzan
 
PPTX
Power BI Streaming Data Flow e Azure IoT Central
Marco Parenzan
 
PPTX
Developing Actors in Azure with .net
Marco Parenzan
 
PPTX
Math with .NET for you and Azure
Marco Parenzan
 
PPTX
Power BI data flow and Azure IoT Central
Marco Parenzan
 
PPTX
.net for fun: write a Christmas videogame
Marco Parenzan
 
PPTX
Building IoT infrastructure on edge with .net, Raspberry PI and ESP32 to conn...
Marco Parenzan
 
PPTX
Anomaly Detection with Azure and .NET
Marco Parenzan
 
PPTX
Deep Dive Time Series Anomaly Detection in Azure with dotnet
Marco Parenzan
 
PPTX
Azure IoT Central
Marco Parenzan
 
PPTX
Anomaly Detection with Azure and .net
Marco Parenzan
 
PPTX
Code Generation for Azure with .net
Marco Parenzan
 
PPTX
Running Kafka and Spark on Raspberry PI with Azure and some .net magic
Marco Parenzan
 
PPTX
Time Series Anomaly Detection with Azure and .NETT
Marco Parenzan
 
PPTX
Code Generation for Azure with .net
Marco Parenzan
 
Azure IoT Central per lo SCADA engineer
Marco Parenzan
 
Azure Hybrid @ Home
Marco Parenzan
 
Static abstract members nelle interfacce di C# 11 e dintorni di .NET 7.pptx
Marco Parenzan
 
Azure Synapse Analytics for your IoT Solutions
Marco Parenzan
 
Power BI Streaming Data Flow e Azure IoT Central
Marco Parenzan
 
Power BI Streaming Data Flow e Azure IoT Central
Marco Parenzan
 
Power BI Streaming Data Flow e Azure IoT Central
Marco Parenzan
 
Developing Actors in Azure with .net
Marco Parenzan
 
Math with .NET for you and Azure
Marco Parenzan
 
Power BI data flow and Azure IoT Central
Marco Parenzan
 
.net for fun: write a Christmas videogame
Marco Parenzan
 
Building IoT infrastructure on edge with .net, Raspberry PI and ESP32 to conn...
Marco Parenzan
 
Anomaly Detection with Azure and .NET
Marco Parenzan
 
Deep Dive Time Series Anomaly Detection in Azure with dotnet
Marco Parenzan
 
Azure IoT Central
Marco Parenzan
 
Anomaly Detection with Azure and .net
Marco Parenzan
 
Code Generation for Azure with .net
Marco Parenzan
 
Running Kafka and Spark on Raspberry PI with Azure and some .net magic
Marco Parenzan
 
Time Series Anomaly Detection with Azure and .NETT
Marco Parenzan
 
Code Generation for Azure with .net
Marco Parenzan
 
Ad

Recently uploaded (20)

PPTX
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
PDF
Revenue streams of the Wazirx clone script.pdf
aaronjeffray
 
PPTX
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
PDF
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
PPTX
Migrating Millions of Users with Debezium, Apache Kafka, and an Acyclic Synch...
MD Sayem Ahmed
 
PDF
Linux Certificate of Completion - LabEx Certificate
VICTOR MAESTRE RAMIREZ
 
PPTX
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
PPTX
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
PDF
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
PPTX
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
PDF
4K Video Downloader Plus Pro Crack for MacOS New Download 2025
bashirkhan333g
 
PPTX
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
PDF
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
PDF
Online Queue Management System for Public Service Offices in Nepal [Focused i...
Rishab Acharya
 
PPTX
Homogeneity of Variance Test Options IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
PDF
Thread In Android-Mastering Concurrency for Responsive Apps.pdf
Nabin Dhakal
 
PDF
The 5 Reasons for IT Maintenance - Arna Softech
Arna Softech
 
PDF
Digger Solo: Semantic search and maps for your local files
seanpedersen96
 
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
iTop VPN With Crack Lifetime Activation Key-CODE
utfefguu
 
Revenue streams of the Wazirx clone script.pdf
aaronjeffray
 
Home Care Tools: Benefits, features and more
Third Rock Techkno
 
Wondershare PDFelement Pro Crack for MacOS New Version Latest 2025
bashirkhan333g
 
Migrating Millions of Users with Debezium, Apache Kafka, and an Acyclic Synch...
MD Sayem Ahmed
 
Linux Certificate of Completion - LabEx Certificate
VICTOR MAESTRE RAMIREZ
 
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
4K Video Downloader Plus Pro Crack for MacOS New Download 2025
bashirkhan333g
 
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
Online Queue Management System for Public Service Offices in Nepal [Focused i...
Rishab Acharya
 
Homogeneity of Variance Test Options IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
Thread In Android-Mastering Concurrency for Responsive Apps.pdf
Nabin Dhakal
 
The 5 Reasons for IT Maintenance - Arna Softech
Arna Softech
 
Digger Solo: Semantic search and maps for your local files
seanpedersen96
 

Deploy Microsoft Azure Data Solutions

  • 2. Senior Solution Architect @ beanTech Microsoft Azure MVP Community Lead 1nn0va // Pordenone 1nn0va After Hour https://bit.ly/1nn0va-video Linkedin: https://www.linkedin.com/in/marcoparenzan/ Marco Parenzan
  • 4. Da SQL Server a Azure SQL e oltre: le opportunità per i dati in Azure Dati relazionali in Azure Dati non relazionali Dati blob in Azure Architetture a Messaggi, Comandi ed Eventi in Azure Big Data in Azure Gestire i dati dell'IoT in Azure Servizi Cognitivi per i dati Sicurezza Agenda
  • 6. • Polyglot Persistence • …non «spendere per spendere»… • No more SQL...only!  • Microservizi • Ogni servizio porta con sé uno storage • «Next,Next,Next,Next,...» • Operations & DevOps • Quanto è «dato» e quanto è «servizio»? • Un confine sempre più labile • Questo è tutto ciò che è il cloud Alcuni temi
  • 11. Pricing Azure Storage Blobs Pricing | Microsoft Azure Files and API Events External Tables Azure Blobs
  • 13. With no any specific request... what is IoT all about? Efficiency Anomalies Batch Streaming
  • 18. • Azure Stream Analytics • Azure Container Apps & co • Apache Spark (and again Synapse) Event Processing
  • 20. • ML.NET • Spark • ADX (and again Synapse) • Notebooks • Azure Functions • Cognitive Services • Anomaly Detector • Custom Vision Anomaly Detection
  • 22. • Azure love Blobs • Developers love Blobs • Developers love services that handles Blobs as first class citizens • No relational • No file system Lesson learned
  • 24. Necessità: complex messages/entities, no mapping { "UserId": "123456", "UserName": "Mario Rossi", "EMail": "[email protected]", "StoreInfo": { "PreferredStore": { "PreferredStoreId": "12876", "PreferredStoreName": «Dovunque di Corso Vittorio" } } } { "UserId": "123456", "Reservations": [ { "LastUpdate": "", "CodiceArticolo": "", "StoreId": "", "AvailableParts": 4, "ProductWarehouseHash": "638AB6D83H" } ] } { "CodiceArticolo": "AK2457", "Descrizione": "Prodotto bello", "GiacenzaAttuale": 100, "Negozi": { "CodiceNegozio": { "GiacenzaAttuale": 20, "LastUpdate": "2021-04-26T10:20:30" } }, "LastUpdate": "2021-04-26T10:20:30" } { "CodiceArticolo": "AK2457", "Descrizione": «Prodotto", "Codici": { "B3812": { "Descrizione": "Prodotto bello", "Usage": "blablabla" } }, "DistintaBase": [ { "CodiceParte": "AK2457-P1" }, { "CodiceParte": "AK2457-P2" } ], "LastUpdate": "2021-04-26T10:20:30" }
  • 25. Column-family Document Graph Turnkey global distribution Elastic scale out of storage & throughput Guaranteed low latency at the 99th percentile Comprehensive SLAs Five well-defined consistency models Table API Key-value Cosmos DB’s API for MongoDB Azure Cosmos Db
  • 26. Modeling challenge #1: To embed or reference? { "menuID": 1, "menuName": "Lunch menu", "items": [ {"ID": 1, "ItemName": "hamburger", "ItemDescription":...} {"ID": 2, "ItemName": "cheeseburger", "ItemDescription":...} ] } { "menuID": 1, "menuName": "Lunch menu", "items": [ {"ID": 1} {"ID": 2} ] } {"ID": 1, "ItemName": “hamburger", "ItemDescription":...} {"ID": 2, "ItemName": “cheeseburger", "ItemDescription":...}
  • 28. Multi-Master – Read/Write in any region Benefits • Write scalability around the world • Low latency (<10ms P99 for 1kb document) writes around the world • 99.999% High Availability around the world • Well-defined consistency models • Automatic conflict management
  • 29. Common Change Feed Scenarios
  • 30. Request Units Each request consumes # of RU Approx. 1 RU = 1 read of 1 KB document Approx. 5 RU = 1 write of a 1KB document Query: Depends on query & documents involved GET POST PUT Query … = = = =
  • 31. Operation Type # Requests per sec # RU's per Request RU's Needed Write Single Document 10,000 10 100,000 Top Query #1 700 100 70,000 Top Query #2 200 100 20,000 Top Query #3 100 100 10,000 Total RU/s 200,000 RU/s Estimating Required RU’s
  • 32. Storage Cost Avg Record Size (KB) 1 Number of Records 100,000,000 Total Storage (GB) 100 Monthly Cost per GB $0.25 Expected Monthly Cost for Storage $25.00 Throughput Cost Operation Type Number of Requests per sec Avg RU's per Request RU's Needed Create 100 5 500 Read 400 1 400 Total RU/sec 900 Hourly Cost per 100 RU/sec $0.008 Monthly Cost per 100 RU/sec $6.00 Expected Monthly Cost for Throughput $54.00 Total Monthly Cost [Total Monthly Cost] = [Monthly Cost for Storage] + [Monthly Cost for Throughput] = $25 + $54 = $79 per month * pricing may vary by region; for up-to-date pricing, see: https://azure.microsoft.com/pricing/details/cosmos-db/ Pricing Example
  • 33. Service Bus Architettura generica per User Story • Microservizio responsabile del dominio applicativo e della sua consistenza e persistenza • API di frontend per gestire le richieste delle singole user stories specifiche verso il microservizio • Gestione in memory con Redis • In caso di fail lettura da Cosmos • In caso di update • Aggiornamento di Redis • Accodamento in serviceBus • Batch aggiorna Cosmos per abbassare RU API #1 Micro Service Cosmos DB REDIS Caching Update Batch API #n Read Write Command Read Invoke Invalidate Cache Write Invoke Read Command
  • 35. • Azure love No SQL and Cosmos DB • Developers love No SQL and Json • Developer hates(?) Cosmos Db pricing? • Developers loves (fuffa style ) microservices • Everything has solution... Lesson learned
  • 37. • È tutto Azure! • Le dashboard, se si può, si fanno in Power BI (8€/utente/mese, ricordatevi, anche solo per visualizzare) • Governance dello sviluppo • Premium Capacity • Developers • Deployment BI? Power BI? Cosa c’entra con Azure?
  • 38. • Developer maleducati • Power BI developer MOLTO maleducati Scenario
  • 39. • PowerShell • Azure DevOps • Git non è pbix-friendly • Ma è comunque un repo versionato • E le pipeline sono un ottimo strumento di automation Ops, Deployments
  • 41. • DR of a DataWarehouse: big problem • BIOps is not a practice Lesson learned
  • 43. SQL virtual machines Managed instances Azure SQL Databases • SQL Server surface area (vast majority) • Native virtual network support • Fully managed service • SQL Server and OS server access • Expansive SQL and OS version support • Automated manageability features for SQL Server • Hyperscale storage (up to 100TB) • Serverless compute • Fully managed service • Resource sharing between multiple databases to price optimize • Simplified performance management for multiple databases • Fully managed service
  • 44. Azure SQL Database — Everything built-in •Business continuity High availability Automated backups Long term backup retention Geo-replication •Scale Advanced security Automatic tuning Built-in monitoring Built-in intelligence
  • 46. On-demand flexible scale Operate at the true rhythm of your business Fully managed & intelligent Focus on your applications, not your infrastructure Cost-effective Pay for performance. Period. Adapts compute resources to the workload without sacrificing performance Automatically pauses and resumes Fully-managed and intelligent database service Built-in 99.99% availability Pay only for compute resources you consume, on a per-second basis Further optimize costs with configurable compute thresholds Best for unpredictable and intermittent workloads on single databases, such as: Dev/test E-commerce Line of Business

Editor's Notes

  • #26: Azure Cosmos DB offers the first globally distributed, multi-model database service for building planet scale apps. It’s been powering Microsoft’s internet-scale services for years, and now it’s ready to launch yours. You can add Azure locations to your database anywhere across the world, at any time, with a single click. Cosmos DB will seamlessly replicate your data and make it highly available.   Cosmos DB allows you to scale throughput and storage elastically, and globally! You only pay for the throughput and storage you need – anywhere in the world, at any time.
  • #28: 27
  • #31: The number of RU’s each operation consumes depends on many factors which include: Document size Number of indexed fields Type of indexes Consistency model choice Not all queries will consume equal numbers of RU’s. Some operations are more computationally complex or require scans through more documents and therefore use more RU’s.
  • #45: Business continuity enables your business to continue operating in the face of disruption, particularly to its computing infrastructure. High availability of Azure SQL Database guarantees your databases are up and running 99.99% of the time, no need to worry about maintenance/downtimes. 2 read replicas - GP 3 replicas, 1 read-scale replica, zone-redundant HA - BC Primary read/write replica + up to 4 read replicas - HS Automated backups are created and use Azure read-access geo-redundant storage (RA-GRS) to provide geo-redundancy. Also Point in Time restore Long term backup retention enables you to store specific full databases for up to 10 years. Geo-replication by creating readable replicas of your database in the same or different data center (region). You can manually failover to readable replicas Also Auto-failover groups allows an application to recover in case of a data center outage Scale by easily adding more resources (CPU, memory, storage) without long provisioning. Advanced security detects threats and vulnerabilities in your databases and enables you to secure your data. Automatic tuning analyzes your workload and provides you the recommendations that can optimize performance of your applications by adding indexes, removing unused indexes, and automatically fixing the query plan issues. Built-in monitoring capabilities enable you to get the insights into performance of your databases and workload, and troubleshoot the performance issues. Built-in intelligence automatically identifies the potential issues in your workload and provides you the recommendations that can help you to fix the problems.
  • #47: Azure SQL Database serverless is our answer to cost-effectively resourcing these types of unpredictable and intermittent workloads. It is a dynamically scaling, on-demand version of Azure SQL Database that eliminates the complexity of resourcing and managing unpredictable workloads. Simply create a serverless database and connect your application, with no upfront resource configuration required. Serverless SQL databases automatically pause, resume and scale compute based upon your app’s requirements, making them particularly cost effective for variable or unpredictable workloads – or when you’re simply unsure of your requirements. With resources available on-demand, serverless SQL databases optimize costs with pay per-second billing that aligns with the app’s performance, so you only pay for the compute resources you use . Built upon the SQL Server architecture, serverless SQL databases are fully-managed, always up to date and highly available with a 99.99% uptime guarantee. Combined with built-in intelligence to optimize database performance and security, serverless SQL Databases help you be more productive so you can focus more on what you do best, building great apps faster and more cost effectively.