MongoDB Milan MUG June 2015 MongoDB + Azure + Siope OpenData
2 likes1,777 views
The document discusses the integration of Azure and MongoDB, highlighting features such as replica sets and sharding for scalability. It includes practical demonstrations and links to resources, templates, and tools for using Azure with MongoDB effectively. The information is aimed at enhancing performance and availability for developers and data management in real-time scenarios.
MongoDB Milan MUG June 2015 MongoDB + Azure + Siope OpenData
- 1. Azure +MONGODB MUG Milan June 2015
- 2. Ivan Fioravanti 4ward CTO co-founder Azure & MongoDB Fan WHO? http://www.4ward.it http://www.4ward365.com MASSIMILIANO SCOTTI 4ward Software Developer Machine Learning Big Data @ivanfioravanti @maxsv15
- 3. AZURE +MONGODB It’s a kind of magic
- 4. 2PARTS AZURE +MMS Azure CLI Service Manager Resource Manager SIOPE Open Data Aggregation Framework Python
- 5. FIRST MAGICTRICK:REPLICASET AZURE+ MONGODB MMS Photo by Nasa (http://www.nasa.gov/mission_pages/station/multimedia/gallery/iss030e078095.html) East Coast... USA... Virginia... Azure “G” GODZILLA!
- 6. REPLICASET? Group of servers that maintain the same data set. PRIMARY SECONDARY SECONDARY 20GB 20GB 20GB
- 7. REPLICASET? AVAILABILITY Group of servers that maintain the same data set. PRIMARY SECONDARY SECONDARY PRIMARY 20GB 20GB 20GB
- 8. DEMOREPLICASET ONAZURE REPLICASET WITH MMS 1 Azure Subscription Azure Command Line MongoDB MMS Account GitHub: https://github.com/4ward/easy-azure-opensource
- 9. SECOND MAGICTRICK:SHARDING Azure + MongoDB MMS
- 10. SHARDING? SCALABILITY Dataset distributed on more servers or replica sets SHARD A 200GB SHARD B 200GB CONFIG SERVERS 400GB RouterMONGOS
- 11. DEMOSHARDINGONAZURE 1 DEV-TEST SERVER MMS, Azure Single Server with 1 config server 4 mongod server 1 mongos
- 12. COMING SOON: AZURE RESOURCE MANAGER Quick Demo: MongoDB with MMS template
- 13. AZURE TYPE “G”+ MONGODB =PERFORMANCE! Up to: 32 Core 448 GB di RAM 6.144 GB di SSD HOW TO TEST IT?...
- 14. IDEA! SIOPE DATA IN REALTIME! Same used in soldipubblici.gov.it
- 15. Siope-MONGODB
- 16. TRY YOURSELF! Download zip from Siope.it Import RDS data from CSV Convert to Document Convert to Time Series github.com/4ward/siope-mongodb
- 17. DEMO AZURE “G”+ MONGODB+ SIOPE
- 18. SIOPE:Aggregation Framework in action! QUERY on PLAINVS TIMESERIES: 7 VS 2,5 SECS! queries.uscitePerEnteSlow(2014) queries.uscitePerEnte(2014) REGIONS & PROVINCES? queries.uscitePerRegione(2014) queries.uscitePerProvincie(2014) EXPENSES BYMACRO ENTITY? queries.uscitePerSottoComparti(2014)
- 19. SIOPE:Aggregation Framework in action! COMUNE DI MILANO DETAIL? queries.uscitePerEnteDettaglio(2014,"COMUNE DI MILANO”) TOP SPENDERS FOR EDUCATION? uscitePerRegioniPerCategoriaGestionale(2014,"Servizi scolastici") queries.uscitePerProvinciePerCategoriaGestionale(2014,"Servizi scolastici") queries.uscitePerEntePerCategoriaGestionale(2014, "Servizi scolastici")
- 20. ONE MORE THING... MTOOLS! https://github.com/rueckstiess/mtools
- 21. MTOOLS:3QUICKSAMPLES slowqueriesDynamic graph mplotqueries mongodb.log SLOWQUERIESPORTABLE HTML REPORT? mlogvis mongodb.log LOGDETAILEDANALYSIS FORSLOWQUERIES? mlogfilter mongodb.log --slow 5000 –word QUER
- 22. Q&AELINKS Easy-azure-opensource https://github.com/4ward/easy-azure-opensource Siope – MongoDB translator https://github.com/4ward/siope-mongodb Azure Quickstart Templates https://github.com/Azure/azure-quickstart-templates MTools https://github.com/rueckstiess/mtools @ivanfioravanti @maxsv15