Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experience London 2017
1 like•341 views
Cognitive Services are a collection of AI services from Microsoft that allow developers to add capabilities like speech, language, vision, and knowledge to their applications using simple APIs. The services cover areas like computer vision, language processing, speech, search, and knowledge and can be combined to build intelligent applications. Developers can easily integrate Cognitive Services into their apps to handle tasks like image recognition, text translation, sentiment analysis and more with just a few lines of code.
![Computer Vision
Description:
{
"Tags": [ "water", "mountain", "outdoor", "surfing", "blue", "wave", "riding", "man", "ocean", "top", …],
"Captions": [ { "Text": "a man riding a wave on a surfboard in the water", "Confidence": 0.437939942 } ]
}
Tags:
[
{ "Name": "sky", "Confidence": 0.9965861 },
{ "Name": "water", "Confidence": 0.9905544 },
{ "Name": "mountain", "Confidence": 0.945281565 },
{ "Name": "outdoor", "Confidence": 0.9212041 },
{ "Name": "water sport", "Confidence": 0.783058345 },
{ "Name": "sport", "Confidence": 0.6974723 },
{ "Name": "blue", "Confidence": 0.684881747 },
{ "Name": "wave", "Confidence": 0.6782906 }
]](https://image.slidesharecdn.com/cognitiveservicespowerofiot-jorgeborralho-170928135527/85/Mobile-Cognitive-Services-Harnessing-the-Power-of-IoT-Xamarin-Experience-London-2017-12-320.jpg)
![Language Understanding
Intelligent Service - LUIS
Example:
“News about flight delays”
{
"entities":[{"entity":flight_delays", "type":"Topic"}],
"intents":[
{"intent":"FindNews", “score":0.99853384},
{"intent":"None", "score":0.07289317},
{"intent":"ReadNews", "score":0.0167122427},
{"intent":"ShareNews", "score":1.0919299E-06}
]
}](https://image.slidesharecdn.com/cognitiveservicespowerofiot-jorgeborralho-170928135527/85/Mobile-Cognitive-Services-Harnessing-the-Power-of-IoT-Xamarin-Experience-London-2017-16-320.jpg)
Mobile & Cognitive Services | Harnessing the Power of IoT – Xamarin Experience London 2017
- 2. Jorge Borralho Mobile & Cognitive Services
- 3. We need smarter apps but the processing power of our smartphones is not enough for everything
- 4. We need smarter services that can be easily integrated with our apps
- 5. What are Cognitive Services? The Cognitive Services are a collection of complimentary ‘services’ (29 and counting) to assist developers with the processing of natural input sources such as speech, images and emotion. The Cognitive Services are organized into five pillars: Speech, Language, Vision, Knowledge and Search. By combining different Cognitive Services developers can build intelligent apps with powerful algorithms using just a few lines of code.
- 6. Cognitive Services Vision SpeechLanguage KnowledgeSearch Computer Vision Content Moderator Emotion Face Video Indexer LUIS Spell Check Linguistic Analysis Text Analytics Translator Web Language Model Speech Speaker Recognition Translator Custom Academic Entity Linking Exploration QnA Maker Recommendations Autosuggest Image News Video Web Entity Custom Vision Service Custom Language Understanding Custom Speech Custom Search Custom Decision Service
- 7. Language Language Understanding Intelligent Service PREVIEW Teach your apps to understand commands from your users Bing Spell Check API Detecting and correcting spelling mistakes in your app Web Language Model API PREVIEW Use the power of predictive language models trained on web-scale data Translator Speech API Easily conduct real-time speech translation with a simple REST API call Text Analytics API PREVIEW Easily evaluate sentiment and topics to understand what users want Translator Text API Easily conduct machine translation with a simple REST API call Linguistic Analysis API PREVIEW Simplify complex language concepts and parse text with the Linguistic Analysis API.
- 8. Computer Vision APIs Computer Vision API Distil actionable information from images Content moderator Automated image, text and video moderation Video Indexer PREVIEW Unlock video insights Face API Detect, analyse, organise and tag faces in photos Emotion API PREVIEW Personalise user experiences with emotion recognition Custom Vision Service PREVIEW Easily customise your own state-of-the-art computer vision models for your unique use case.
- 9. Speech APIs Custom Speech Service PREVIEW Overcome speech recognition barriers like speaking style, background noise and vocabulary Bing Speech API Convert speech to text and back again to understand user intent Speaker Recognition API PREVIEW Use speech to identify and authenticate individual speakers
- 10. Search APIs Bing Autosuggest API Give your app intelligent autosuggest options for searches Bing News Search API Search for news and get comprehensive results Bing Web Search API Get enhanced search details from billions of web documents Bing Entity Search API Enrich your app by infusing knowledge search into existing content Bing Image Search API Search for images and get comprehensive results Bing Video Search API Search for videos and get comprehensive results Bing Custom Search PREVIEW An easy-to-use, advert-free, commercial-grade search tool that lets you deliver the results you want.
- 11. Knowledge APIs Recommendations API PREVIEW Predict and recommend items that your customers want Knowledge Exploration Service PREVIEW Enable interactive search experiences over structured data via natural language inputs Entity Linking Intelligence Service API PREVIEW Power your app’s data links with named entity recognition and disambiguation. Academic Knowledge API PREVIEW Tap into the wealth of academic content in the Microsoft Academic Graph QnA Maker API PREVIEW Distil information into conversational, easy-to- navigate answers. Custom Decision Service PREVIEW A cloud-based, contextual decision-making API that sharpens with experience
- 12. Computer Vision Description: { "Tags": [ "water", "mountain", "outdoor", "surfing", "blue", "wave", "riding", "man", "ocean", "top", …], "Captions": [ { "Text": "a man riding a wave on a surfboard in the water", "Confidence": 0.437939942 } ] } Tags: [ { "Name": "sky", "Confidence": 0.9965861 }, { "Name": "water", "Confidence": 0.9905544 }, { "Name": "mountain", "Confidence": 0.945281565 }, { "Name": "outdoor", "Confidence": 0.9212041 }, { "Name": "water sport", "Confidence": 0.783058345 }, { "Name": "sport", "Confidence": 0.6974723 }, { "Name": "blue", "Confidence": 0.684881747 }, { "Name": "wave", "Confidence": 0.6782906 } ]
- 13. Linguistic Analysis Programmatically analyze text and get: • Language • Key phrase extraction • Sentiment analysis • Topic detection examples: I had a wonderful experience! The rooms were wonderful and the staff was helpful. I had a terrible time at the hotel. The staff was rude and the food was awful.
- 14. Video Indexing
- 15. Language Understanding Intelligent Service - LUIS Build custom language models: LUIS provides simple tools that enable you to build your own language models (intents/entities) which allow any application/bot to understand your commands and act accordingly Define Concepts (intents and entities) Provide Examples (uterances) Deploy Active Learning
- 16. Language Understanding Intelligent Service - LUIS Example: “News about flight delays” { "entities":[{"entity":flight_delays", "type":"Topic"}], "intents":[ {"intent":"FindNews", “score":0.99853384}, {"intent":"None", "score":0.07289317}, {"intent":"ReadNews", "score":0.0167122427}, {"intent":"ShareNews", "score":1.0919299E-06} ] }
- 17. Language Understanding Intelligent Service - LUIS Powering bots: • Microsoft bot framework • Bots built with Node.js or C# • Dialogs to model a conversation • Many types of dialogs
- 18. Roll your own with REST APIs Simple to add: just a few lines of code required Integrate into the language and platform of your choice Breadth of offerings helps you find the right API for your app Built by experts in their field from Microsoft Research, Bing, and Azure Machine Learning Quality documentation, sample code, and community support Easy Flexible Tested GET A KEY Cognitive Services
- 20. Jorge Borralho Harnessing the power of IoT
- 21. The Internet of Things isn’t a technology revolution…
- 22. …IoT is a business revolution, enabled by technology
- 24. Capitalizing on IoT Connect your things Turn data into insights Transform your business
- 25. The IoT Customer Journey Objective Increase efficiency by reducing costs and eliminating bottlenecks Organizational readiness This is a great place to start if you have: • An identified business process you want to gain visibility into • An understanding of the kind of data you need in order to support your business objective • Devices capable of providing the type of data required
- 26. Objective Drive innovation and increase revenue using your existing assets Organizational readiness This is a great place to start if you have : • The ability to interpret incoming data from devices • An understanding of the impacted business groups and processes • A plan for using data to drive business process improvements The IoT Customer Journey
- 27. The IoT Customer Journey Objective Create new lines of business and new business models Organizational readiness This is a great place to start if you have : • A strong baseline of historical device data (e.g. usage trends, repair records) • Visibility into real-time operational status • Predictive capability from advanced analytics • Stakeholder commitment to support change efforts
- 28. IoT projects are complex Time-consuming to stand up Hard to scaleIncompatible with current infrastructure Difficult to maintain cohesive security
- 29. Gain defense in depth with Azure IoT Suite security
- 30. Obtain flexibility with an open approach
- 31. Devices Azure IoT Services Back end systems and processes IoT Hub Stream Analytics Azure MLAzure Functions Stream Analytics Azure MLAzure Functions *Preview
- 32. Q&A Harnessing the power of IoT