Download as PDF, PPTX
Uploaded byMaxim Salnikov
ChatGPT and not only: how can you use the power of Generative AI at scale
This document discusses Microsoft's Azure OpenAI Service and how it can be used to build applications using large language models. Some key points: - Azure OpenAI Service provides access to models from OpenAI like GPT-3 through Microsoft's Azure cloud platform while ensuring security, privacy and responsible AI. - It allows generating complex documents, steering models with nuanced instructions, and customizing models for any language or dialect. - Example capabilities include content generation, summarization, code generation, and semantic search. These can be applied to use cases like call center analytics, software documentation, and marketing content creation. - Tools are discussed for developing applications using prompt engineering, grounding models with domain-specific
More Related Content
![[DSC Europe 23] Ivan Petrovic - Approach to Architecting Generative AI Solutions](https://cdn.slidesharecdn.com/ss_thumbnails/ivanpetrovic-approachtoarchitectinggenerativeaisolutions-231129100808-16df2918-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to ChatGPT and not only: how can you use the power of Generative AI at scale
![[DSC Europe 23] Igor Ilic - Redefining User Experience with Large Language Mo...](https://cdn.slidesharecdn.com/ss_thumbnails/igorilicdscpresentation-231129092442-b7067fff-thumbnail.jpg?width=640&height=640&fit=bounds)
ChatGPT and not only: how can you use the power of Generative AI at scale
- 1. ChatGPT and notonly: how can you use the power of Generative AI at scale Maxim Salnikov DeveloperProductivity Business LeadatMicrosoft
- 2. • Building onweb platform since 90s • Organizing developer communities and technical conferences • Speaking, training, blogging: Webdev, Cloud, OpenAI Helping developers to succeed with the Cloud & AI technology I’m Maxim Salnikov
- 3. 87% of organizations believe AIwill give them a competitive edge 50% of organizations have adopted AI in at least one business area Sources: MIT Sloan Management Review, The state of AI in 2022--and a half decade in review | McKinsey Why AI?
- 4. Powered by AzureAI Applications Partner Solutions Application Platform AI Builder Power BI Power Apps Power Automate Power Virtual Agents Scenario-Based Services Bot Service Cognitive Search Document Intelligence Video Indexer Metrics Advisor Immersive Reader Customizable AI Models Vision Speech Language Decision Azure OpenAI Service ML Platform Azure Machine Learning
- 5. Microsoft and OpenAIpartnership Ensure that artificial general intelligence (AGI) benefits humanity Empower every person and organization on the planet to achieve more Azure OpenAI Service GPT-4 & GPT-3.5-Turbo Text Chat Completions Conversation DALL·E 3 Image Whisper Transcription & Translation
- 6. Generate complex documents Steerwith nuanced instructions Instruct and annotate in any language, slang, dialect GPT-4
- 7. Azure OpenAI Service Deployed inyour Azure subscription, secured by you, and tied to your datasets and applications Large, pretrained AI models to unlock new scenarios Custom AI models fine-tuned with your data and hyperparameters Built-in responsible AI to detect and mitigate harmful use Enterprise-grade security with role-based access control (RBAC) and private networks
- 8. Capabilities and usecases
- 9. Main capabilities Content generationSummarization Code generation Semantic search Call center analytics: automatically generate responses to customer inquiries Generate personalized UI for your website Call center analytics: summary of customer support conversation lo gs Subject matter expert document: summarization (e.g. Financial reporting, analyst articles) Social media trends summarization Convert natural language to SQL (or vice versa) for telemetry data Convert natural language to query proprietary data models Code documentation Search reviews for a specific product/service Information discovery and knowledge mining
- 10. Solving business problems Business Problem Productivityis lagging Need for process Automation Degraded Customer Experience Creating Content is Time Consuming Business Needs Increase Productivity Automate Processes Improve Customer Experience Build Creative Content Solutions • Conversational Search/Knowledge Insights • Code Generation and Documentation • Trend Forecasting • Report Summarization & Generation • Document Processing • Workflow Management • Fraud Detection • Supply Chain Optimization • Intelligent Contact Center • Agent/Employee Assistance • Virtual Assistance • Call Analytics • Call Summarization • Marketing/Sales Content Generation • Personalized Content Generation • Product Design & Development • Digital Art What can Generative AI Do? Generate New Revenue Streams Deliver Differentiated Customer Experiences Modernize Internal Processes
- 11. Content creation & design Accelerated automation Personalized marketing Chatbots and virtualassistants Product & service innovation Language translation & language processing Fraud detection Predictive analytics & forecasting Creative writing & content generation Medical research & diagnosis Top 10 usecases
- 12. How to startbuilding LLM app?
- 13. Build on solidfoundation Apps Plugin extensibility Copilots Prompt Flow & Model Evaluation Metaprompt Data grounding Plugin execution Foundation models AI infrastructure Your applications Azure OpenAI Service Built-in safety system and responsible AI tools
- 14. Grounding is the processof using large language models (LLMs) with information that is use-case specific, relevant, and not available as part of the LLM's trained knowledge.
- 15. Prompt Engineering Fine-tuning LLMOps Responsible AI Vastmajority of use cases Grounding options Training
- 16. Prompt Engineering Is theprocess of designing, refining, and optimizing input prompts to guide a model toward producing more accurate outputs while keeping cost efficiency
- 17. Prompt Text input thatprovides some framing as to how the engine should behave You are an intelligent assistant helping Contoso Inc employees with their healthcare plan questions and employee handbook questions. Answer the following question using only the data provided in the sources below. Question: Does my health plan cover annual eye exams? Sources: 1. Northwind Health Plus offers coverage for vision exams, glasses, and contact lenses, as well as dental exams, cleanings, and fillings. 2. Northwind Standard only offers coverage for vision exams and glasses. 3. Both plans offer coverage for vision and dental services. User provided question that needs to be answered Sources used to answer the question Response Based on the provided information, it can be determined that both health plans offered by Northwind Health Plus and Northwind Standard provide coverage for vision exams. Therefore, your health plan should cover annual eye exams. Bringing data into the prompt
- 18. Will my sleeping bagwork for my trip to Patagonia next month? User input Historical weather lookup Intent mapping Personalization Product info Recommendations engine ??? Prompt engineering LLM Yes, your Elite Eco sleeping bag is rated to 21.6F, which is below the average low temperature in Patagonia in September Output More context
- 19.
- 20.
- 21. 1. Prototyping 2. Experimenting 3.Iterating 4. Deploying 5. Optimizing LLM applications
- 22. Operationalize LLM app development • Privatedata access and controls • Prompt engineering • CI/CD • Iterative experimentation • Versioning and reproducibility • Deployment and optimization • Safe and Responsible AI Design and development Develop flow based on prompt to extend the capability Debug, run, and evaluate flow with small data Modify flow (prompts and tools etc.) No If satisfied Yes Evaluation and refinement Evaluate flow against large dataset with different metrics (quality, relevance, safety, etc.) If satisfied Yes Optimization and production Optimize flow Deploy and monitor flow Get end user feedback
- 23. Prompt flow forLLMOps • Extensive evaluation capabilities for prompt engineering workflows • Prompt flow definitions as first-class entities (YAML) • Managed API connections for CI/CD across dev, test, prod • Multiple authoring interfaces including code-first, CLI and UI • Inter-op with Python libs like Guidance, Semantic Kernel, and LangChain Capabilities https://github.com/microsoft/promptflow
- 25. Prompt flow ecosystem VScode extension + Prompt flow package Install Prompt flow extension in VS Code desktop Flexibly switch envs with promptflow package installed Enjoy the similar authoring UI with AzureML workspace Prompt flow package in your IDE Pip install promptflow package Develop flow in any IDE Execution: flow init, test, run, visualize, etc.
- 26. Authoring Develop your LLMflow from scratch • Construct a flow using pre-built tools • Support custom code • Clone flows from samples • Track run history
- 27. Prompt variants • Createdynamic prompts using external data and few shot samples • Edit your complex prompts in full screen • Quickly tune prompt and LLM configuration with variants
- 28. Evaluation • Evaluate flowperformance with your own data • Use pre-built evaluation flows • Build your own custom evaluation flows • Compare multiple variants or runs to pick best flow • Add new evaluations to a finished run • Ensure accuracy by scaling the size of data in evaluation
- 29. Deploy • Seamless transitionfrom development to production with Azure ML’s managed online endpoints
- 30. Prompt flow inAzure ML benefits • Integrates into existing CI/CD processes to manage prompts • Shorter time to higher quality prompts through experimentation • Historical tracking of prompt authoring, metric validation and certification • Enterprise security for API connectivity, data access and deployment https://ml.azure.com
- 31. App or Copilot agent API& SDK Azure OpenAI Service on your data Data Sources (search, files, databases, storage etc.) Additional 3P Data Sources (files, databases, storage data etc.) Azure OpenAI Service on your data https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data
- 32. Ingest / Connect ●Connect your data source whatever it is & wherever it is Ground, Chunk, Tune & Tone ● Unlock the full protentional of your data Share & Use ● Share with your customers & organization Index, semantic search, vector search, authenticate, personalize, company policies and more Documents, files, Cognitive Search, blob, local file upload …. Easy to integrate within your organization or with your customers simple APIs, SDK, Customized Web App End-to-end RAG scaffolds
- 33.
- 34. The solution tothe size and fit problem is knowing the personal preferences of your customers on each item-model-level
- 35. • Skjold.ai protectsall traffic • Check suspicious messages • Cybercrime is growing • Criminals adopt new technology • Instant feedback if the messages is considered safe or could be fraud • Explain red flags and educate the user • Integrated in the iPhone SMS app • Available for iPhone and Android AI-powered scan of suspicious messages Read more www.skjold.ai/demo Mobile first protection against cyber crime built with Azure OpenAI
- 37. Kahoot! launches newAI features for more powerful learning https://kahoot.com/kahoot-news/kahoot-microsoft-collaboration-ai-features/
- 38. Introducing Cognite AI,the Generative AI Accelerator for Industrial Data and Value Realization https://www.cognite.com/en/press-release/introducing-cognite-ai
- 39.
- 40. How is mydata used in Azure OpenAI Service? When you use Azure OpenAI Service, your prompts (inputs) and completions (outputs), your embeddings, and your training data Are NOT available to other customers. Are NOT available to OpenAI. Are NOT used to improve OpenAI models. Are NOT used to improve any Microsoft or 3rd party products or services. Are NOT used for automatically improving Azure OpenAI models for your use in your resource (The models are stateless, unless you explicitly fine-tune models with your training data). Your fine-tuned Azure OpenAI models are available exclusively for your use. The Azure OpenAI Service is fully controlled by Microsoft; Microsoft hosts the OpenAI models in Microsoft’s Azure environment and the Service does NOT interact with any services operated by OpenAI (e.g., ChatGPT, or the OpenAI API). https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy
- 41. How does ResponsibleAI work in Azure OpenAI? Customer Application Prompt Azure OpenAI Endpoint RAI Filtered Response Responsible AI Model Ensemble Hate Sexual Text Images Abuse Concern? Azure AI Content Safety – Configurable Filters
- 42. [email protected] Let’s connect andchat! Thank you!