Large Language Models, Data & APIs - Integrating Generative AI Power into your solutions - with Python and .NET
0 likes312 views
The document discusses integrating large language models (LLMs) into solutions using Python and .NET, highlighting their capabilities in generating and understanding text. It emphasizes that while LLMs are powerful tools, they should be viewed as part of broader software architectures and not as standalone solutions. The text also covers the evolving landscape of AI, including open-source innovations and various applications of LLMs.
Large Language Models, Data & APIs - Integrating Generative AI Power into your solutions - with Python and .NET
- 1. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Christian Weyer @christianweyer CTO, Technology Catalyst
- 2. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for Developer Technologies & Azure ASPInsider, AzureInsider § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Christian Weyer Co-Founder & CTO @ Thinktecture AG 2
- 3. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Our journey 3 AI all-the- things? Integrating LLMs Selected Scenarios End-to-End Exciting Times… Democratizing Generative AI
- 4. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET AI all-the-things? 4
- 5. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET AI all-the-things? 5 Data Science Artificial Intelligence Machine Learning Unsupervised, supervised, reinforcement learning Deep Learning ANN, CNN, RNN etc. NLP Generative AI GAN, VAE, Transformers etc. Image / Video Generation GAN, VAE Large Language Models Transformers
- 6. § LLMs generate text based on input § LLMs can understand text – this changes a lot § Prompts are the universal interface (“UI”) → unstructured text with semantics § Human language evolves as a first-class citizen in software architecture 🤯 Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Large Language Models (LLMs) 6 Text… – really, just text?
- 7. § LLMs are programs § LLMs are highly specialized neural networks § LLMs use(d) lots of data § LLMs need a lot of resources to be operated § LLMs have an API to be used through Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Large Language Models demystified 7
- 8. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Integrating LLMs 8
- 9. § LLMs are always part of end-to-end architectures § HTTP/Web/REST APIs § Databases § Client apps (Web, desktop, mobile) § etc. § An LLM is ‘just’ an additional asset in your architecture § It is not the Holy Grail for everything! Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET End-to-end architectures with LLMs 9
- 10. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Using LLMs: It’s just APIs ! Inference, FTW. 10
- 11. GPT-4 API access via OpenAI Playground Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 11
- 12. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET The best tool for .NET developers to talk to LLMs! 12 🙈
- 13. § OSS framework for developing applications powered by LLMs § > 1000 contributors § Python and Typescript versions § Chains for sequences of LLM-related actions in code § Abstractions for § Prompts & LLMs (local and remote) § Memory § Vector stores § Tools § Loading text from a wide range of sources Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET LangChain - building LLM-based applications 13
- 14. § Microsoft’s OSS framework to integrate LLMs into applications § .NET, Python, and Java versions § .NET 1.0 RTM in December 2023 § Plugins encapsulate AI capabilities § Semantic functions for prompting § Native functions to run local code § Planners are orchestrating LLMS interactions § Not as broad feature set as LangChain § E.g., no concept/abstraction for loading data Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Semantic Kernel - building LLM-based applications 14
- 15. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Selected Scenarios 15
- 16. Text generation § LLMs are good in generating text § Regular text § Code § SQL (beware!) § JSON § etc. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Typical LLM scenarios: 16
- 17. Extracting meaning in text § LLM can be instructed to, e.g. § Do sentiment analysis § Extract information from text § Extracting structured information § JSON, TypeScript types, etc. § Via tools like Kor, TypeChat, or Open AI Functions Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Typical LLM scenarios: 17
- 18. Extracting structured data (LangChain + Kor) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 18
- 19. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Answering Questions on Data - Retrieval-augmented generation (RAG) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Text Question A n s w e r LLM 19 Vector DB Embedding model Embedding model 💡 Indexing / Embedding QA
- 20. Learning about my company’s policies via Slack (LangChain) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 20
- 21. Chat with web site documents (Semantic Kernel) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 21
- 22. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Democratizing Generative AI 22
- 23. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET LLMs everywhere OpenAI-related (cloud) OpenAI Azure OpenAI Service Big cloud providers Google Model Garden on Vertex AI Amazon Bedrock Other providers Antrophic Cohere HuggingFace … Open-source Edge IoT Server Desktop Mobile Web Open-source 23
- 24. § Open-source community drives innovation in Generative AI § HuggingFace is central place for it § Literally, every week a new and “better” LLM shows up 🤓 § Important factors § Use case § Parameter size § Quantization § Processing power needed § Mistral-based family shows big potential for local use cases (7B params) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Open-source LLMs thrive 24
- 25. Local RAG with Mistral OSS LLM (llama.cpp & LM Studio) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 25
- 26. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET End-to-End 26
- 27. Ask for expert availability in my company systems (Speech-to-text, my own API, GPT-4, Text-to-speech) Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET DEMO 27
- 28. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Exciting Times 28
- 29. § LLMs enable new scenarios & use cases to incorporate human language into software solutions § Fast moving and changing field § Every week something “big” happens in LLM space § Frameworks & ecosystem are evolving together with LLMs § Closed vs open LLMs § Competition drives invention & advancement § SISO (sh*t in, sh*t out) § Quality of results heavily depends on your data & input Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Current state 29
- 30. Potential for LLM-powered human-machine workflows via universal interface agents Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Outlook 30
- 31. Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Outlook 31
- 32. Thank you! Christian Weyer https://thinktecture.com/christian-weyer 32 Selected demos: Extract structured information: https://github.com/thinktecture-labs/llm-extract-structured-information-langchain-kor Local RAG with PDFs: https://github.com/thinktecture-labs/rag-chat-with-pdf-local-llm
- 33. § Semantic Kernel § https://learn.microsoft.com/en-us/semantic-kernel/overview/ § LangChain § https://www.langchain.com/ § LangChain Agents § https://python.langchain.com/docs/modules/agents/ § ReAct: Synergizing Reasoning and Acting in Language Models § https://react-lm.github.io/ § Prompt Engineering Guide § https://www.promptingguide.ai/ § OpenAI API reference § https://platform.openai.com/docs/api-reference § Azure OpenAI Service REST API reference § https://learn.microsoft.com/en-us/azure/ai-services/openai/reference § Hugging Face Inference Endpoints (for various OSS LLMs) § https://huggingface.co/docs/inference-endpoints/api_reference § Llama2 on Hugging Face § https://huggingface.co/blog/llama2 § OWASP Top 10 for LLM Applications § https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-slides-v1_0_1.pdf Large Language Models, Data & APIs Integrating Generative AI Power into your solutions - with Python & .NET Links 33