11-OCT-2024_AI_101_CryptoOracle_UnstructuredData
0 likes•71 views
The document introduces the concept of unstructured data, vector databases, and generative AI with a focus on retrieval-augmented generation (RAG). It outlines key strategies for AI model development, including training from scratch, fine-tuning existing models, and utilizing RAG, while emphasizing the role of vector databases like Milvus in improving AI efficiency. Participants will learn about tools and frameworks that support RAG applications, as well as the challenges and advantages associated with different AI model approaches.
![36 | © Copyright Zilliz
36
Vector Similarity Metric: L2 Euclidean)
Queen = [0.3, 0.9]
King = [0.5, 0.7]
d(Queen, King) = √(0.3-0.5)2
+ (0.9-0.7)2
= √(0.2)2
+ (0.2)2
= √0.04 + 0.04
= √0.08 ≅ 0.28](https://image.slidesharecdn.com/11-oct-2024ai101cryptooracle-241011211937-380ca411/85/11-OCT-2024_AI_101_CryptoOracle_UnstructuredData-36-320.jpg)
![37 | © Copyright Zilliz
37
Vector Similarity Metric: Inner Product IP
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Queen · King = (0.3*0.5) + (0.9*0.7)
= 0.15 + 0.63 = 0.78](https://image.slidesharecdn.com/11-oct-2024ai101cryptooracle-241011211937-380ca411/85/11-OCT-2024_AI_101_CryptoOracle_UnstructuredData-37-320.jpg)
![38 | © Copyright Zilliz
38
Queen = [0.3, 0.9]
King = [0.5, 0.7]
Vector Similarity Metric: Cosine
𝚹
cos(Queen, King) = (0.3*0.5)+(0.9*0.7)
√0.32
+0.92
* √0.52
+0.72
= 0.15+0.63 _
√0.9 * √0.74
= 0.78 _
√0.666
≅ 0.03](https://image.slidesharecdn.com/11-oct-2024ai101cryptooracle-241011211937-380ca411/85/11-OCT-2024_AI_101_CryptoOracle_UnstructuredData-38-320.jpg)
11-OCT-2024_AI_101_CryptoOracle_UnstructuredData
- 1. 1 | © Copyright 2024 Zilliz 1 Introduction to Unstructured Data, Vector Database and Gen AI Tim Spann @ Zilliz
- 2. 2 | © Copyright 2024 Zilliz 2 2 | © Copyright 10/22/23 Zilliz 2 | © Copyright 2024 Zilliz Tim Spann Principal Developer Advocate Zilliz [email protected] https://www.linkedin.com/in/timothyspann/ https://x.com/PaaSDev
- 3. 3 | © Copyright Zilliz 3 W A New Data and Compute World
- 4. This is an introduction to the various approaches for building and fine-tuning machine learning models, with a primary focus on Retrieval-Augmented Generation (RAG) and the tools that support it. The session will guide participants through three key strategies for creating AI models: training a model from scratch, fine-tuning an existing model, and utilizing RAG. In the first section, participants will receive an overview of the following options: 1. Training Your Own Model: Understand the challenges and opportunities of building models from the ground up. 2. Fine-tuning a Pre-trained Model: Learn how to adapt existing models to specific tasks and datasets. 3. The RAG Approach: Dive into Retrieval-Augmented Generation, a hybrid method that leverages both knowledge retrieval and generative models for more efficient, accurate AI applications. In the second section, the focus will shift to the RAG Approach, with a detailed explanation of its core components: ● Vector Databases: Explore how vector databases (such as Milvus) are used to store and retrieve embeddings, enabling efficient information retrieval. ● Open Source Models on Hugging Face: Learn how to integrate various open-source models available on platforms like Hugging Face with RAG workflows. ● Langchain and Other Frameworks: Understand how Langchain and similar frameworks simplify building and deploying RAG-based solutions. Throughout the session, Milvus will be a central point of discussion, highlighting its role in vector storage and retrieval, and how it can be paired with models to enhance AI applications.
- 5. Learning Outcomes: ● Gain an understanding of different AI model development strategies. ● Understand the principles and advantages of the RAG approach. ● Learn how to use tools like Milvus, open-source models, and frameworks like Langchain to create efficient AI systems.
- 6. 6 | © Copyright Zilliz 6 Cloud Service Provider Data Platform GenAI Tooling Chip Manufacturer Partner with Industry Leaders
- 7. 7 | © Copyright Zilliz 7 T Training Your Own Model
- 8. 8 | © Copyright Zilliz 8 Problems with LLM Training ● It requires a large amount of time ● It requires a lot of money ● It requires a large cluster and electricity ● So many free and open models available ● Works for a specific domain of data ● Requires 100s of GPUs ● Requires lots of data (maybe synthetic…) ● Look at Fine Tuning and RAG instead ● Requires Data Engineers and Data Scientists
- 9. 9 | © Copyright Zilliz 9 Meta LLama 3.x Models Mistral Models Microsoft Models QWen Models Google Models Hugging Face Open LLM Leaderboard https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
- 10. 10 | © Copyright Zilliz 10 Exception: Training Your Own Text Embedding Model https://zilliz.com/learn/training-your-own-text-embedding-model
- 11. 11 | © Copyright Zilliz 11 Perhaps Make a Small Language Model https://medium.com/@nageshmashette32/small-language-model s-slms-305597c9edf2
- 12. 12 | © Copyright Zilliz 12 F Fine-tuning a Pre-trained Model
- 13. 13 | © Copyright Zilliz 13 RAG vs Fine Tuning LLM Fine Tuning Augment an LLM by training it on your data Your Data “New” LLM Query Primary Use Case - Style transfer
- 14. 14 | © Copyright Zilliz 14 Problems with Fine Tuning ● Retraining takes time ● Retraining takes money ● Retraining takes human supervision to organize and label data ● Works for a specific domain of data ● Requires GPUs
- 15. 15 | © Copyright Zilliz 15 Problems with Fine Tuning ● Works for the version of a model you fine tuned against ○ Models change always) ● Data changes constantly, everywhere, always ● Requires Skilled Data Scientists ● Requires GPUs ● LLM Can Still Hallucinate
- 16. 16 | © Copyright Zilliz 16 R The RAG Approach
- 17. 17 | © Copyright Zilliz 17 How do You Prevent LLM Hallucinations? Vector Database serve as a knowledge base for your LLM Application so that you can: - Build on Your Data - Eliminate Hallucinations - Solve Customer Problems with AI
- 18. 18 | © Copyright Zilliz 18 RAG vs Fine Tuning Load Your Data Instantly Access it Searchable Data Platform Reusable Data Assets Can be simple to code
- 19. 19 | © Copyright Zilliz 19 RAG vs. LLM - Knowledge of LLM is out-of-date - LLM can not get your private knowledge - Help reducing Hallucinations - Transparency and interpretability RAG vs. Fine-tune - Fine-tune is expensive - Fine-tune can be time consuming - RAG is pluggable Why RAG?
- 20. A New tool emerged. The Vector Database
- 21. Vector Database : making sense of unstructured data 2024 A vector database stores embedding vectors and allows for semantic retrieval of various types of unstructured data.
- 23. 23 | © Copyright 2024 Zilliz 23 | © Copyright 8/16/23 Zilliz 23 Milvus is an Open-Source Vector Database to store, index, manage, and use the massive number of embedding vectors generated by deep neural networks and LLMs. contributors 400 stars 29K docker pulls 66M forks 2.7K + Milvus: The most widely-adopted vector database
- 24. 24 | © Copyright Zilliz 24 O Open Source Models on Hugging Face
- 25. 25 | © Copyright 2024 Zilliz 25 Well-connected in LLM infrastructure to enable RAG use cases Framework Hardware Infrastructure Embedding Models LLMs Software Infrastructure Vector Database
- 26. 26 | © Copyright Zilliz 26 C Langchain and Other Frameworks
- 27. 27 | © Copyright Zilliz 27 What is LangChain - LLM Chaining Framework - Plugins with most popular tools - Focus on “chainingˮ results or “orchestrationˮ
- 28. 28 | © Copyright Zilliz 28
- 29. 29 | © Copyright 2024 Zilliz 29 Well-connected in LLM infrastructure to enable RAG use cases Framework Hardware Infrastructure Embedding Models LLMs Software Infrastructure Vector Database
- 30. 30 | © Copyright 2024 Zilliz 30
- 31. 31 | © Copyright Zilliz 31 U Under the Covers
- 32. 32 | © Copyright Zilliz 32 Use math to quantify relationships between entities
- 33. 33 | © Copyright Zilliz 33 Vector databases efficiently store, index, and relate entities by a quantitative value
- 34. 34 | © Copyright Zilliz 34 Vector similarity is a mathematical measure of how close two vectors are
- 35. 35 | © Copyright Zilliz 35 Similarity metrics are ways to measure distance in vector space
- 36. 36 | © Copyright Zilliz 36 Vector Similarity Metric: L2 Euclidean) Queen = [0.3, 0.9] King = [0.5, 0.7] d(Queen, King) = √(0.3-0.5)2 + (0.9-0.7)2 = √(0.2)2 + (0.2)2 = √0.04 + 0.04 = √0.08 ≅ 0.28
- 37. 37 | © Copyright Zilliz 37 Vector Similarity Metric: Inner Product IP Queen = [0.3, 0.9] King = [0.5, 0.7] Queen · King = (0.3*0.5) + (0.9*0.7) = 0.15 + 0.63 = 0.78
- 38. 38 | © Copyright Zilliz 38 Queen = [0.3, 0.9] King = [0.5, 0.7] Vector Similarity Metric: Cosine 𝚹 cos(Queen, King) = (0.3*0.5)+(0.9*0.7) √0.32 +0.92 * √0.52 +0.72 = 0.15+0.63 _ √0.9 * √0.74 = 0.78 _ √0.666 ≅ 0.03
- 39. 39 | © Copyright Zilliz 39 Vector Similarity Metrics Euclidean - Spatial distance Cosine - Orientational distance Inner Product - Both With normalized vectors, IP Cosine
- 40. 40 | © Copyright Zilliz 40 Indexes organize the way we access our data
- 41. 41 | © Copyright Zilliz 41 | © Copyright Zilliz 41 RESOURCES
- 42. 42 | © Copyright Zilliz 42 Vector Database Resources Give Milvus a Star! Chat with me on Discord! https://github.com/milvus-io/milvus
- 43. 43 Unstructured Data Meetup https://www.meetup.com/unstructured-data-meetup-new-york/ This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs. This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
- 44. 44 | © Copyright Zilliz 44 https://zilliz.com/learn/generative-ai
- 45. 45 | © Copyright 2024 Zilliz 45 45 This week in Milvus, Towhee, Attu, GPT Cache, Gen AI, LLM, Apache NiFi, Apache Flink, Apache Kafka, ML, AI, Apache Spark, Apache Iceberg, Python, Java, Vector DB and Open Source friends. https://bit.ly/32dAJft https://github.com/milvus-io/milvus AIM Weekly by Tim Spann
- 46. 46 | © Copyright 2024 Zilliz 46 milvus.io github.com/milvus-io/ @milvusio @paasDev /in/timothyspann Connect with me! Thank you!