Using Generative AI and Content Service Platforms together
0 likes112 views
The document outlines the integration of generative AI with content service platforms, specifically through the usage of Docker and various AI components. It discusses the capabilities of open-source large language models (LLMs), deployment methods, and the APIs available for document processing tasks. Additionally, it highlights resources and licenses associated with these technologies and includes integration details with Alfresco's content service platform.
![Content Service Platform Integration
alfresco-ai-listener REST API
ACTIVEMQ
Listen to ASPECT setting
Apache License 2.0
document.move(
document.parent.childByNamePath(
document.properties["genai:term"]
)
);](https://image.slidesharecdn.com/hyland-genai-csp-fosdem-2024-240202111748-3cd6652d/85/Using-Generative-AI-and-Content-Service-Platforms-together-16-320.jpg)
Using Generative AI and Content Service Platforms together
- 1. February 1, 2024 February 1, 2024 ©2024 Hyland Software, Inc. and its affiliates. All rights reserved. All Hyland product names are registered or unregistered trademarks of Hyland Software, Inc. or its affiliates in the United States and other countries. Using Generative AI and Content Service Platforms together FOSDEM’24 Angel Borroy Developer Evangelist
- 2. Agenda Content credentials Generated with AI ∙ 25 January 2024 at 10:09 am
- 3. GenAI Stack
- 4. Docker GenAI Components ollama • Local management of open source LLMs • Catalog of preconfigured LLMs, such as Llama2 or Mistral neo4j • Graph and native vector search capabilities • Ground LLMs for more precise GenAI predictions and outcomes LangChain • Communication between the LLM, your application, and the database • Python framework for developing applications powered by LLMs
- 5. pull-model Docker GenAI Deployment https://github.com/docker/genai-stack pull-model ollama neo4j streamlit LangChain REST API GPU
- 6. Docker GenAI https://ollama.ai/library LLM 3B 7B 13B 13+B 2 GB Size 4 GB RAM 4 GB Size 8 GB RAM 8 GB Size 16 GB RAM 40+ GB Size 64+ GB RAM phi dolphin-phi orca-mini deepseek-coder MIT License MIT License cc-by-nc-sa-4.0 Deepseek License Agreement llama2 codellama vicuna mistral mistral-openorca llava orca-mini deepseek-coder orca2 falcon Llama2 Community License Agreement Llama2 Community License Agreement Llama2 Community License Agreement Apache License 2.0 Apache License 2.0 Apache License 2.0 cc-by-nc-sa-4.0 Deepseek License Agreement Microsoft Research License Falcon TII License 1.0 llama2 codellama vicuna orca-mini llava orca2 Llama2 Community License Agreement Llama2 Community License Agreement Llama2 Community License Agreement cc-by-nc-sa-4.0 Apache License 2.0 Microsoft Research License llama2 codellama mixtral deepseek-coder vicuna falcon Llama2 Community License Agreement Llama2 Community License Agreement Apache License 2.0 Deepseek License Agreement Llama2 Community License Agreement Falcon TII License 1.0
- 7. Docker GenAI Apache License 2.0 Ethical AI Rating Mistral 7B Is the software (both for inferencing and training) open source? https://github.com/mistralai/mistral-src Is the trained model freely available for self-hosting? https://ollama.ai/library/mistral Is the training data available and free to use? No Apache License 2.0
- 8. Docker GenAI Apache License 2.0 Ethical AI Rating LlaVA with Visual Encoder Is the software (both for inferencing and training) open source? https://github.com/haotian-liu/LLaVA Is the trained model freely available for self-hosting? https://ollama.ai/library/llava Is the training data available and free to use? https://github.com/haotian-liu/LLaVA/blob/main/docs/Data.md Apache License 2.0
- 9. Features MacBook Pro 2021 Apple M1 Pro 32 GiB RAM GPU
- 11. GENAI STACK GenAI App Python application that provides REST API endpoints Document Loader Split document into smaller chunks LLM Embeddings Store in VectorDB Embed and save chunks Create QA Chain With the ability to lookup Vector DB LLM Response LLM=mistral LLM_VISION=llava SUMMARY_LANGUAGE=English SUMMARY_SIZE=120 TAGS_NUMBER=3 Handle picture Response
- 12. GenAI REST API curl --location 'http://localhost:8506/summary' --form 'file=@"./file.pdf"' { "summary": " The text discusses...", "tags": " Golang, Merkle, Difficulty", "model": "mistral" } curl --location 'http://localhost:8506/classify?termList="Japanese,Spanish,Vietnamese"' --form 'file=./file.pdf"' { "term": " English", "model": "mistral" }
- 13. GenAI REST API curl --location 'http://localhost:8506/prompt?prompt="What is the name of the son?"' --form 'file=./file.pdf"' { "answer": "The name of the son is Musuko.", "model": "mistral" } curl --location 'http://localhost:8506/describe' --form 'image=@"image.jpg"' { "description": "The image features a man standing confidently. He is wearing glasses, a beanie hat, and a jacket.", "model": "llava" }
- 14. Integration
- 15. Integration with Content Service Platform Deployment https://github.com/aborroy/alfresco-genai ALFRESCO GENAI STACK Docker Network model-repo model-share llm neo4j langchain compose compose alfresco-ai-applier alfresco-ai-listener app service http://genai:8506/summary http://genai:8506/classify http://genai:8506/prompt http://genai:8506/describe http://alfresco:8080 tcp://activemq:61616
- 16. Content Service Platform Integration alfresco-ai-listener REST API ACTIVEMQ Listen to ASPECT setting Apache License 2.0 document.move( document.parent.childByNamePath( document.properties["genai:term"] ) );
- 17. What Else?
- 18. Find your way Docker AI/ML Hack-a-thon • Readme AI • Techdocs • Docker Image Analyzer • Docker Log Sentiment Analyzer • GitChats AI ollama alternatives • https://gpt4all.io/index.html • https://localai.io • https://www.secondstate.io/run-llm • https://huggingface.co/docs/hub/spaces-sdks-docker-first-demo
- 19. Thanks!