Gloo AI Gateway
Gloo AI Gateway by Solo.io is a cloud-native solution designed to manage AI applications with enhanced security, control, and observability. Built on the Envoy Proxy and Kubernetes Gateway API, Gloo AI Gateway enables seamless integration of large language models (LLMs) and AI-driven services across cloud environments. It offers features like prompt management, fine-grained access control, and real-time analytics to monitor and optimize AI consumption. The platform also includes safeguards to protect against abuse and ensure model security, improving both model performance and operational efficiency in AI-powered applications.
Learn more
Portkey
Launch production-ready apps with the LMOps stack for monitoring, model management, and more. Replace your OpenAI or other provider APIs with the Portkey endpoint.
Manage prompts, engines, parameters, and versions in Portkey. Switch, test, and upgrade models with confidence!
View your app performance & user level aggregate metics to optimise usage and API costs
Keep your user data secure from attacks and inadvertent exposure. Get proactive alerts when things go bad.
A/B test your models in the real world and deploy the best performers.
We built apps on top of LLM APIs for the past 2 and a half years and realised that while building a PoC took a weekend, taking it to production & managing it was a pain!
We're building Portkey to help you succeed in deploying large language models APIs in your applications.
Regardless of you trying Portkey, we're always happy to help!
Learn more
Vercel
Vercel is an AI-powered cloud platform that helps developers build, deploy, and scale high-performance web experiences with speed and security. It provides a unified set of tools, templates, and infrastructure designed to streamline development workflows from idea to global deployment. With support for modern frameworks like Next.js, Svelte, Vite, and Nuxt, teams can ship fast, responsive applications without managing complex backend operations. Vercel’s AI Cloud includes an AI Gateway, SDKs, workflow automation tools, and fluid compute, enabling developers to integrate large language models and advanced AI features effortlessly. The platform emphasizes instant global distribution, enabling deployments to become available worldwide immediately after a git push. Backed by strong security and performance optimizations, Vercel helps companies deliver personalized, reliable digital experiences at massive scale.
Learn more
BentoML
Serve your ML model in any cloud in minutes. Unified model packaging format enabling both online and offline serving on any platform. 100x the throughput of your regular flask-based model server, thanks to our advanced micro-batching mechanism. Deliver high-quality prediction services that speak the DevOps language and integrate perfectly with common infrastructure tools. Unified format for deployment. High-performance model serving. DevOps best practices baked in. The service uses the BERT model trained with the TensorFlow framework to predict movie reviews' sentiment. DevOps-free BentoML workflow, from prediction service registry, deployment automation, to endpoint monitoring, all configured automatically for your team. A solid foundation for running serious ML workloads in production. Keep all your team's models, deployments, and changes highly visible and control access via SSO, RBAC, client authentication, and auditing logs.
Learn more
