Darren Jackson

Providers like AWS give you the tools needed to ensure your apps can withstand almost any kind of failure. The problem is that many of these situations almost never happen so it's hard to test. Using tools like the Fault Injection Simulator (FIS) can help. There are various FIS tools available for different services. They allow you to make it seem like very rare failure scenarios are happening now. This is super valuable to ensure your code to handle these cases actually works. I recommend setting up tests that run in automation using these tools. This article from Tolgahan Demirbaş shows how to use Terraform to setup Node and Availability Zone failure simulations with an Elastic Kubernetes Service (EKS) cluster. https://lnkd.in/e-rgY3sp

65

Let me simplify what is happening around the "𝗣𝗿𝗶𝘃𝗮𝘁𝗲 𝗔𝗜 𝗶𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲" landscape. Today, if an org wants to build an AI data center for GenAI applications at production scale, they have the following challenges to solve - • Infrastructure cost - Which compute (GPU/TPU/accelerator) to choose from? • Operational complexities - Which LLM and how to deploy? Size, tuning, compression, APIs, etc • Deployment constraints - How to have a similar runtime for LLMs across hybrid platforms? One option is to choose all the commoditised open-source components and stitch them together, but still, no guarantee of a 𝘤𝘰𝘯𝘴𝘪𝘴𝘵𝘦𝘯𝘵, 𝘧𝘢𝘴𝘵, 𝘢𝘯𝘥 𝘤𝘰𝘴𝘵-𝘦𝘧𝘧𝘪𝘤𝘪𝘦𝘯𝘵 𝘪𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦 𝘢𝘵 𝘴𝘤𝘢𝘭𝘦? Here is when Red Hat is offering its AI services in its decade-old signature style, i.e., building a service around open-source components with Red Hat's guarantee & credibility 𝗣𝗶𝗰𝘁𝘂𝗿𝗲 𝟭 - shows three AI services for different use cases • RH AI inference server - Only serving ready LLMs, out-of-the-box (Consistent, fast, and cost-effective inference) • RH Enterprise Linux AI - Tuning and serving LLMs via a single machine i.e. (Single Linux Server + AI inference) • RH OpenShift AI - (AI inference + multi-machine OpenShift) with complete setup for GenAI and Predictive AI solution, including MLops. 𝗤𝘂𝗲𝘀𝘁𝗶𝗼𝗻: What is cooking inside Red Hat AI? 𝗣𝗶𝗰𝘁𝘂𝗿𝗲 𝟮 gives us the answer. RedHat acquired a firm in 2024 called 𝗡𝗲𝘂𝗿𝗮𝗹 𝗠𝗮𝗴𝗶𝗰--> Neural magic built 𝘃𝗟𝗟𝗠-->vLLM is the core of 𝗥𝗛 𝗔𝗜 "𝘃𝗟𝗟𝗠" - is a cheap and fast LLM serving engine that connects model creators to accelerated hardware providers In simple terms, vLLM is the open-source equivalent of AWS Sagemaker & Google Vertex AI platforms in the context of private cloud, but with the freedom to choose your own model & underlying infrastructure In cloud-native terminology, if - Hugging Face is the 𝘎𝘪𝘵𝘏𝘶𝘣 𝘰𝘧 𝘈𝘐 Ollama is the 𝘋𝘰𝘤𝘬𝘦𝘳 𝘰𝘧 𝘈𝘐, then vLLM is the 𝙆𝟪𝙨 𝙤𝙛 𝘼𝙄. That makes it possible to deploy any model on any infrastructure with a performance guarantee. 𝙎𝙤𝙪𝙧𝙘𝙚 - Official Red Hat sources. Link in comments

36

3 Comments

𝗪𝗵𝗮𝘁 𝘂𝘀𝗲𝗱 𝘁𝗼 𝗴𝗲𝘁 𝗽𝗲𝗼𝗽𝗹𝗲 𝘁𝗲𝗰𝗵 𝗷𝗼𝗯𝘀 𝗻𝗼 𝗹𝗼𝗻𝗴𝗲𝗿 𝘄𝗼𝗿𝗸𝘀. Learning a little Linux, a touch of AWS, and a bit of Terraform; for example, is not enough today. That approach might have opened doors in the past, 𝗯𝘂𝘁 𝗻𝗼𝘁 𝗮𝗻𝘆𝗺𝗼𝗿𝗲. Automation and AI have replaced many of the entry-level roles that once helped people get started. Employers now look for professionals who already 𝗽𝗲𝗿𝗳𝗼𝗿𝗺 𝗮𝘁 𝗮 𝗵𝗶𝗴𝗵 𝗹𝗲𝘃𝗲l. If you want a great tech job, do not aim for the bottom. That path has mostly disappeared. Train to 𝗯𝗲 𝗶𝗻 𝘁𝗵𝗲 𝘁𝗼𝗽 𝟮𝟱 𝗽𝗲𝗿𝗰𝗲𝗻𝘁. Build real skills. Position yourself to start mid-career, not entry-level. And make sure those skills match the 𝗿𝗼𝗹𝗲 𝘆𝗼𝘂 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝘄𝗮𝗻𝘁, be focused. That means getting real, honest feedback from someone who's been succeful in that role for many years. Find out 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲 𝗷𝗼𝗯 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗿𝗲𝗾𝘂𝗶𝗿𝗲𝘀, then train for that, not just what looks popular online. Let me know what you're doing to be in the top 25% of your desired role, in the comments below! #TechCareers #CloudComputing #CareerDevelopment #CareerGuidance

58

17 Comments

Agents can now manually prioritize the next important task, email, or chat in a queue. For example, when a customer calls in to enquire about their previously submitted refund request, an agent can search for any pending tickets related to the case, assign it to themselves, and resolve it immediately. https://lnkd.in/gEvgZ8zd

24