Future Trends in Software Engineering with Generative AI

Andreessen Horowitz shared their enterprise adoption report for GenAI last week, and here's some key trends they've shared+I've observed as a GenAI consultant. Growth in Generative AI (GenAI) Adoption: Generative AI consumer spend exceeded $1 billion quickly in 2023. Anticipated enterprise revenue from GenAI expected to surpass consumer market in 2024. Initial Enterprise Engagement with GenAI: Mostly limited to a few obvious use cases and "GPT-wrapper" products. Skepticism existed regarding GenAI scaling in enterprises and its profitability. Increasing Enterprise Resource Allocation to GenAI: Significant increase in budgets for GenAI within six months; nearly tripling in some cases. Expansion into a variety of use cases and transitioning workloads into production. GenAI considered a strategic initiative; foundational models being built and deployed. Budget Allocation and Return on Investment (ROI): Average spend on GenAI in 2023 was $7M among surveyed companies. Future spending projected to increase 2x to 5x in 2024. Budget reallocation from one-time innovation funds to recurring software lines. ROI measurement focuses on productivity, customer satisfaction, revenue generation, savings, efficiency, and accuracy. Talent and Implementation Needs: Demand for highly specialized technical talent to scale GenAI solutions. Professional services offered by model providers for custom development are in demand. Trends Towards Multi-Model and Open Source: Enterprises are adopting multiple models to avoid vendor lock-in and stay ahead. A shift from dominance of closed-source models towards open-source adoption is notable. Preference for open-source due to control, customization, and security concerns. Customization and Cloud Influence: Enterprises prefer fine-tuning over building models from scratch. Cloud service providers influence purchasing decisions, with preferences divided by CSP loyalty. Early Features and Model Performance: Early-to-market features and model performance are key factors in adoption. Perception that model performances are converging, especially after fine-tuning. Designing for Flexibility: Applications are being designed for easy model interchangeability to avoid dependency. Building In-House Versus Buying: Enterprises focus on building in-house applications, incorporating APIs from foundational models. Potential shift expected when enterprise-focused AI apps enter the market. Internal Versus External Use Cases: Greater enthusiasm for internal use cases due to concerns about public perception and safety. Cautious approach to deploying genAI in consumer-facing sectors due to risks. Market Opportunity and Future Growth: Model API and fine-tuning market projected to reach $5B run-rate by end of 2024. Increase in genAI deal size and faster closure times indicating rapid market growth. Wider opportunities beyond foundational models, including tooling, model serving, and application building.

Happy New Year! If you are an Enterprise CTO, you are probably thinking about your GenAI strategy. Here's a decent write-up by Gartner: https://gtnr.it/3RQodsK. To augment that, here are some #genai trends to track and act on in 2024: --Open and Smaller Models: Open models like Llama, Mistral, BERT, and FLAN are becoming competitive with larger, closed-source models. They're suitable for many use cases and offer transparency for Responsible AI. In my opinion, open & closed models are not in a zero-sum game; BOTH should be used for the right use case. Action: Implement a clear plan for using different models. Amazon Web Services (AWS) users can leverage Bedrock & SageMaker (https://bit.ly/3vkX4qa). -- Domain-Adapted Models: Use your enterprise proprietary data to extend a large language model via continued pre-training (CPT) for domain-specific tasks. Action: Assess your use cases and data for CPT alongside Fine-tuning. Learn more: https://lnkd.in/eNjbQm-m -- Multi-modal Models (MMMs): MMMs will gain prominence in 2024. Both commercial (like GPT-4V) and open-source models (like Llava) will be popular. Action: Expand into business cases served by MMMs. More about Llava: https://lnkd.in/eTvn82iM. -- AI Agents (RAG+++): AI agents using LLMs can improve upon RAG by intelligently utilizing multiple data sources. Action: Prepare APIs and Data Sources for AI agents. More information: https://lnkd.in/eVf_J_bA. -- LLMs with Graphs: Graphs are one of the best representations of real-world knowledge which when combined with LLMs can be very effective in various domains. Action: Identify suitable business cases and explore Graphs+LLMs. Details: https://lnkd.in/eF68FbVA. -- AI Routers: Most enterprises will end up using a dozen or more models and it will become necessary to manage multiple models - Auth, Audit and Smart Model Selection. Action: Build an AI router. AWS Bedrock can assist, but more is needed. Info: https://go.aws/4aFLkyA. -- FinOps meets MLOps: Focus on cost optimization for GenAI projects. 2023 was all about GenAI POCs; 2024 will be about production & big bills! Action: Learn about GenAI business cases and FinOps for GenAI: https://lnkd.in/ezfV8NTa. --Make AI Invisible. Technology is at its best when its invisible and seamless to the end user. Action: Look at existing enterprise applications and look for ways to rethink the user experience using GenAI while keeping the tech invisible. (https://bit.ly/3vkXvAO) What are you tracking? Watch out for more on domain-focused AI trends in areas like AI for the Edge, robotics, and Drug discovery etc. in upcoming posts.

Generative AI's multi billion $ problem I've spent the last 3 months meeting CTOs, CIOs and technology leaders at mega enterprises as well as leading tech cos. All of them share one common concern with generative AI and copilot - how do we measure business level outcomes? Many organizations are piloting copilots that promise to make their employees more productive. These copilots help write better emails, create better slides, write blog articles, summarize meeting transcripts. In terms of usage, reality seems divorced from hype. It seems that after an initial burst of engagement, ongoing usage is low at this stage. That is reasonable to expect. However, even when there is ongoing usage in pockets, business leaders are struggling to understand the impact at a business outcomes level. If an employee is writing better emails, how does that tangibly improve business results? This question is fundamental to justifying hefty price tags associated with copilots. So what are forward-thinking CIOs doing? Here are the five steps. 1. Focus on business value - They are focusing their teams on identifying end to end value streams in their business. These range from the lead to cash process, software development life cycles, employee service delivery, customer support delivery, etc. 2. Research value streams - They are organizing their teams to identify work process within these value streams that lends itself to better / efficient output through generative AI solutions 3. Experiment - they are running hundreds of experiments targeting these work processes within these value streams eg test case development, sales email generation, calls summarized to opportunity updates in CRM, gen aI for data analysis. 4. Value and feasibility analysis - These experiments help teams understand the value and difficulty of applying GenAI to end to end value streams. 5. Roadmap development - based on experiments and understanding of end to end value streams, CIOs are developing a roadmap for the future of GenAI in their organizations which will help them deploy these solutions with conviction. In our Moveworks world, we are increasingly hearing CIOs set audacious goals for building a Generative AI service desk. This goal often takes the shape of "zero service desk", or "touch less service delivery". I'm proud that many customers are well on their way to this goal - and I predict that by 2025, most of our customers would have eliminated L1 service desks entirely, and reduced L2 /L3 by 50%. Generative AI has real value at the enterprise level, and individual productivity copilots are merely the obvious (but not so useful) starting point.

I  recently joined the "Decoding GenAI" program for a fascinating discussion with Eugina Jordan and Mariana Saddakni about the transformative effects of Generative AI (GenAI) on software engineering. The episode explored how GenAI is impacting developers across three categories: 1️⃣GenAI Engine and Large Language Model (LLM) Developers: These researchers at companies like OpenAI, Meta (Llama3), and Google (Gemini) are building the core foundations of GenAI technology. 2️⃣AI Application Developers: These engineers leverage tools like the LLMs we mentioned before, TensorFlow, and others to create groundbreaking AI applications in areas like image recognition, recommendation engines, scientific computing, and many more. 3️⃣Software Engineers (Non-AI Focused): This is the largest group and is being significantly impacted by GenAI in different ways than the second group. Their work is evolving as GenAI automates repetitive tasks and introduces innovative approaches to software development. We talked about  the rise of the AI assistant: GenAI-powered tools like Microsoft Github Copilot, Amazon Q, IBM WatsonX, and Tabnine are becoming invaluable allies for software engineers. These AI assistants handle tasks like: ✅ Developing code snippets: They can generate boilerplate code, suggest function calls, and complete code based on context. ✅ Automating tests: They help write unit tests and integration tests, streamlining the testing process. ✅ Debugging: They can help identify and fix bugs in code. ✅ Performing code reviews: They can analyze code for potential issues and suggest improvements. ✅ Creating documentation: They can automatically generate documentation from code, saving developers valuable time. We talked about the challenges and solutions. One of the key takeaways is that this is just the beginning. We can expect even more powerful automation capabilities as GenAI technology continues to develop. Link to the full episode in the comments. I'd love to hear your thoughts on how GenAI is impacting your work. Share your experiences and predictions about the AI-powered future in the comments below! #aiforleaders #ai #artificialintelligence   _______________ ➡️ About Me: I'm Talila Millman a management advisor, keynote speaker, and executive coach. I help CEOs and C-suites create a growth strategy, increase profitability, optimize product portfolios, and create an operating system for excellence. 📘 Get My Book: "The TRIUMPH Framework: 7 Steps to Leading Organizational Transformation" launched as the Top New Release on Organizational Change 🎤 Invite me to Speak at your Event about Leadership, Change Leadership, Innovation, and AI Strategy

What is the future direction of Generative AI? Considering the insights from the latest research, here are three prominent themes shaping the future trajectory of Generative AI: 𝟭. 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗺𝗲𝗻𝘁 𝗶𝗻 𝗦𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘇𝗲𝗱 𝗔𝗜 𝗦𝘆𝘀𝘁𝗲𝗺𝘀: This includes the development of AI systems tailored for specific tasks or domains, showcasing significant improvements in performance and efficiency without relying on traditional methods. Examples include “Grandmaster-Level Chess Without Search” by Google Deepmind team “DeepSeekMath” by DeepSeek AI focusing on mathematical reasoning. 𝟮. 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗜𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝗼𝗻 𝘄𝗶𝘁𝗵 𝗘𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝗦𝘆𝘀𝘁𝗲𝗺𝘀: This theme highlights the growing ability of AI to interact with, and leverage, external systems and databases, thus broadening its application scope and utility in real-world tasks. Examples include “AnyTool” which enables accessing vast range of online data soucrces without additional training and “Corrective RAG” which performs online rearch to enhance retrieved information in RAG systems. 𝟯. 𝗘𝗻𝗵𝗮𝗻𝗰𝗲𝗱 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗖𝗮𝗽𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝗲𝘀: Research is increasingly focused on endowing AI with better reasoning, learning from less data, and adapting to new tasks more flexibly, mirroring aspects of human cognitive processes.Examples include “Indirect Reasoning with LLMs” and “Self-Discovered Reasoning Structures”. In the near term, these themes suggest a convergence towards AI systems that are not only more efficient and specialized but also more integrated with the digital ecosystem and capable of more human-like reasoning. This progress promises to expand AI's utility in solving complex, real-world problems across various domains, leading to innovations in healthcare, finance, education, and more.