Future Forward - Emerging Tech & AI Newsletter - 17th Edition

Welcome to the 17th Edition of Future Forward - the Emerging Tech & AI Newsletter!

This newsletter aims to help you stay up-to-date on the latest trends in emerging technologies. Subscribe to the newsletter today and never miss a beat!

Subscribe to the newsletter here.

Here's what you can expect in each new issue of the Emerging Tech & AI Newsletter:

• A summary of the top AI / emerging technology news from the past week
• Introductory details/Primer on any emerging technology or a key topic in AI (We explore Large Vision Models this week)
• Examples of how AI is being used or How it will impact the future ( We explore Coding in the AI era )

Last Week in AI/Emerging Tech

The field of AI is experiencing rapid and continuous progress in various areas. Some of the notable advancements and trends from the last week include:

Big Tech in AI:

1. Google’s AI-assisted NotebookLM note-taking app is now open to users in the US.
2. Google reveals its most powerful AI model Gemini. Admits AI viral video was edited to look better.
3. Meta and Microsoft say they will buy AMD’s new AI chip as an alternative to Nvidia’s.
4. New Microsoft Purview features use AI to help secure and govern all your data.
5. Microsoft is poised to roll out AI tools for CFOs.
6. Meta announced a bunch of updates to its generative AI tools.
7. Meta and IBM launch ‘AI Alliance’ to promote open-source AI development.
8. Meta releases PurpleLlama for safe and responsible AI development.
9. Meta released SeamlessExpressive to create translations that follow your style.
10. Apple releases MLX, ML framework for Apple Silicon.
11. Amazon to introduce advanced coding and AI modules in 100 Karnataka schools.

Funding & VC Landscape:

1. GenAI startup Sarvam AI raises $41 mn in a funding round led by Lightspeed.
2. San Francisco-based AssemblyAI, which builds speech Al models, raised a $50 million Series C
3. SenseAI Floats $25 Mn Maiden VC Fund To Back Indian AI Startups.
4. Pimento, the French GenAI startup captures $3.2M for transforming ideation and moodboarding.
5. AI-Powered Alternative Investments Assistant Helix by HL Secures $6M Seed+ Funding Led by FINTOP Capital.
6. French AI start-up Mistral secures €2bn valuation.
7. Yellow.ai raises $20 Mn Series B funding led By Lightspeed.
8. VAST Data Closes Series E Funding Round, Nearly Triples Valuation to $9.1 Billion.
9. Harriet, the AI assistant transforming HR teams gets £1.2M funding boost.
10. Mulberri’s AI-driven insurance tech attracts $6.75m in Series A funding.

Other AI news:

1. New AI model Magic Animate can help animate anything.
2. McDonald’s says it will apply generative AI to its operations starting in 2024.
3. AI helps in understanding whale languages.
4. AI networks are more vulnerable to malicious attacks than previously thought.
5. BlackRock to Roll Out Generative AI Tools Next Month.

Liked the news summary. Subscribe to the newsletter to keep getting updates every week. Check the comments section on the LinkedIn article for links to the Other AI news.

Large Vision Models

Just as Large Language Models (LLMs) have revolutionized how we interact with text, LVMs are poised to transform how we understand and interact with the visual world.

What are Large Vision Models?

LVMs are complex deep learning algorithms trained on massive datasets of images and videos. These datasets contain millions of images labelled with detailed information about their content. By analyzing these images, LVMs learn to recognize patterns and relationships, allowing them to perform a wide range of tasks, such as:

• Image classification: Identifying objects, scenes, and activities in images.
• Object detection: Locating and recognizing specific objects within an image.
• Image segmentation: Dividing an image into different regions based on content.
• Image generation: Creating entirely new images based on textual descriptions.
• Video understanding: Analyzing and summarizing the content of videos.

Current Applications of LVMs

LVMs are already being used in various applications across diverse industries:

• Manufacturing: LVMs are used for quality control, anomaly detection, and predictive maintenance in factories.
• Healthcare: LVMs are used for medical image analysis, disease diagnosis, and personalized medicine.
• Retail: LVMs are used for product recognition, personalized recommendations, and fraud detection.
• Security: LVMs are used for object detection and anomaly detection in surveillance systems.
• Art and Design: LVMs are used for image generation, style transfer, and creative exploration.

Challenges and Future Directions

Despite their impressive capabilities, LVMs face several challenges:

• Data bias: LVMs can perpetuate biases present in the data they are trained on.
• Explainability: It can be difficult to understand how LVMs make decisions, which can raise concerns about transparency and fairness.
• Computational cost: Training and running LVMs require significant computational resources, limiting their accessibility.

LVM examples:

Here are some of the latest LVMs:

Vision Transformer (ViT): This LVM broke away from the traditional convolutional neural networks (CNNs) and uses a transformer architecture, achieving state-of-the-art performance in image classification and object detection tasks. ViT models like DeiT, Swin Transformer, and ViT-G/L continue to push the boundaries of performance and efficiency.

CLIP: This LVM excels at aligning image and text data, allowing for tasks like image captioning, visual question answering, and zero-shot learning. New CLIP-based models like UNIMO and ALIGN explore additional functionalities like image generation and attribute manipulation.

LaMDA-ViT: This LVM combines the strengths of Google's LaMDA language model with a ViT visual model, enabling it to understand and respond to complex image-based queries and generate creative text descriptions of images.

PaLM-ViT: Similar to LaMDA-ViT, this model combines PaLM (another powerful LLM) with a ViT model, leading to even more advanced capabilities in image understanding and generation, including image editing, style transfer, and video captioning.

VQ-VAE-2: This LVM utilizes a variational autoencoder architecture for image generation, allowing it to create high-quality and diverse images with greater control over details and style.

Diffusion Models: These models, like Dall-E 2 and Imagen, use a diffusion process to gradually generate images from noise, offering impressive photorealism and artistic potential.

BEiT: This model focuses on pre-training LVMs on massive datasets of unlabeled images, enabling them to learn generalizable representations useful for diverse downstream tasks without the need for extensive labelled data.

LVMs are also expanding beyond image analysis, tackling tasks like video understanding and multi-modal learning. Models like Video Swin Transformer and MEGATRON-Turing NLG are leading the way in these areas. CogVLM is a powerful open-source visual language model (VLM). CogVLM-17B has 10 billion vision parameters and 7 billion language parameters.

Curious to know more? Let us know what follow-up details you would like in the comments and we will plan a detailed article on Tecnologia.

Coding in the AI Era

We wrote a primer on AI-assisted Software Engineering in our fourth edition. Since then there has been tremendous inquiry around the code generation aspect. Tool-based support in writing code has a history spanning over 40 years, originating with features such as syntax highlighting, autocompletion in Integrated Development Environments (IDEs), and code analysis with Linting. In the more recent past, tools like DeepCode (now Synk) have utilized machine learning to provide more sophisticated and intelligent coding suggestions.

AI-assisted coding is not new but has received tremendous attention since the release of tools that can generate code based on text prompts. In Dec 2022, Google DeepMind 's AlphaCode wrote computer programs at a competitive level and has achieved a rank in the top 54%. AlphaCode uses transformer-based language models to generate code at an unprecedented scale, and then smartly filters to a small set of promising programs.

(A) In pretraining, files from GitHub are randomly split into two parts. The first part goes to the encoder as input, and the decoder is trained to produce the second part. (B) In fine-tuning, problem descriptions (formatted as comments) are given to the encoder, and the decoder is trained to generate the solutions. (C) For evaluation, AlphaCode generates many samples for each problem description, then it executes them to filter out bad samples and cluster the remaining ones before finally submitting a small set of candidates.

Source - https://www.science.org/stoken/author-tokens/ST-905/full

OpenAI 's ChatGPT(Codex model ) triggered rapid advancements in the area of CodeLLMs. In August this year, Meta released Code Llama which scored 53.7% on HumanEval and 56.2% on MBPP. Github's Copliot has become the world's most adopted AI developer tool. Google offers similar services through Vertex AI and Amazon Web Services (AWS) has CodeWhisperer.

Here are some of the most common use cases these AI-based coding tools/CoPilot's serve:

1. Converse about code base - Whether you’re hunting down a bug or designing a new feature, you can ask relevant queries from CoPilots and get answers.
2. Improve code quality and security. Some model targets the most common vulnerable coding patterns like hardcoded credentials, SQL injections, and path injections. Many models identify and rectify any errors in code thereby improving overall quality.
3. Ask for suggestions - No need to google or stack overflow, get all general programming questions answered.
4. Help with documentation.
5. Help with code review based on general rules and pre-defined project-specific rules
6. Code Summarization
7. Code generation - Generate code suggestions ranging from snippets to full functions.

Many enterprises have also started using open-source models/ paid models to create bespoke solutions that fit their needs of code generation and other use cases as mentioned above. There are several wrappers available to build something similar to GitHub CoPilot - Faux Pilot and continue.

We covered LLMs from big tech AI companies above. Here's a list of additional LLMs :

1. Reface Code LLM: 1.6b parameters, 20 programming languages, 4096 tokens context, code completion and chat capabilities, pre-trained on permissive licensed code and available for commercial use.
2. StarCoderBase: 15.5B parameter, 80 programming languages, context window of 8192 tokens, trained using the stack. There is another variant named starcoder which is co-developed by ServiceNow .
3. Replit code: This is a foundational model focussed on code completion.
4. Stable Code: This product is designed to assist programmers with their daily work while also providing a great learning tool for new developers ready to take their skills to the next level.
5. WizardCoder: It is built by adapting the Evol-Instruct method specifically for coding tasks. This involves tailoring the prompt to the domain of code-related instructions. Different models use different Code LLMs, StarCoder or Code LLama for fine-tuning. The WizardCoder-Python-34B-V1.0 has a very high HumanEval score of 73.2.
6. CodeT5 and CodeT5+: CodeT5 and CodeT5+ models for Code Understanding and Generation from Salesforce Research.

Here's how different code LLMs compare

Source - Awesome LLM on github

Challenges: LLMs, or Language Models, are known to occasionally produce hallucinations, and this phenomenon is not exclusive to code-generating LLMs. Similar to other instances of LLM hallucinations, the generated code may appear well-structured and function adequately, but it might not perform the intended actions, leading to the creation of subtle bugs that can be challenging to identify.

Coding in the AI era: The Code LLM technology is improving day by day. In the AI era, development teams must adapt to this new way of AI-assisted coding. They should learn how to best work with code-generating LLMs: which LLMs to use, what tooling is available, what prompts should be used to get desired results and how to ensure that there are no errors in the AI-generated code.

Disclosure: Content in Large Vision Models in the article was written with the help of Google Bard.

Thanks for reading. See you next week!

Let's explore the future of technology together!