Timothee Lacroix

🚀 Anthropic just upgraded Claude Sonnet 4.6 — and it’s built for serious agentic workloads. The new version focuses on what actually matters in production: 🔹 Plans complex tasks more carefully and sustains long-running workflows 🔹 More reliable across massive codebases and large projects 🔹 Better self-correction — catches mistakes earlier 🔹 1M-token context window (beta) for handling huge documents and sessions 🔹 Strong gains across coding, reasoning, knowledge work, and agentic search Anthropic is also expanding integrations across Excel, PowerPoint, Claude Code, and the API — pushing Claude further into real workplace automation. The trend is clear: Models aren’t just getting smarter… they’re getting built for long-running autonomous work. #AI #Claude #Anthropic #AIAgents #LLM #GenAI https://lnkd.in/gZ9qMQdT

27

🚀 Can RL alone unlock better reasoning in LLMs? Mistral’s new Magistral shows it might — boosting AIME 2024 accuracy from 26.8% → 73.6% with zero distillation or teacher models. Insights: - GRPO tweaks remove the critic and scale up RL efficiently on massive clusters. - Text-only RL surprisingly preserved vision + coding skills — a kind of "multimodal free lunch." - The 24B Magistral Small weights are Apache-2 licensed, so enterprises can fine-tune freely. - Rather than relying on existing LLMs, Magistral learns entirely from self-generated synthetic reasoning data. A bold direction — and it works. 🧪 Perhaps most refreshingly, the paper openly shares failed experiments and learnings — a rare level of transparency in frontier research. Full paper link in comments 👇

18

3 commentaires

🔍 Failing fast can teach us more than early success. The Magistral team openly shares experiments that didn’t work—and that’s pure gold for researchers and builders. 🚫 What didn’t pan out: 1. KL‑penalty removal without tuning: Simply dropping the KL divergence term led to unstable training. It required carefully adjusted clipping and normalization to regain stability 2. Over‑simplified reward shaping: Omitting individual reward terms (formatting, multilingual, correctness) degraded specific capabilities—demonstrating that each reward axis is vital 3. Ablations on small vs. medium models: On smaller models, pure-RL without any trace-distillation underperformed compared to a supervised + RL approach—showing that scale matters 🎯 Key takeaways: - RL stability needs balance—clipping alone isn’t enough without careful loss/advantage normalization. - Reward shaping must be multi-faceted—dropping a single axis can hurt downstream performance. - One-size doesn't fit all—the best recipe depends on model size and existing capabilities. By publishing what didn't work, Magistral offers a roadmap of pitfalls—and how to avoid them—on the path to pure‑RL reasoning.

11

In case you missed it last week: French lab pleias released a super small language model (SLM) trained mostly on synthetic data that outperforms much larger models in its class. Meet 𝗕𝗮𝗴𝘂𝗲𝘁𝘁𝗼𝘁𝗿𝗼𝗻: 321 million parameters (~5,000x smaller than GPT-4), trained on 200 billion tokens (30–60× less than frontier models) and creating a new efficiency frontier. The interesting part? It was trained largely on AI-generated data: reasoning traces generated by other LLMs rather than web content (𝗦𝗬𝗡𝗧𝗛). Since these models don't train on as much web knowledge, it is even more important for them to search the Internet at the right moment. Fortunately, there's a bunch of companies (𝘤𝘰𝘶𝘨𝘩 Linkup 𝘤𝘰𝘶𝘨𝘩) that build APIs to power exactly that. I think we're going to see a lot more of these ; as Andrej Karpathy has been saying, these smaller models are much more efficient if you give them the right retrieval tools. Exciting times 🥖 cc Anastasia Stasenko Pierre-Carl Langlais https://lnkd.in/eUbKtJVD

75

2 commentaires

If you are interested in human-level AI, don't work on LLMs. This was one of Yann LeCun’s core messages during his September 16 Harvard CMSA talk on Self-Supervised Learning, JEPA, and the Future of AI. Key takeaways: → Text-only AI won’t get us to human-level intelligence Typical LLMs are trained on ~30 trillion tokens, an amount of text a human would need 450,000 years to read. Yet a 4-year-old child absorbs roughly the same volume of data in just 16,000 hours, and in far richer sensory form: vision, motion, physical interaction. Thus, intelligence needs redundancy, perception, and grounded experience, not just text. → Feed-forward inference is fundamentally limited Current models rely on a feed forward propagation through a fixed number of layers to perform inference. This is too weak for real reasoning. LeCun argues that intelligent systems need inference through optimization: searching for an output that minimizes an “energy” or incompatibility function (EBMs). This allows zero-shot reasoning without exhaustive training. → Predicting at pixel-level is wrong problem Predicting pixels, particularly future video frames directly, is mathematically intractable. The world is too high-entropy at the pixel level. LeCun’s proposal: JEPA (Joint Embedding Predictive Architecture), which predicts in representation space that captures relevant structure. Intelligence depends on abstract representations that allow us to predict, not reconstructing every detail. → Auto-regressive generation causes hallucinations by design LLMs predict the next token using their own previous predictions. Thus, errors accumulate and drift grows. Hallucination isn’t an accident; it’s baked into the mechanism. → Reinforcement Learning is useful but inefficient RL requires enormous numbers of trials. According to LeCun, it should be a tool of last resort, not the foundation of intelligence. → Humans and animals build layered world models. Machines don’t (yet) Infants form internal models of motion, inertia, and cause-and-effect within their first 9 months. This understanding is hierarchical, operating across multiple levels of abstraction, and enables humans to plan, predict outcomes, and act intentionally. Current AI systems lack this multi-level structure, limiting their ability to reason about actions or anticipate real-world consequences. This lecture made it clear that the path to human-level intelligence won’t likely come from scaling LLMs, but from controllable & safe systems capable of real reasoning, planning, understanding and learning world models from chaotic environment, just as humans and animals do. Watch the full lecture here: https://lnkd.in/eQMEkTEH #AI #ArtificialIntelligence #AIAcademyAsia #MachineLearning #DeepLearning #AIResearch #AIFuture #AISafety #YannLeCun

8

I stand by my claim from earlier this year that Experiment Tracking should not be part of MLOps. It is for data scientists who need detailed feedback on hyperparameter tuning experiments. That is a rapidly diminishing number of developers. The rest of us should skip experiment tracking and concentrate on first building an end-to-end AI/ML system and then iteratively improving it. If you need to hyperparameter tune your model, use AutoML. If you need to dive in further, use an experiment tracking platform. But please, please, please, do not start your MLOps journey by installing an experiment tracking server - it will slow you down in getting your first working version of your AI/ML system! https://lnkd.in/dH4ArKd2

68

20 commentaires