Leandro von Werra

💻 Why we still need small language models – even in the age of frontier AI Following a focused six-week sprint, our Project t0 team showed that small, open-weight language models can achieve near-frontier reasoning performance when applied to focused, domain-specific tasks.  By combining retrieval-augmented generation, reasoning trace fine-tuning and budget forcing at inference time, the team has built a 3B parameter model capable of tackling real-world health queries – and is small enough to run locally on a laptop. Federico Nanni, James Geddes, Ryan Chan and Tomas L. from the project team explore the details in this blog: https://bit.ly/4mfv4t7 To support others working on lean model deployment, we’re publishing: ✅ Two technical reports documenting our pipeline, data generation, fine-tuning on HPC infrastructure and evaluation process. ✅ Our code and trained models, open-sourced for reproducibility and reuse. ✅ The code for setting up a simple interface for multi-turn conversational interaction, allowing users to test the model in a realistic setting. This work is designed to support teams in compute-constrained or privacy-sensitive environments to build practical AI assistants that don’t require frontier-scale resources. 🔎 Learn more about our work and read the technical reports: https://bit.ly/4mfv4t7 #AI #ArtificialIntelligence

245

7 Kommentare

gLSTM extends xLSTM to a graph neural network architecture: https://lnkd.in/da3at-Y9 "gLSTM mitigates sensitivity over-squashing and capacity over-squashing." "gLSTM achieves comfortably state of the art results on the Diameter and Eccentricity Graph Property Prediction tasks, and very strong performance on SSSP"

291

5 Kommentare

Our team has built the first Relational Foundation Model (KumoRFM) – a foundation model purpose-built for relational databases (https://kumorfm.ai). Just as LLMs transformed unstructured data like text and images, KumoRFM brings the power of foundation models to structured, relational, table-based data. 🔍 What does this mean? Instead of building and tuning separate machine learning models for every predictive task (fraud detection, churn prediction, recommendations, etc.), enterprises can now query their data warehouse directly—in natural language or with our Predictive Query Language (PQL)—and get predictions in seconds. 🧠 Under the hood, KumoRFM: - Represents relational data as a temporal heterogeneous graph. - Uses a Relational Graph Transformer for in-context learning. - Seamlessly adapts to unseen database schemas and diverse predictive tasks—no manual tuning required. ⚡ Key benefits: - Accurate predictions for any predictive task across industries (finance, e-commerce, healthcare). - Trust and transparency via explanations and quantitative evaluation. - Flexible deployment: Kumo SaaS or Data Warehouse Native (Snowflake, Databricks). I’m proud to have co-authored the KumoRFM paper, which details how we’re rethinking AI for enterprise data. This is just the beginning of a new era where relational data becomes truly predictive. 👉 Read more about the work and see the full article at Blocks & Files: https://lnkd.in/d4vTsUMT

304

13 Kommentare