Jan Eric Lenssen

DeepSeek just published a wild paper that makes long-context LLMs 10x faster. Not only does it cut costs and boost inference speed, but it also beats full attention models. Here's everything you need to know: Long-context models can change everything—handling full books, giant codebases, and deep reasoning. But standard attention scales poorly and large sequences become slow and expensive to process. DeepSeek's NSA (Native Sparse Attention) changes that. How does it work? NSA blends three techniques: ▸ Dynamic hierarchical sparsity – skips redundant computations ▸ Coarse-grained token compression – groups less important tokens ▸ Fine-grained token selection – keeps critical details sharp It speeds up attention by compressing less important tokens, selecting only the most relevant ones for precise attention, and using a sliding window to retain local context. This reduces unnecessary computations but preserves long-range relationships. How fast is it? NSA crushes full attention in efficiency: ▸ 11.6x speedup in decoding ▸ 9.0× speedup in forward propagation ▸ 6.0× speedup in backward propagation How accurate is it? NSA outperforms Full Attention on 7 out of 9 benchmarks, achieving higher overall performance. It also shows big improvements in reasoning tasks, with a +4.2% gain in complex reasoning and +3.4% in problem-solving Most sparse attention methods only optimize for inference. NSA is trainable by nature, meaning it slashes pretraining costs while keeping model quality high. It’s also built for real-world performance to maximize GPU efficiency—making it even faster on modern hardware. Read the paper: https://lnkd.in/gNTYCUGx ↓ Are you an AI developer? Check out https://AlphaSignal.ai to get a daily summary of breakthrough models, repos and papers in AI. Read by 200,000+ devs.

1.076

36 Kommentare

NVIDIA 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗱𝗿𝗼𝗽𝘀 𝗻𝗲𝘄 𝗽𝗮𝗽𝗲𝗿 𝗼𝗻 “𝗦𝗺𝗮𝗹𝗹 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗙𝘂𝘁𝘂𝗿𝗲 𝗼𝗳 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜”. ⬇️ Everyone is chasing bigger models. But this paper argues the exact opposite: Most agent workloads don’t need 175B params — they need precision, speed, and control. 𝗧𝗵𝗲 𝗸𝗲𝘆 𝘁𝗮𝗸𝗲𝗮𝘄𝗮𝘆? Small Language Models (SLMs) are not only good enough — they’re better for the majority of agentic use cases. 𝗛𝗲𝗿𝗲 𝗶𝘀 𝗮 𝗾𝘂𝗶𝗰𝗸 𝘀𝘂𝗺𝗺𝗮𝗿𝘆 𝗼𝗳 𝘁𝗵𝗲 𝗸𝗲𝘆 𝗽𝗼𝗶𝗻𝘁𝘀: ⬇️ 1. SLMs can already match or beat 30–70B LLMs on task-specific reasoning  → From Phi-3 to DeepSeek Distill, we now have 2–9B models outperforming legacy LLMs with 10–70× faster inference. 2. Most agents just run repetitive, scoped tasks  → Parsing. Routing. Tool calls. Summaries. You don’t need an all-knowing LLM — you need a fast, fine-tuned SLM that gets the job done. 3. LLMs are economically unsustainable at scale  → They dominate cloud costs and energy use. SLMs offer massive savings in latency, memory, and operational overhead. 4. SLMs run on edge and consumer devices  → Tools like ChatRTX show real-time agents can live on laptops or embedded systems — without phoning home to a GPU cluster. 5. Heterogeneous agent stacks are the path forward  → Use LLMs sparingly for general reasoning. Let SLMs handle 80% of workflows. More modular. More efficient. More robust. 6. SLMs are easier to fine-tune and align  → Lower hallucination risk, tighter output control, and better format consistency. Perfect for tool-driven agent environments. More in the comments and the paper below to download — but I’ll say this now: This paper might age like gold for every team trying to ship serious agents in production. 𝗜 𝗲𝘅𝗽𝗹𝗼𝗿𝗲 𝘁𝗵𝗲𝘀𝗲 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁𝘀 𝗮𝗿𝗼𝘂𝗻𝗱 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀 — 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲𝘆 𝗺𝗲𝗮𝗻 𝗳𝗼𝗿 𝗿𝗲𝗮𝗹-𝘄𝗼𝗿𝗹𝗱 𝘂𝘀𝗲 𝗰𝗮𝘀𝗲𝘀 — 𝗶𝗻 𝗺𝘆 𝘄𝗲𝗲𝗸𝗹𝘆 𝗻𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿. 𝗬𝗼𝘂 𝗰𝗮𝗻 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲 𝗵𝗲𝗿𝗲 𝗳𝗼𝗿 𝗳𝗿𝗲𝗲: https://lnkd.in/dbf74Y9E --- Read the full paper here: https://lnkd.in/gYyzgRzR

674

103 Kommentare

"On Incorporating Scale into Graph Networks" by Christian Koke, SHEN Yuesong, Abhishek Saroha, Marvin Eisenberger, Bastian Grossenbacher-Rieck, Michael Bronstein, Daniel Cremers "Standard graph neural networks assign vastly different latent embeddings to graphs describing the same physical system at different resolution scales. This precludes consistency in applications and prevents generalization between scales as would fundamentally be needed in many scientific applications. We uncover the underlying obstruction, investigate its origin and show how to overcome it." Paper: https://lnkd.in/eDGCPzMd #graphneuralnetworks

69

1 Kommentar

Our paper on “Diffuse Your Data Blues: Augmenting Low-Resource Datasets via User-Assisted Diffusion” is conditionally accepted at #ACM Intl Conf on Intelligent User Interface (SIGAI + SIGCHI), which generally has less than 25% acceptance rate for full papers. Our paper is about generating synthetic data using a bespoke #DiffusionModel and showing it improves performance of Mixed Reality Interaction. The term ‘#AugmentedReality’ is attributed to former Boeing researcher Tom Caudell, who is believed to have coined the term in 1990. When the synthetic imagery is rendered in 3D, the system is often known as Mixed Augmented Reality, Mixed Reality or Augmented Virtuality system. However, the name #MixedReality pushes people towards Unity or Unreal Engine based 3D content development overlooking the fact that the success of any interactive system depends on user acceptance of the accuracy and latency of the system, and for #MR systems the accuracy and latency depends on underlying computer vision system, not that much on 3D content. From 2010 onwards, most practitioners use #DeepLearning based computer vision system requiring large amount of training data for novel applications where pre-trained models are not available. For defense and many industrial applications, it is often not possible to capture images to create training dataset. We investigated synthetic data generation techniques in details and provided multiple solutions as follows Using 3D content from a VR gaming Engine PT Gaikwad.. P Biswas, Developing A Computer Vision based System for Autonomous Taxiing of Aircraft, Aviation, Taylor & Francis 2023, https://lnkd.in/gCyTmpSc  A Mukhopadhyay.. P Biswas, Virtual-reality-based digital twin of office spaces with social distance measurement feature, Virtual Reality & Intelligent Hardware 4(1), 2022, Pages 55-75, ISSN 2096-5796, https://lnkd.in/gdS5c_kE Using GAN based Image Translation Models Y Sahu.. P Biswas, A Comparative Study on Image Translation GAN Models to improve Object Detection Accuracy on Low-Resource Domains, International Conference on Vehicular Technology and Transportation Systems (ICVTTS) 2024, https://lnkd.in/g3XHdpdC Majority of Diffusion Models are prompt based requiring textual description of objects while in our case we work with amorphous parts of objects, which may even be proprietary impeding training public LLMs on textual description #GAN Models often require large amount of data to start with Our research allows to automatically annotate part of image and mix it with either real or synthetic or prompt generated background images. We have also shown two use cases on training standard CV model with such synthetic data and use it in a MR headset. A demonstration video can be found at  https://lnkd.in/gsQ8uagv

32

1 Kommentar

Neural audio codecs have transformed how we process and generate audio, but most struggle significantly with out-of-domain content, leading to error propagation in downstream tasks. Meta's ComplexDec tackles this fundamental challenge by modelling audio in the complex spectral domain. The researchers demonstrate that information loss from temporal and dimensional compression is a key factor degrading out-of-domain robustness. What makes ComplexDec particularly impressive is its architectural approach. Unlike waveform-based codecs, it adopts a fully convolutional architecture without downsampling layers to bypass temporal compression. By coding speech in the complex spectral domain with a 150Hz frame rate and 256-dimension embedding, it maintains the same 24kbps bitrate as baseline models while dramatically reducing compression ratio. The results speak for themselves—trained on just 30 hours of reading-style VCTK corpus speech, ComplexDec shows remarkable robustness when tested on expressive speech from the EARS dataset. It achieves similar performance in both in-domain and out-of-domain scenarios, while baselines like AudioDec and ScoreDec suffer significant degradation. This breakthrough suggests that merely scaling up training data isn't enough to solve the out-of-domain robustness problem. The architecture's fundamental approach to minimising information loss proves more effective than throwing more data at the problem. Paper: https://lnkd.in/eJdaRAus Demo: https://lnkd.in/eZVMRi5G Authors: Yi-Chiao Wu, Dejan Marković, Steven Krenn, Israel D. Gebru, and Alexander Richard #NeuralAudioCodecs #Meta #AudioAI

61

A new rendering model just appeared for full global illumination. It's ... a transformer! The start of Mesh-to-image, end-to-end physics based rendering? Let's have a look 👇👇👇 Zhejiang University, Microsoft Research Asia and College of William & Mary present "Transformer-based Neural Rendering of Triangle Meshes with Global Illumination". This paper will be presented at ACM SIGGRAPH 2025. RenderFormer is a neural rendering pipeline that directly renders an image from a triangle-based representation of a scene with full global illumination effects and that does not require per-scene training or fine-tuning. Instead of taking a physics-centric approach to rendering, they formulate rendering as a sequence-to-sequence transformation where a sequence of tokens representing triangles with reflectance properties is converted to a sequence of output tokens representing small patches of pixels. RenderFormer follows a two stage pipeline: a view-independent stage that models triangle-to-triangle light transport, and a view-dependent stage that transforms a token representing a bundle of rays to the corresponding pixel values guided by the triangle-sequence from the the view-independent stage. Both stages are based on the transformer architecture and are learned with minimal prior constraints. No rasterization, no ray tracing Amazing stuff, more info in the comments #globalillumination #neuralrendering #transformers #machinelearning #endtoend #deeplearning

335

11 Kommentare

Why #SmallLanguageModels are the real future? ⚡ Most agents route every task, no matter how trivial - through GPT-4 or Claude → That’s expensive, slow, and overkill. ⚡ NVIDIA’s new paper argues→ #SLMs models small enough to run on consumer hardware are faster, cheaper, and just as capable for most agentic tasks. ⚡ Prioritize SLMs for cost-effective deployment → Cut latency, energy use, and infra costs, especially for real-time or on-device inference. ⚡ Design modular agentic systems → Use SLMs for routine, narrow tasks; reserve #LLMs only for complex reasoning. ⚡ Leverage SLMs for rapid specialization → Fine-tune small models quickly for evolving use cases, enabling faster iteration and easier adaptation. ⚡Agentic workloads are rarely open-ended conversations. They’re structured, repetitive, and scoped: ✸ Summarize a doc. ✸ Extract a field. ✸ Call an API. ✸ Generate boilerplate code. For these, you don’t need a trillion-parameter giant. A well-tuned 2–7B model often does the job just as well or better. ⚡ Proof in Numbers: ✸ Phi-3 Small (7B) rivals 70B on reasoning + code. ✸ DeepSeek-R1-Distill (7B) beats Claude 3.5 + GPT-4o on reasoning. ✸ SmolLM2 (≤1.7B) matches 70B models from 2 years ago. ✸ Nemotron-H (2–9B) delivers 30B-level tool use at a fraction of compute. ✸ Toolformer (6.7B) outperforms GPT-3 (175B) by learning APIs. Smaller ≠ weaker. ⚡The Efficiency Edge: ✸ 10–30× cheaper to run. ✸ Lower energy use + faster response. ✸ Overnight fine-tuning with LoRA/QLoRA. ✸ More reliable with strict formats (JSON, XML, Python). ✸ Deployable locally for cost savings + data control. ⚡Smarter Architectures: Don’t build monolithic agents. ✸ Use SLMs by default. ✸ Call LLMs only when absolutely necessary. This modular approach is cheaper, more controllable, and easier to debug. ⚡Migration path (LLM → SLM): ✸ Log usage data. ✸ Cluster recurring tasks. ✸ Fine-tune SLMs. ✸ Replace LLM calls. ✸ Iterate + refine. ⚡Real-world Results: ✸ MetaGPT → 60% of LLM calls replaceable. ✸ Cradle → 70%. ✸ Open Operator → 40%. ⚡ Why hasn’t Flipped yet: ✸ Billions sunk into LLM infra. ✸ Benchmarks biased toward generalist models. ✸ Less hype for SLMs But none of these are technical blockers. ⚡ The Takeaway: ✸ The future of agents isn’t bigger models. ✸ It’s smarter architectures built on SLMs. ✸ SLMs give you speed, control, and affordability.

57

6 Kommentare