Sepp Hochreiter

𝗤𝘂𝗮𝗻𝘁𝘂𝗺𝗔𝗻𝗱𝗵𝗿𝗮 – 𝗛𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗠𝗼𝗱𝗲𝗹𝘀 𝗼𝗳 𝗤𝘂𝗮𝗻𝘁𝘂𝗺 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿𝘀 (𝟮𝟬𝟮𝟱) Quantum computers are not all built the same. The way 𝗵𝗮𝗿𝗱𝘄𝗮𝗿𝗲 is designed and controlled defines what kind of 𝗰𝗼𝗺𝗽𝘂𝘁𝗮𝘁𝗶𝗼𝗻 is possible. Broadly, there are 𝟯 𝗵𝗮𝗿𝗱𝘄𝗮𝗿𝗲-𝗯𝗮𝘀𝗲𝗱 𝗺𝗼𝗱𝗲𝗹𝘀 in focus today: 1️⃣ 𝗔𝗻𝗮𝗹𝗼𝗴 𝗤𝘂𝗮𝗻𝘁𝘂𝗺 𝗖𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴 𝗛𝗼𝘄 𝗶𝘁 𝘄𝗼𝗿𝗸𝘀: Directly evolves a 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗾𝘂𝗮𝗻𝘁𝘂𝗺 𝘀𝘆𝘀𝘁𝗲𝗺 in time by tuning control parameters. 𝗛𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲𝘀: Quantum annealers (𝘀𝘂𝗽𝗲𝗿𝗰𝗼𝗻𝗱𝘂𝗰𝘁𝗶𝗻𝗴 𝗳𝗹𝘂𝘅 𝗾𝘂𝗯𝗶𝘁𝘀). 𝗦𝘁𝗿𝗲𝗻𝗴𝘁𝗵𝘀: Fast, efficient for specific tasks like 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 and 𝗾𝘂𝗮𝗻𝘁𝘂𝗺 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻. 𝗟𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀: Specialized hardware, not suited for 𝘂𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹 𝗰𝗼𝗺𝗽𝘂𝘁𝗮𝘁𝗶𝗼𝗻. 2️⃣ 𝗡𝗜𝗦𝗤 (𝗡𝗼𝗶𝘀𝘆 𝗜𝗻𝘁𝗲𝗿𝗺𝗲𝗱𝗶𝗮𝘁𝗲-𝗦𝗰𝗮𝗹𝗲) 𝗚𝗮𝘁𝗲-𝗕𝗮𝘀𝗲𝗱 𝗛𝗼𝘄 𝗶𝘁 𝘄𝗼𝗿𝗸𝘀: Implements computation through 𝗾𝘂𝗮𝗻𝘁𝘂𝗺 𝗴𝗮𝘁𝗲𝘀 on a limited number of 𝗻𝗼𝗶𝘀𝘆 𝗾𝘂𝗯𝗶𝘁𝘀. 𝗛𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲𝘀: 𝗦𝘂𝗽𝗲𝗿𝗰𝗼𝗻𝗱𝘂𝗰𝘁𝗶𝗻𝗴 𝗰𝗶𝗿𝗰𝘂𝗶𝘁𝘀, 𝘁𝗿𝗮𝗽𝗽𝗲𝗱 𝗶𝗼𝗻𝘀, 𝗻𝗲𝘂𝘁𝗿𝗮𝗹 𝗮𝘁𝗼𝗺𝘀. 𝗦𝘁𝗿𝗲𝗻𝗴𝘁𝗵𝘀: Flexible—can run many algorithms (𝗩𝗤𝗔𝘀, 𝗾𝘂𝗮𝗻𝘁𝘂𝗺 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻, 𝗲𝗮𝗿𝗹𝘆 𝗾𝘂𝗮𝗻𝘁𝘂𝗺 𝗠𝗟). 𝗟𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀: Sensitive to 𝗻𝗼𝗶𝘀𝗲 and 𝗱𝗲𝗰𝗼𝗵𝗲𝗿𝗲𝗻𝗰𝗲; circuit depth limited by 𝗵𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝘀𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆. 3️⃣ 𝗙𝗮𝘂𝗹𝘁-𝗧𝗼𝗹𝗲𝗿𝗮𝗻𝘁 𝗚𝗮𝘁𝗲-𝗕𝗮𝘀𝗲𝗱 𝗛𝗼𝘄 𝗶𝘁 𝘄𝗼𝗿𝗸𝘀: Uses 𝗾𝘂𝗮𝗻𝘁𝘂𝗺 𝗲𝗿𝗿𝗼𝗿 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻 (𝗤𝗘𝗖) to build 𝗹𝗼𝗴𝗶𝗰𝗮𝗹 𝗾𝘂𝗯𝗶𝘁𝘀 from thousands of 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗾𝘂𝗯𝗶𝘁𝘀. 𝗛𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲𝘀: Scaled 𝘀𝘂𝗽𝗲𝗿𝗰𝗼𝗻𝗱𝘂𝗰𝘁𝗶𝗻𝗴 and 𝗶𝗼𝗻-𝘁𝗿𝗮𝗽 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 under development. 𝗦𝘁𝗿𝗲𝗻𝗴𝘁𝗵𝘀: Reliable, scalable, supports algorithms for 𝗰𝗵𝗲𝗺𝗶𝘀𝘁𝗿𝘆, 𝗳𝗮𝗰𝘁𝗼𝗿𝗶𝗻𝗴, 𝗹𝗮𝗿𝗴𝗲-𝘀𝗰𝗮𝗹𝗲 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻. 𝗟𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀: Requires 𝗺𝗶𝗹𝗹𝗶𝗼𝗻𝘀 𝗼𝗳 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗾𝘂𝗯𝗶𝘁𝘀 and complex 𝗲𝗿𝗿𝗼𝗿-𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻 𝗼𝘃𝗲𝗿𝗵𝗲𝗮𝗱. 𝗕𝗶𝗴 𝗣𝗶𝗰𝘁𝘂𝗿𝗲 There is 𝗻𝗼 𝘂𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹 𝗵𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝘄𝗶𝗻𝗻𝗲𝗿 𝘆𝗲𝘁. Each model addresses different challenges. What’s common is the push toward 𝘀𝗰𝗮𝗹𝗮𝗯𝗹𝗲, 𝗲𝗿𝗿𝗼𝗿-𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗲𝗱, 𝗳𝗮𝘂𝗹𝘁-𝘁𝗼𝗹𝗲𝗿𝗮𝗻𝘁 𝘀𝘆𝘀𝘁𝗲𝗺𝘀, with roadmaps pointing to breakthroughs around 𝟮𝟬𝟮𝟲–𝟮𝟬𝟯𝟬 . #QuantumAndhra #QuantumComputing #QuantumHardware #QuantumModels #QuantumTechnology #FutureOfComputing #QuantumInnovation #QuantumResearch #QuantumEngineering #QuantumSimulation #QuantumAlgorithms #QuantumApplications #QuantumAdvantage #QuantumSystems #EmergingTech #DeepTech #TechTrends2025 #NextGenComputing #AnalogQC #NISQ #FaultTolerantQC #SuperconductingQubits #TrappedIons #NeutralAtoms #QuantumDots

91

3 Kommentare

Delighted that our working paper “Catching Bid-rigging Cartels with Graph Attention Neural Networks”, joint work with D. Imhof and E. Viklund, is out! We propose a novel #DeepLearning algorithm based on Graph Attention Neural Networks to detect collusive behavior in markets and tenders. Trained on data from 7 countries across 13 markets, our method achieves impressive accuracy rates of up to 91% in classifying cartels - significantly outperforming classical #MachineLearning approaches: https://lnkd.in/dmnFAt-3 #DataScience

70

3 Kommentare

#variational #QML #strikesback #discretely In the recent work (#link in the first comment) we show that adaptivity in variational quantum algorithms can give an exponential advantage over non-adaptive approaches. By framing circuit recompilation as a discrete optimization problem, we identify a setting where hybrid quantum–classical loops are not just helpful, but necessary. Big thanks to my collaborators — 𝗭𝗼ë 𝗛𝗼𝗹𝗺𝗲𝘀, for super-energising discussions and pushing this through, and Chukwudubem Umeano, for helping to compile this piece together. Careful, long read below ❗️ 𝗔𝗱𝘃𝗮𝗻𝘁𝗮𝗴𝗲 𝗳𝗼𝗿 𝗗𝗶𝘀𝗰𝗿𝗲𝘁𝗲 𝗩𝗮𝗿𝗶𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗤𝘂𝗮𝗻𝘁𝘂𝗺 𝗔𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺𝘀 𝗶𝗻 𝗖𝗶𝗿𝗰𝘂𝗶𝘁 𝗥𝗲𝗰𝗼𝗺𝗽𝗶𝗹𝗮𝘁𝗶𝗼𝗻 The question of whether variational quantum algorithms really need the hybrid quantum–classical loop has been a matter of debate in the last two years. The challenge comes from the ever-growing power of surrogating quantum models, and advances in classical simulation methods for problems with structure. This has been framed as a question to the community: 𝘋𝘰𝘦𝘴 𝘵𝘩𝘦 𝘱𝘳𝘰𝘷𝘢𝘣𝘭𝘦 𝘢𝘣𝘴𝘦𝘯𝘤𝘦 𝘰𝘧 𝘣𝘢𝘳𝘳𝘦𝘯 𝘱𝘭𝘢𝘵𝘦𝘢𝘶𝘴 𝘪𝘮𝘱𝘭𝘺 𝘴𝘪𝘮𝘶𝘭𝘢𝘣𝘪𝘭𝘪𝘵𝘺? Counterexamples exist, but most of them correspond to specific Boolean function learning or Shor-type algorithms in disguise. What we tried to do is to identify a setting where adaptivity appears more naturally, and relies on a different structure. The starting point is a simple observation: in classical discrete optimization, exponential separations between adaptive and non-adaptive search are known. If this logic carries over, then there should be quantum problems where adaptivity gives the same kind of exponential edge. In our preprint, we propose such a problem — quantum circuit recompilation. The task is to recover a “hidden” circuit structure, starting with quantum data (a state) prepared with randomly placed T-gates (“puzzles”) embedded in partially random unitaries. The ansatz has the reverse structure, but we don’t know where the gates are, and there are exponentially many options. We show that discrete quantum optimizers can efficiently unravel these puzzles, finding the right locations with a quadratic number of circuit evaluations. In contrast, non-adaptive searches face an exponentially hard landscape: non-separable, highly entangled, and full of magic. Importantly, we also find that optimization remains feasible under noise, using finite-shot samplers, even as the solution space grows exponentially. This points to a very particular (and largely overlooked) role for discrete optimization in variational quantum algorithms. It also raises a bigger question: what new problems can benefit from this kind of adaptivity? While we remain open to new discoveries, we already see clear connections to fault-tolerant protocols with partial structure, and to blind quantum computing scenarios. Grateful for the support from EPSRC and the QCi3 Hub!

92

7 Kommentare

🧬 The concept of Neural DNA (nDNA), initially proposed by Dr. Amitava Das, opens up a new avenue for interpreting LLM behavior. nDNA treats an LLM as something we can sequence (just like DNA sequencing) by reading its internal signals -- its lineage and layer-level changes. It captures a model’s lineage, tuning-induced mutations, and functional expression so we can compare LLM families, spot drift, and explain why similar benchmarks behave differently across a range of LLMs. In practice, nDNA looks for signals such as: (i) layer shifts, (ii) MoE routing patterns, and (iii) controlled probes. It lets us audit change over time and pick models for tasks with evidence. Collaborating with Dr. Amitava Das, and his Pragya group (https://pragyaai.github.io) at the Department of CSIS BITS Pilani Goa Campus, we've been building and testing the concept of nDNA to understand LLM behavior across major job families: - Llama tends to refine mid–to-late layers while early layers remain stable; - Mistral localizes expertise while the early layers remain steady; - Gemma aligns with light-touch parameter changes; - Qwen rewrites more in its middle layers, especially in multilingual scenarios; - DeepSeek adds dialogue capability in mid layers, while preserving base geometry. One of the other distinct advantages of nDNA is the ability to identify model collapse. On the surface you might see a slow loss of creativity and nuance in a model’s behavior. Inside, through the nDNA lens, the model’s internal landscape flattens -- fewer distinct patterns light up -- much like reduced neural diversity in biology. With nDNA, that shift becomes measurable over time, which makes collapse diagnosable and monitorable. Dr. Amitava Das's novel thought process and insistence on clear hypotheses has led to the inception of nDNA as a practical lens for AI -- one that detects drift before deployment, keeps model lineages auditable, and matches models to tasks based on evidence rather than guesswork. Taken together, these nDNA lenses reframe how we interpret LLM behavior. Our other recent works, DPO-Kernels (ACL 2025; https://lnkd.in/gXGQvXwX), Yin-Yang Align (ACL 2025; https://lnkd.in/gWVCxG8B), QuickSilver (https://lnkd.in/gpZQKMmP), and the Counter Turing Test (EMNLP 2023 Outstanding Paper Award; https://lnkd.in/gaVHeXgJ), share a similar first-principle approach to LLM refinement/understanding. If nDNA is something you'd like to explore further -- we'll be launching a website for nDNA soon, stay tuned! #nDNA #AI #FoundationModels #ModelInterpretability

283

5 Kommentare

🚨 A new paper accepted to #EMNLP2025! "The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs" We challenge how the field evaluates hallucination detection methods - and we found a lot of problems 👀 Bottom line: It's time to rethink how we evaluate hallucination detection! The Problem: Most hallucination detection methods are evaluated using ROUGE metrics 📊 But hROUGE only measures lexical overlap, not semantic understanding. It's like judging a translation by counting matching words instead of meaning! We conducted comprehensive human studies and found that ROUGE has: ✅ High recall (finds lots of potential issues) ❌ Extremely low precision (mostly false positives) This leads to seriously misleading performance estimates! When we evaluated established detection methods using human-aligned metrics (like LLM-as-Judge), performance dropped by up to 45.9%! Nearly half the supposed "progress" disappears under proper evaluation 📉 We found that simple heuristics based on response LENGTH can rival complex detection techniques! This reveals a fundamental flaw in how we've been approaching this problem. Our takeaway: We need semantically aware and robust evaluation frameworks to assess hallucination detection methods accurately. Only then can we build truly trustworthy LLM systems that users can rely on Read the full paper: https://lnkd.in/eHKnBRsi Thanks to all the people who did such a great job:  Denis Janiak, Jakub Binkowski, Albert Sawczyn, Bogdan Gabrys, Tomasz Kajdanowicz

282

12 Kommentare