Nikhar Khandelwal

Feeding spreadsheets to ChatGPT felt like jamming a square peg into a round hole. I wanted insights from my data — not token errors, mangled formats, or hallucinated summaries. At first, I tried cleaning and chunking manually. Then converting to CSV. Then JSON. Still, LLMs struggled. Either too much context lost, or too many tokens used. So I built SheetWise — a Python tool that compresses spreadsheets intelligently for LLMs. It keeps the structure, data types, and context — and cuts token use by up to 25x. Somehow, no one was doing this well. So I made it open source. Clean API. Built on modern Python. Ready to plug into your pandas workflow or CLI. SheetWise helps LLMs understand spreadsheets the way humans do — with structure, not just raw rows. If you've ever tried getting real insight from a spreadsheet with an LLM and hit a wall — this tool is for you. Github: https://lnkd.in/gK65pahY Docs: https://lnkd.in/giV7znbi Question - Ever tried summarizing a spreadsheet with AI and ended up with gibberish?

186

18 Comments

You’re in an ML Engineer interview at Google. The interviewer asks: “Your API traffic spikes 10x during peak hours. How do you scale inference infra without burning cash?” Most answers? “Add more GPUs.” ❌ Wrong. That’s how you bankrupt the company. ⸻ The real problem: • LLMs don’t scale like web servers. • Spinning up GPUs takes minutes, not milliseconds. • Idle GPUs = thousands of dollars wasted per day. ⸻ Levels of thinking: • Junior: Provision for peak load (most infra sits idle). • Senior: Autoscale GPU clusters with warm pools. • Principal: Predictive scaling using traffic analytics + quantized backup models. ⸻ Real-world techniques: • GPU warm pools: Keep a few standby GPUs hot, ready to accept jobs. • Spot instances / preemptibles: Cut costs for non-critical inference jobs. • Model quantization: Serve lighter versions during peak load to save memory. • Speculative decoding: Cut latency so fewer GPUs are needed per request. • Hybrid scaling: Mix GPU + CPU inference for lightweight requests. ⸻ Follow-up question: “How do you scale down without dropping active sessions?” ✅ Answer: Graceful draining — finish current requests, migrate caches, then release GPU. Bottom line: Autoscaling LLMs is not “just Kubernetes.” It’s about prediction, warm starts, and smart fallbacks. #llm #scaling #cloud #machinelearning #ai

182

1 Comment

You're in a Machine Learning Interview at Perplexity, and the interviewer asks: "Why do we need rerankers in RAG? Isn't semantic search enough?" Here's how you answer: Don't say: "To get better results" or "To improve accuracy." Too vague. The real answer is the two-tower bottleneck. > Your embedding model creates separate vectors for query and document. > No interaction = no understanding of relevance. You're ranking without reading. Here's why pure vector search fails: - Your bi-encoder computes similarity as dot(query_vec, doc_vec). - But "How to prevent heart attacks?" and "Heart attacks kill millions" have high cosine similarity - yet one is a question, the other a statistic. Semantic ≠ Relevant. NOTE - consider subscribing my free newsletter - https://lnkd.in/gCPD6fUz for such content daily, now back to post. The retrieval-reranking gap is brutal: - Retrieval@100 with embeddings: ~85% recall - Top-10 from those 100: ~60% precision - After reranking top-100: ~85% precision You're leaving 25% accuracy on the table by not reranking. That's the difference between a useful system and a broken one. The fundamental tradeoff everyone misses: > Bi-encoders (retrieval): Encode once, compare millions. Fast but shallow. > Cross-encoders (rerankers): Joint encoding of query+doc. Sees full interaction but O(n) inference cost. This is why you can't rerank your entire corpus. You need both. Why the two-stage pipeline exists: Stage 1 (Retrieval): 10M docs → Top 100 (1ms, $0.0001) Stage 2 (Reranking): 100 docs → Top 10 (50ms, $0.01) Skip reranking = Fast but wrong Rerank everything = Accurate but $100/query Smart pipeline = Best of both The architectural difference that matters: - Bi-encoder: encode(query) • encode(doc) - Cross-encoder: encode([query, SEP, doc]) That [SEP] token changes everything. Now your model sees: 1. Query-document word interactions 2. Actual answer containment 3. Contextual relevance Why cross-encoders are expensive: > 100 documents to rerank > Each needs full forward pass through BERT/similar > 100 × 512 tokens × 12 layers = 614K token computations > Can't precompute (query-dependent) > Must run at query time This is why rerankers are the last step, not the first. 'So how do you make reranking fast enough for production?' Interviewer leans in. This is where you mention: > Distillation (6-layer vs 12-layer cross-encoders) > Listwise reranking (batch scoring) > Hybrid approaches (ColBERT's late interaction) Speed vs quality tradeoff is everything. The answer that gets you hired: - Rerankers solve the two-tower limitation of embeddings - Cross-encoders see full query-document interaction - The cost is O(n) inference, so you only rerank top-k candidates - It's not optional for production RAG—it's the precision multiplier That's it! Drop your questions about rerankers and RAG architecture below. #chatgpt #rag #ai #ml #machinelearning #interview #job #hiring #datascience #l…

1,935

42 Comments

After +9 years working in AI, here's my advice for aspiring ML engineers: If you're just starting out, let me ask you this: "𝐃𝐨 𝐲𝐨𝐮 𝐟𝐞𝐞𝐥 𝐥𝐨𝐬𝐭?" Chances are, you said yes. And I get it. You're drowning in noise: "Try this tool" "Check this framework" "Test this LLM" You keep grinding ... but at night, you wonder: "Did I actually learn anything today? ... Or did I just collect 𝐦𝐨𝐫𝐞 𝐛𝐮𝐳𝐳𝐰𝐨𝐫𝐝𝐬?" That anxious pit in your stomach? It's called 𝐅𝐎𝐌𝐎. And LinkedIn is a FOMO factory. I've been there, and trust me, it's brutal. Mentally, even physically. But here's the 𝐜𝐮𝐫𝐞 (and it's not more tools): 𝐒𝐡𝐮𝐭 𝐨𝐮𝐭 𝐭𝐡𝐞 𝐧𝐨𝐢𝐬𝐞. Pick one thing. Maybe two. Go deep. Learn the fundamentals. 𝐁𝐮𝐢𝐥𝐝 𝐬𝐨𝐦𝐞𝐭𝐡𝐢𝐧𝐠 𝐫𝐞𝐚𝐥. You don't need 40 frameworks ... you need focus. 𝐒𝐨 𝐡𝐞𝐫𝐞'𝐬 𝐦𝐲 𝐚𝐝𝐯𝐢𝐜𝐞: ✨ Come up with a project idea 📵 Close LinkedIn 💻 Start building It'll do more for your growth than any post, course, or thread. And the results will blow your mind. 🫶

235

45 Comments

Training gets all the attention. But inference is where your LLM either flies or dies. LLM inference is the process of using a trained language model to generate responses or predictions based on prompts. Essentially, training builds the model, while inference applies it to generate real-time outputs from new inputs. Here's how it happens: 1️⃣ 𝗧𝗼𝗸𝗲𝗻𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Your prompt gets broken down into tokens 2️⃣ 𝗣𝗿𝗲𝗳𝗶𝗹𝗹 𝘀𝘁𝗮𝗴𝗲: The model processes your input tokens 3️⃣ 𝗗𝗲𝗰𝗼𝗱𝗲 𝘀𝘁𝗮𝗴𝗲: The model generates new tokens one by one But where does all this computation actually happen? 𝗜𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲 𝗢𝗽𝘁𝗶𝗼𝗻𝘀: • CPUs: Good for smaller models, cost-effective but slower • GPUs: The gold standard for most LLM inference workloads   • TPUs: Optimized for specific model architectures 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 𝗖𝗵𝗼𝗶𝗰𝗲𝘀: You've got two main paths - serverless APIs (like OpenAI) or self-hosted deployments. Tools like Ollama make local inference super accessible, letting you download quantized models and run them on your own hardware. Why does this matter? Inference directly impacts three critical factors: • 𝗟𝗮𝘁𝗲𝗻𝗰𝘆: How fast users get responses • 𝗖𝗼𝘀𝘁: Especially important at scale • 𝗨𝘀𝗲𝗿 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲: The difference between a snappy chat and a frustrating wait While training is about learning patterns from massive datasets, inference is about generating responses as efficiently as possible - often with techniques like KV caching and various quantization methods. BentoML has put together an amazing LLM Inference Handbook covering everything from basics to advanced optimization strategies. Highly recommend checking it out. 📖 🔗 LLM Inference Handbook: https://lnkd.in/gxfDA6nG

224

9 Comments

👩🏽‍💻 Just read a fascinating paper out of CMU: Agentic-R1: Distilled Dual-Strategy Reasoning 🔗 https://lnkd.in/gif6sj7h 🔹 The paper introduces DualDistill, a training framework that merges two distinct reasoning paradigms into a single student model: one teacher specializes in tool-augmented, code-based reasoning , while the other focuses on natural language, long chain-of-thought reasoning. 🔹 The resulting model, Agentic-R1, learns to dynamically switch between these strategies, deciding when to reason abstractly and when to invoke computational tools, all via supervised fine-tuning. 🔹 This is achieved through trajectory composition, where the outputs of both teachers are evaluated and strategically combined with handcrafted transition signals (e.g., “Wait, let’s try code reasoning instead”). 🔹 An additional self-distillation phase helps the student refine its behavior by generating solutions and receiving targeted corrections or validations from the teachers. This ensures its strategy choices reflect its actual strengths, especially important when tool usage lags behind expert agents. 📊 Results: Agentic-R1 outperforms both tool-only and text-only models on DeepMath-L and Combinatorics300, and performs competitively on standard benchmarks like AIME and AMC. Even with just 2.6k training examples, the model effectively learns when and how to apply each reasoning strategy. Read this and other paper summaries that Aman Chadha and I collaborate on at vinija.ai!

67

🚨 Transformers: The Beating Heart of AI — A Must-Know for Interviews! Transformers are the backbone of today’s AI revolution—powering models like GPT, BERT, LLaMA, and beyond. Whether you’re preparing for roles in Data Science, Machine Learning, or Generative AI, a deep understanding of the Transformer architecture is non-negotiable. When I started diving into AI interviews, I realized just knowing about RNNs or CNNs wasn’t enough—interviewers wanted to see clarity on attention mechanisms, positional encoding, encoder-decoder flow, and scalability of Transformers. That’s why I created a detailed set of handwritten notes to simplify and master these concepts. Here’s what you’ll find inside my notes: ✔️ The “Attention Is All You Need” paper broken down simply ✔️ Step-by-step working of Self-Attention & Multi-Head Attention ✔️ Clear explanation of Encoder, Decoder, and Positional Encoding ✔️ Insights into why Transformers outperform RNNs/LSTMs ✔️ Interview-ready breakdowns that connect theory with real-world applications 💡 Pro Tip: Don’t just say “Transformers use attention”—explain how attention changes computation, scales with data, and impacts performance. That’s what makes you stand out. 🚨 Remember: AI interviews aren’t about buzzwords—they’re about depth of understanding. 👉 Found it helpful? ♻️ Save & Repost!

276

51 Comments

As a Senior ML Scientist, there’s one industry secret I want to tell you. It’s changing the data science field drastically, But no one is talking about it. Here it is: AI is now doing 70–80% of a data scientist’s work. When I realized this, genuinely, deeply it shook me. Because for a while, that was my job. That was my identity. But here’s what separates people who will thrive from those who’ll quietly get automated: → They’re not clinging to the old way of doing data science. → They’re stepping into the one thing AI can’t do: Strategic thinking. Let me explain. In this new world, your value isn’t in what you build, it’s in why you build it. Can you: ✅ Frame the right questions? ✅ Align your work with real business outcomes? ✅ Influence decisions - not just show dashboards? This is what makes you irreplaceable. So if you're a working professional thinking of switching into data science, don’t make the mistake of just learning Python and SQL. That’s the baseline now. And luckily, we have platforms like Bosscoder Academy, who will help you from basics to advanced level of data science skills. Check them here - http://bit.ly/4mdsR1h They have helped 2200+ working professionals through structured curriculum, personal mentorship from professionals, real business case studies, placement support and much more. It’s time to make AI your best-friend. #DataScience #AI #career #sponsoredpost

269

16 Comments