Learn to build reliable data agents with Snowflake's new course

As the CEO of www.jaiinfoway.com, I believe building trustworthy AI systems starts with visibility and evaluation. Andrew Ng’s new course on “Building and Evaluating Data Agents” highlights exactly what modern AI development needs — transparency, reliability, and measurable performance. At www.jaiinfoway.com we design intelligent agents using structured frameworks that ensure accuracy, traceability, and adaptive improvement across every workflow. Because in AI, confidence means nothing without correctness. #Jaiinfoway #AI #DataAgents #ArtificialIntelligence #AgenticAI #AIInnovation #MachineLearning #DigitalTransformation #FutureOfWork #AIEthics

💡 Ever wondered why we often see ".toList()" When fetching data? When you fetch data from a database or an API (like SQLite or JSON), the result often comes as a List of Maps (for example, List<Map<String, dynamic>> or List<dynamic>) __ raw data that you need to convert into your model Objects. ✨ For example, if you have a model called Challenge, and you're converting that raw data like this: final challenges = maps.map((e) => Challenge.fromMap(e)); At first glance, this looks like a list, but it’s not a List! The .map() method gives you an Iterable<Challenge>, not a List<Challenge>. 🔸Now you might ask what is "Iterable" ? Iterable is a lazy collection , it doesn’t create all elements right away. It just promises to produce them when you actually need them (for example, in a for() loop or .forEach()). 🔸If you want a real list you need to end it with .toList() final challengeList = challenges.toList(); ✅ And now you have a fully usable List<Challenge>! 🔸Why we call .toList()? - you can access items by index, update them easily - skip re-computing the same elements every time ✨ So next time you see .toList(), remember it's transforming your promised data into a concrete, usable collection!

📊 Being data-driven isn’t just about collecting good data, it’s about ensuring everyone can see and trust its quality. Otherwise, we all know what happens: garbage in → garbage out 🗑️ . A while ago, I wrote about the importance of Data quality and Testing, outlining a phased approach to implementing them effectively. Today, I wanted to share a short article where I explain how to build a Data Quality Score using DBT + Elementary Data and answer key questions like: ➡️ How trustworthy are our tables? ➡️ Where are issues emerging? ➡️ Are we improving over time? Just like we monitor availability or latency, we should also be monitoring data health. Because data only creates value if we can trust it. https://lnkd.in/dPEiE-hv

Analyzing data doesn't require excessive complexity. Simple tools can deliver powerful insights without needing complex frameworks or libraries. This approach allows for quick backtesting using readily available data, such as a daily CSV file. Results reveal key performance indicators like maximum drawdown (-8%) and total return (-2.85%), offering a clear view of performance metrics. #DataAnalysis #Backtesting #InvestmentStrategy #PerformanceMetrics

Analyzing data doesn't require excessive complexity. Simple tools can deliver powerful insights without needing complex frameworks or libraries. This approach allows for quick backtesting using readily available data, such as a daily CSV file. Results reveal key performance indicators like maximum drawdown (-8%) and total return (-2.85%), offering a clear view of performance metrics. #DataAnalysis #Backtesting #InvestmentStrategy #PerformanceMetrics

Most people use ANY_VALUE() to make a query stop complaining. They get hit by a GROUP BY error, throw in to fix ANY_VALUE(), and move on . It works — but usually for the wrong reasons. You’re not fixing the logic, you’re muting it and assuming that any value is fine when it might result in the wrong status. That said… used intentionally, ANY_VALUE() can be brilliant. If you know the data’s at the right grain (e.g. you've left joined your dim_customers to their orders) and you just want the optimiser to relax, it’s clean, efficient, and faster than wrapping everything in pointless aggregates. So: Lazy when it hides ambiguity. Elegant when it avoids redundancy. Like most SQL features, it’s not good or bad. it just reveals whether you knew what you were doing.

Recently I got questioned about how to get data from a new data source; first we need to understand the need, if we need to ingest that data into our data lake or we need just 1 call of a dataset to look at the data and understand its value. Very often we need to put ourselves on our customers/business shoes so we can provide the best solution to what they need. Is not just about what I want, or what the architecture team told me to do, is also about what makes sense, what is faster yet secure, and what solves the need and allow me to have also a good template I can use later to provide similar solutions to someone else, without sacrificing speed, accuracy and agility ✅ . In this particular case we needed to look into 2 alternatives in particular, MCP vs API, are we using an LLM call or just retrieving data? 🚀 , do we need something we can communicate in 2 ways or just a simple send-acknowledge-response-aknowledge-receive solution is enough?. Here I left you a small but comprehensive table if you need to take a decision like this, I hope you like it or find it useful. 👍

A fun perk of having an incremental query engine: you can structure your data to be optimized for both reads and writes at the same time. Instead of constantly tweaking your write path to optimize for reads—pre-aggregating data, merging multiple tables into one to avoid expensive JOINs, and so on—you can simply write data in the most efficient way for ingestion, transform and aggregate it incrementally, and consume that transformed data as the most optimal format for reads. Your event-collecting micro-service doesn’t need to know—or care—how other services slice, join, or aggregate that data downstream. In other words, you can have the most optimal read path AND the most optimal write path both at the same time — having the incremental engine act as some sort of "bridge" between these two optimal layers.

“Can I just get the data?” This question sounds reasonable. It’s actually expensive. Because what’s really needed isn’t the spreadsheet. It’s why you collected it, what you learned along the way, the three things you tried that failed first. But we’ve built systems that treat data like it’s self-explanatory. So someone gets the CSV, misinterprets it, makes a decision, and six months later everyone’s wondering why it didn’t work. The request feels efficient. The outcome costs more than starting over. https://lnkd.in/d9iS7_Yx

What happens when you need to do sector categorization but there’s no sector column in your financial dataset? I explored this problem using SQL, discussed query design tradeoffs, and how analytics integrity can be preserved. Read👉🏿 https://lnkd.in/dWZPjSSv