Phil Goerdt

Snowflake is introducing *snapshots* as long-term immutable backups of tables, schema, or databases. You can create snapshot policies that define the snapshot life cycle, e.g. to automatically add new snapshots to a snapshot set, set schedules and expiration dates, or define retention locks and legal holds to address compliance requirements. https://lnkd.in/ekh9XwzP https://lnkd.in/ejj5F8k9

724

22 Comments

When data agents fail, they often fail silently - giving confident-sounding answers that are wrong, and it can be hard to figure out what caused the failure. "Building and Evaluating Data Agents" is a new short course created with Snowflake and taught by Anupam Datta and Josh Reini. This course teaches you to build data agents with comprehensive evaluation built in. Skills you'll gain: - Build reliable LLM data agents using the Goal-Plan-Action framework and runtime evaluations that catch failures mid-execution - Use OpenTelemetry tracing and evaluation infrastructure to diagnose exactly where agents fail and systematically improve performance - Orchestrate multi-step workflows across web search, SQL, and document retrieval in LangGraph-based agents The result: visibility into every step of your agent's reasoning, so if something breaks, you have a systematic approach to fix it. Sign up to get started: https://lnkd.in/gVwKuCCQ

6,637

223 Comments

We tested Snowflake's Gen2 warehouses on 2 billion query profiles across 14K warehouses here at Capital One. Here is where we saw improvement: 🍎 Approximately 30% of queries fell into categories that could benefit from Gen2—specifically those involving DELETE, UPDATE or MERGE. 🍎 Long-running DML operations: A significant subset of queries running longer than 30 minutes were performing complex DML tasks. These are prime candidates for Gen2 migration. 🍎 Large scans: Numerous queries scanned over 1 TB of data, where Gen 2’s improved scan performance can provide material runtime reduction. 🍎 Combined heavy patterns: A smaller but critical set of queries both scanned over 1 TB and ran for 30 minutes or more, often involving DML operations—these represent the highest potential gain from Gen2. These findings support a targeted migration strategy: focus on warehouses with a significant concentration of eligible workloads rather than migrating all warehouses indiscriminately. Check out the full blow written by our resident expert and all around awesome guy - Ganesh Bharathan https://lnkd.in/g8XufZaV

292

14 Comments

Real-time streaming in Databricks with Apache Spark’s new Real-Time trigger! How real is real-time? The chart below gives a good indication - it's super fast. I am excited about this both launching into preview in Databricks [1] and also the SPIP to (ideally) introduce it into Apache Spark 4.1.0 [2]. Something for everyone! 1: https://lnkd.in/gXStGRk9 2: https://lnkd.in/gPM2TZD9

254

5 Comments