Philippe de Cuzey

Exciting news for the open data community: Variant, the native data type for semi-structured data, is now ratified in the Apache Parquet™ community — with support across Delta Lake, Apache Iceberg™, and Apache Spark™. Variant brings a unified, open standard to how the lakehouse stores and queries flexible data—making it faster, simpler, and more consistent across formats and engines. https://lnkd.in/gESP97Nm

1 127

23 commentaires

Monitoring streaming data pipelines can be challenging due to their asynchronous nature. Datadog Data Streams Monitoring gives you an end-to-end view across data streaming technologies like Kafka by automatically mapping interdependencies and surfacing key health metrics. ✅ Detect slowdowns such as consumer lag ✅ Pinpoint bottlenecks and failures ✅ Remediate floods of backed-up messages Learn how unified visibility helps teams avoid downtime and build resilient event-driven applications ➡️ https://bit.ly/47MbFfj

174

2 commentaires

The Fivetran Connector SDK is now available! 🚀 Build custom, Python-based connectors for any data source — internal tools, niche systems, or proprietary APIs — without managing infrastructure. You write the logic and Fivetran handles orchestration, scaling, error handling, and delivery. Accelerate analytics, eliminate brittle pipelines, and bring every source into your stack. Learn more: https://5tran.co/3Fiuona

74

Let's build a DeepSeek-R1 chatbot in Snowflake! The DeepSeek-R1 model is making waves with its powerful reasoning capabilities and low training costs. In this tutorial, we’ll show you how to build an LLM-powered chatbot inside Snowflake using Streamlit—all in under 100 lines of code! 💡 What you’ll learn: ✅ How to integrate DeepSeek-R1 into Snowflake ✅ Using Streamlit to create an interactive chatbot UI ✅ Implementing real-time reasoning & chat history management 🔗 Watch the full tutorial now: https://lnkd.in/gMGiRnDE

108

1 commentaire

Discover, Compose, Deploy, and Evaluate – these are key phases of multi-agent software development. AGNTCY Collective is pleased to announce the release of the observability data schema and SDK, foundational components of the Evaluate phase. This release delivers upon the best practices for multi-agent software observability and evaluation. Read our blog to gain insights on: 🔹 Why observability and evaluation matter in multi-agent software development 🔹 The architecture and critical metrics required 🔹 How to enhance transparency and ensure AI agent performance in enterprise environments ➡️ Read our blog and join the discussion: http://cs.co/90044lBIj #InternetOfAgents #AGNTCY #AIagents #MultiAgent

51

Struggling with Airflow's complexity? Our latest comparison shows how Dagster modernizes data orchestration: 🎯 Assets over DAGs - organize work naturally without artificial boundaries 🐍 Pythonic development - write clean code without cryptic operators 🛠️ Intuitive dependencies - automatically track relationships between assets ⚙️ Faster setup - spin up a full environment with a single command See how we rebuilt Airflow's tutorial in Dagster with cleaner code and less boilerplate. Read the full comparison in the comments #DataEngineering #Dagster #Airflow #DataPipelines #DevExperience

26

1 commentaire

Our colleague Fabrice Deseyn will be speaking at Dataminds Connect on October 8th, 2025 at 14:50. 📢 Session: Securing Data at Scale on Databricks: Beyond the Limits of Secret Scopes 🔐 Secret Scopes are often the go-to for handling sensitive values in Databricks. But when organizations scale, these mechanisms quickly show their limits. Fabrice will dive into: ✔️ Why Secret Scopes fall short for credential storage & encryption key management ✔️ How these limitations impact data access, auditability, lifecycle management & key segregation ✔️ Real-world use cases to highlight enterprise challenges ✔️ What a scalable, production-grade secrets management approach should look like Don’t miss this session if you want to learn how to secure your Databricks environment beyond the basics! 👉 Read more: https://lnkd.in/gPwyd_j #ShapingDigitalTogether #AI #DataMindsConnect #Databricks

45

You don't have to choose between stream and batch processing. Unify them under a single framework with our 🆕 Flink Snapshot Queries and start: ✅ Querying historical and real-time data ✅ Tapping into long-retention Iceberg and Delta tables ✅ Accelerating pipeline development and debugging Learn all about this feature on our blog: https://cnfl.io/3H1wo3A

56

4 commentaires

Smarter Insights, Faster Decisions Snowflake Semantic Views + Cortex Analyst open the door to a new way of working with data — where teams can query in plain language, trust the answers, and access governed, real-time insights all within Snowflake. In our latest #TechnicalTuesday blog, Daksh Trehan explores how this approach standardizes KPIs, streamlines analytics, and empowers everyone to go from question to insight in seconds. 📖 Read here: https://lnkd.in/dTXAz_jS #WNSKipiAI #SemanticModeling #Snowflake #TechnicalTuesday

25

📝 Best practices for using dbt and Apache Airflow! In the final video of this YouTube series, Marc Lamberti walks through best practices for orchestrating your dbt models from Airflow using Astronomer Cosmos. https://lnkd.in/gtku-Nuw

133

3 commentaires

With the gander package, RStudio users can now chat with LLMs, including Databricks models, directly in their IDE. Here’s how it works: • You install gander and ellmer • OAuth handles the Databricks connection securely via Posit Workbench • You open a built-in chat pane in RStudio • Ask a question in plain English and get R code back instantly • Iterate and refine with no copying or switching apps It’s a fast, secure, low-friction way to experiment, debug, and explore data with the help of powerful foundation models, all inside the IDE you already use. See a full tutorial on the Posit blog: https://lnkd.in/gkfSQQ_a

163

3 commentaires

The all-new dbt Fusion engine is now available in public beta. Fusion is a major upgrade to dbt: it’s far faster, more cost-efficient, and more intelligent: • Parse times up to 30x faster ⚡️ • A hyper-responsive developer experience, with SQL comprehension for real-time code validation (without hitting the warehouse) 🧠 • Built-in cost savings by avoiding wasteful rebuilds, saving ~10% in warehouse costs 💰 Fusion is built for teams that need to move fast, eliminate waste, and build a trusted foundation for AI-powered workflows. Learn more: https://lnkd.in/gsrAGbki

396

6 commentaires