Behind the Scenes: How We Built Mode's Google Sheets Integration

When we launched Google Sheets integration for Mode in September, it appeared deceptively simple: customers could now upload Google Sheets directly into Mode, use them in code-free visualizations, and combine them with data from their warehouses. However, beneath this seemingly straightforward feature lay months of careful architectural evolution, strategic planning, and operational innovation.

The success of this launch was remarkable not for what happened, but for what didn't happen. With a simple LaunchDarkly flip for Google Sheets, we released a massive architectural change to all customers - with zero incidents, bugs, or regressions.

To appreciate the complexity of the change, it is important to understand how data evolved in Mode.

The Challenge: Evolving Beyond SQL-First

Mode had always been SQL-first. Every assumption in our product - from frontend components to data models, from APIs to user workflows - was built around the idea that data came from SQL queries. Introducing Google Sheets meant more than just developing a new feature; it required us to reimagine Mode's core foundations without disrupting our existing customers' workflows.

Before Google Sheets

Users could only bring data into Mode by executing a query in our SQL IDE. The query is executed against the customer data warehouse and stored in Mode’s persistence layer. The user can then create reports and build visualizations to analyze this data by executing queries against our in-memory data engine, Helix. 

This is a simplified diagram of Mode, where you hit the data warehouse once, and reports use the results many times. The problem with this architecture was that the result was tightly coupled with a single report, and a new report had to recreate the same query.

The innovation of Reusable Datasets is that we decoupled the customer query result from a single report, and allowed it to go into multiple reports. But, the source of data was still a query.

As we researched how to introduce Google Sheets as a new data source in Mode, we were confronted with the unfortunate truth about our data model. What started as a simple model had become complex with duplicate functionality across multiple models. This meant that with each data type, we needed to reimplement many features, including data retrieval, and UI components. This is also why we could not easily share data across different product features.

After Google Sheets

By the end of the Google Sheets project, we moved our architecture to this modern dataset abstraction where the dataset and the user workflows were abstracted from the source of the data, be it queries, Google Sheets, or Notebooks.

Here's how we did it.

1. Developer-First Mindset

At Mode, we pride ourselves on our product uptime and resilience. While incidents are normal in software delivery, we have invested in fine-tuning our CI/CD, SLOs, and Observability to respond and resolve incidents quickly and in most cases, detect issues before customers. 

• Google Sheets was a highly demanded customer ask, so time to market was critical 
• We knew we wouldn’t stop at Google Sheets and support for Joins, CSV, Excel would follow
• Sticking to the existing architecture would increase delivery velocity but at the expense of heavy tech debt
• Evolving our architecture meant making changes in the critical path for customers

We chose to lean on a developer-first mindset where we empowered our developers to build right for high-quality and sustainable code: 

Make it official

• Add technical debt as a product roadmap item
• Clearly distinguish the pre-release and post-release changes
• Zero customer downtime as a Key Performance Indicator (KPI)

Align developers on the “Why”

• Align on terminologies
• What is the change
• Why it’s happening
• Is it sustainable
• What is not changing

2. Team Composition 

Mode Engineering is generally structured around microservices. There are strong contracts in which different application teams talk to each other. This change cut so deep that it touched several teams. 

Google Sheets was also disruptive because it questioned first principles of our architecture design. 

So, it was important our team composition reflect the required outcome of this feature. 

• We started with a small core engineering group with a mix of pioneers and caretakers to develop a plan which would question the status quo but balance it with an eye towards execution
• We paired engineers who had never worked together to prompt fresh perspectives
• Created openings for engineers to become first-time owners of sub-workstreams
• Provided stretch opportunities e.g. enabling application backend engineers to work on data engineering
• Fueled the passion for Refactoring, which can be more challenging than building new code 

By composing the personality of the team around the desired product personality, we could effectively match skill, interest, and temperament in reimagining our architecture and seeing the disruptive change through. This helped us achieve a user-centric feature that our developers love to build and customers love to use!

“We renewed our Mode contract. This was one of the reasons it was a no brainer!”

“A big big fan of the integration and consistently use phrases like "I love it" and "It's so easy to use"

3. Incremental Momentum

Our goal was to stay in a constant demoable state throughout the project. The advantage of doing so was two-fold - it gave the team consistent small wins and served as a faster feedback loop.

(a) Full Stack (Vertical) Slices

While there are many ways, a project can be sliced and diced, we used the full stack (vertical) slices approach. This means that we would get one workflow working E2E at a time. So, instead of working on the application backend, database, and front end as disjoint horizontal tickets, we would identify one user workflow and complete the related functionality across all the layers from the front end to the backend. 

   The benefits of building it right with full-stack (vertical) slices are:                      

• Establishes cleaner expectations on front-end and backend so that neither side is over-engineering 
• Helps identify glue code instead of last-minute surprises when we try to integrate front-end and backend
• Sometimes, a small change in the backend unlocks several front-end workflows
• Incrementally addresses tech debt changes in conjunction with the full stack slices

(b) Imperfect Solutions

While POCs and prototyping are fairly common, we employed strategic shortcuts to keep the project's momentum while production-grade services were being designed and built in parallel. These solutions, albeit imperfect were worth the cost of identifying implementation gaps without extending the timeline. Here are the two examples that we implemented to keep the team moving forward:

• Google OAuth

Imperfect Solution: Built temporary solution using Google OAuth libraries for Ruby using Redis

Win: Established E2E working flows for authentication and token management 

Production Solution: In parallel, built functionality for OAuth in production with our backend key management service in Java

• Google Sheets Ingestion

Imperfect Solution: Build interim solution using Ruby’s in-built CSV handling and Sidekiq

Win: Unblocked frontend development

Production Solution: In parallel, built an ingestion service using DuckDB

(c) Integration Touchstones

As the team enabled functionality in various stack layers, we extensively used Honeycomb Observability to track metrics and traces to assess and debug the integrations.

E.g. this cross-service trace covers the E2E flow spanning the entire stack from the frontend, application, domain, and database:

• Retrieving OAuth credentials
• Extracting the CSV from Google Sheets
• Converting to a Parquet dataset
• Updating our data models
• Storing data and metadata in our cache in Helix 

Impact and Looking Forward

The success of Mode's Google Sheets integration went far beyond the feature itself. This project transformed how we approach cross-cutting technical changes at Mode.

We shipped a massive architectural change to all customers - with zero incidents, bugs, or regressions with these principles:

1. Developer-First Mindset - Tech debt and innovation cannot be sustained without focus on internal developers
2. Team Composition - The personality of the team has a direct impact on the product personality
3. Incremental Momentum - Ship full stack slices for faster feedback

This project underscored Mode’s engineering culture to ship at a consistent pace without compromising on quality and resilience. I’m excited to see us building upon this foundation for upcoming features and innovation in Mode.

Special Thanks to Dimas Guardado for his technical leadership and the talented team of Brittany Miller Brynn Gitt John Coppola Garrett Griffith Kiran Kumar P. Anoop Narang Oliver Sanford Nidhi Mulay Samantha Ferguson Samuel Weick Christine Li who worked with vigor and rigor on delivering the Google Sheets feature for Mode and Eddie A. Tejeda for his leadership and partnership in championing the modern dataset abstraction for Mode.