SlideShare a Scribd company logo
?Google Cloud
Data Platform
GoDataFest Workshop
28-10-2022
Agenda
1. Intro GCP for Data
09:00 - 09:30
2. Roles & tools (per role)
09:35 - 10:10
3. Build on GCP (workgroups)
10:30 - 12:30
● Data Democratization
● Why (Google) Cloud?
● Data platforms
● Data Engineer
● Analytics Engineer
● Analyst
● clean & prep data
● build the model
● create & share insights
Introductions
Thomas van Latum - thomasvanlatum@godatadriven.com
Who are we?
Bas Leenders - bas@gcompany.nl
data analytics & BI
prescriptive
predictive
descriptive
diagnostic
1
2 3
4
Workshop on Google Cloud Data Platform
Workshop on Google Cloud Data Platform
Infrastructure
Big Data and
Machine Learning
Application
Development
G Suite
For the past 15 years, Google
has been building out the fastest,
most powerful, highest quality
cloud infrastructure on the planet. Images by Connie
Zhou
Google Global Cache
(GGC) edge nodes
Points of presence (>100)
Network fiber
FASTER (US, JP, TW) 2016
Unity (US, JP) 2010
SJC (JP, HK, SG) 2013
Monet (US, BR) 2017
Google network
More than a collection of data centers
research.google.com/pubs/papers.html
Google has been innovating data technologies
2002 2004 2006 2008 2010 2012 2014 2016
GFS
MapReduce TensorFlow
Bigtable
Dremel
Colossus
Flume
Megastore
Spanner
Millwheel
Pub/Sub
F1
Google needed to invent data processing methods
2002 2004 2006 2008 2010 2012 2014
Google has been innovating data
technologies
2016
Cloud Storage
Dataproc ML Engine
Bigtable
BigQuery
Cloud Storage
Dataflow
Datastore
Dataflow
Pub/Sub
Google then shared it’s innovations
research.google.com/pubs/papers.html
Auto ML
2018
process & analyze
transform to information
● meaningful
● usable
ingest
read raw data
● streaming
● batch
● ad-hoc
store
store in the right format
● durable
● accessible
explore & visualize
convert to insights
● insightful
● shareable
data lifecycle – 4 steps
@pvergadia #GCPSketchnote
the modern data platform
data life cycle with BigQuery & Looker
raw data clean sources business logic
data engineer analytics engineer explorer
BigQuery / dataform
ERP
source systems
Finance
HR
Marketing
Other
data platform
reports
viewer
Looker
● storage & compute
● extremely fast, very cost-efficient
● use (standard) SQL
● integrate
○ Cloud SQL
○ Data Studio
○ Connected Sheets
● BQ ML
○ machine learning “for business”
○ SQL-powered
○ brings ML to the data
modern data warehouse
BigQuery
What is BigQuery?
Big(!) Data with Big Query
more info→ !
Dataform & BigQuery
● Open source, SQL-based language to manage data transformations
● Fully managed, serverless orchestration for data pipelines
● Fully featured cloud development environment to develop with SQL
Looker
Looker Data Platform
● modern data technology
● modern problems
databases then databases now
maximize efficiency
compensate for inefficiency
database technology has changed
Bottleneck Chaos
and/or
NEXT!
analyst
bottleneck chaos
two problems Looker solves
Data Lake
Data Storage
Best practice for companies
to centralise their data
Data Extraction
Data Analysts extracting your
data into workbooks or
aggregated cubes
HARD TO MAINTAIN
HARD TO SCALE
→ DATA CHAOS
Data Visualisation
BI tool sits on top of these
siloed workbooks to present
dashboards and reports
LIMITED DATA
MULTIPLE TRUTHS
→ DATA BOTTLENECK
Tech team
headcount
legacy BI “workbook” architecture
Looker’s universal semantic model
governed metrics best-in-class APIs in-database
Git version control security Cloud
integrated insights
modern BI & analytics data-driven workflows custom applications
SQL in results back
Agenda
1. Intro GCP for Data
09:00 - 09:30
2. Roles & tools (per role)
09:45 - 10:30
3. Build on GCP (workgroups)
10:45 - 12:30
● Data Democratization
● Why (Google) Cloud?
● Data platforms
● Data Engineer
● Analytics Engineer
● Analyst
● clean & prep data
● build the model
● create & share insights
Looker User
• Looker Explorer
• Looker Dashboarding
Looker Dev
• (BigQuery & Dataform)
• Looker & LookML
Data Engineer
• BigQuery
• Dataform
• Terraform
Tools by role
Pick your preferred role → https://leend.rs/GDF-pick (with your Google-account!)
Analytics Engineer
Data Analyst
Cloud Data Engineer
Looker User
• Looker Explorer
• Looker Dashboarding
Looker Dev
• (BigQuery & Dataform)
• Looker & LookML
Data Engineer
• BigQuery
• Dataform
• Terraform
Tools by role
Pick your preferred role → https://leend.rs/GDF-pick (with your Google-account!)
Data Analyst
Analytics Translator
Machine Learning Engineer
Data Architect
Data Scientist
Business Analyst
Analytics Engineer
Cloud Data Engineer
Tools by role
Looker User
• Looker Explorer
• Looker Dashboarding
Looker Data Explorer - Qwik Start
→ https://leend.rs/GDF-QL-An1
Filtering and Sorting Data in Looker
→ https://leend.rs/GDF-QL-An2
Data Engineer
• BigQuery
• Dataform
• Terraform
console.cloud.google.com/ ...
?project=go-data-fest
→ https://leend.rs/GDF-project
Looker Dev
• (BigQuery & Dataform)
• Looker & LookML
Looker Developer - Qwik Start
→ https://leend.rs/GDF-QL-AE1
Creating Measures and Dimensions
Using LookML
→ https://leend.rs/GDF-QL-AE2
Agenda
1. Intro GCP for Data
2. Roles & tools
(per role)
3. Build on GCP
(mixed workgroups)
● Data Democratization
● Why (Google) Cloud?
● Data platforms
● Data Engineer
● Analytics Engineer
● Analyst
● clean & prep data
● build the model
● create & share insights
Groups & roles
30
Team Google-user Data Engineer Looker Dev Looker User
Arthur erik.clabbers@... 1
fcm073@... 1
haydnruthams@... 1
thomas.hantke@... 1
Ford caiofabiomc@... 1
chung.kally@... 1
debbysmit@... 1
mfharms6@... 1
spstrempel@... 1
Trillian christovvillamon@... 1
e.j.m.hamberg@... 1
saheli.de@... 1
vhverhagen@... 1
1. Build the dataset
2. Build the Model, Explore & Views
3. Create and Share Dashboards
→ leend.rs/GDF-project
→ gcompany.eu.looker.com
Now let’s have some fun!
Insights needed
● What products show yearly
seasonal (sales) trends?
● How are stocks in the distribution
centers doing?
●
● How are order prices related to
product list prices
● Is there a relationship between a
product’s events & sales

More Related Content

What's hot (20)

PPTX
Cohort Analysis at Scale
Blake Irvine
 
PPTX
chatgpt dalle.pptx
Ellen Edmands
 
PDF
Observability at Spotify
Aleksandr Kuboskin, CFA
 
PDF
What you need to know about Generative AI and Data Management?
Denodo
 
PDF
Big Data At Spotify
Adam Kawa
 
PDF
Building an ML Platform with Ray and MLflow
Databricks
 
PDF
Data at Spotify
Danielle Jabin
 
PDF
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
PDF
Managing the Complete Machine Learning Lifecycle with MLflow
Databricks
 
PDF
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
PDF
Tableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
Blake Irvine
 
PPTX
Collaborative Filtering at Spotify
Erik Bernhardsson
 
PDF
Recent Trends in Personalization: A Netflix Perspective
Justin Basilico
 
PDF
Making Netflix Machine Learning Algorithms Reliable
Justin Basilico
 
PPTX
Data Lake Overview
James Serra
 
PPTX
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
Carol Smith
 
PPTX
Google Analytics / Adwords Digital Marketing Presentation
Katelyn Duckworth
 
PDF
Introduction to MLflow
Databricks
 
PDF
[Cloud OnAir] お客様事例紹介 -リクルートライフスタイルにおける デジタルトランスフォーメーションとクラウド活用- 2018年7月12日 放送
Google Cloud Platform - Japan
 
PPTX
Future of Data and AI in Retail - NRF 2023
Rob Saker
 
Cohort Analysis at Scale
Blake Irvine
 
chatgpt dalle.pptx
Ellen Edmands
 
Observability at Spotify
Aleksandr Kuboskin, CFA
 
What you need to know about Generative AI and Data Management?
Denodo
 
Big Data At Spotify
Adam Kawa
 
Building an ML Platform with Ray and MLflow
Databricks
 
Data at Spotify
Danielle Jabin
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Managing the Complete Machine Learning Lifecycle with MLflow
Databricks
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
Tableau Conference 2018: Binging on Data - Enabling Analytics at Netflix
Blake Irvine
 
Collaborative Filtering at Spotify
Erik Bernhardsson
 
Recent Trends in Personalization: A Netflix Perspective
Justin Basilico
 
Making Netflix Machine Learning Algorithms Reliable
Justin Basilico
 
Data Lake Overview
James Serra
 
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
Carol Smith
 
Google Analytics / Adwords Digital Marketing Presentation
Katelyn Duckworth
 
Introduction to MLflow
Databricks
 
[Cloud OnAir] お客様事例紹介 -リクルートライフスタイルにおける デジタルトランスフォーメーションとクラウド活用- 2018年7月12日 放送
Google Cloud Platform - Japan
 
Future of Data and AI in Retail - NRF 2023
Rob Saker
 

Similar to Workshop on Google Cloud Data Platform (20)

PDF
Data Platform on GCP
Patrick Alexander
 
PPTX
Eric Andersen Keynote
Data Con LA
 
PDF
Supercharge your data analytics with BigQuery
Márton Kodok
 
PDF
Executive Intro to BigQuery
William M. Cohee
 
PDF
Hybrid data lake on google cloud with alluxio and dataproc
Alluxio, Inc.
 
PDF
Workflow Engines + Luigi
Vladislav Supalov
 
PDF
Big data in action
Tu Pham
 
PDF
Critical Breakthroughs and Challenges in Big Data and Analytics
Data Driven Innovation
 
PDF
Integrating Google Cloud Dataproc with Alluxio for faster performance in the ...
Alluxio, Inc.
 
PDF
Deploying a Modern Data Stack by Lasse Benninga - GoDataFest 2022
GoDataDriven
 
PDF
Bogdan botea, dmitry nefedkin no fiddle, efficient development on the googl...
Codecamp Romania
 
PPTX
いそがしいひとのための Microsoft Ignite 2018 最新情報 Data 編
Miho Yamamoto
 
PDF
BigQuery for Beginners
Better&Stronger
 
PDF
VoxxedDays Bucharest 2017 - Powering interactive data analysis with Google Bi...
Márton Kodok
 
PDF
[Cloud OnAir] Talks by DevRel Vol.4 データ管理とデータ ベース 2020年8月27日 放送
Google Cloud Platform - Japan
 
PDF
IoT NY - Google Cloud Services for IoT
James Chittenden
 
PDF
Cloud Developer Days - BigQuery
Wlodek Bielski
 
PDF
CodeCamp Iasi - Creating serverless data analytics system on GCP using BigQuery
Márton Kodok
 
PPTX
With Automated ML, is Everyone an ML Engineer?
Dan Sullivan, Ph.D.
 
PDF
Introduction to Google Cloud Platform
Pradeep Bhadani
 
Data Platform on GCP
Patrick Alexander
 
Eric Andersen Keynote
Data Con LA
 
Supercharge your data analytics with BigQuery
Márton Kodok
 
Executive Intro to BigQuery
William M. Cohee
 
Hybrid data lake on google cloud with alluxio and dataproc
Alluxio, Inc.
 
Workflow Engines + Luigi
Vladislav Supalov
 
Big data in action
Tu Pham
 
Critical Breakthroughs and Challenges in Big Data and Analytics
Data Driven Innovation
 
Integrating Google Cloud Dataproc with Alluxio for faster performance in the ...
Alluxio, Inc.
 
Deploying a Modern Data Stack by Lasse Benninga - GoDataFest 2022
GoDataDriven
 
Bogdan botea, dmitry nefedkin no fiddle, efficient development on the googl...
Codecamp Romania
 
いそがしいひとのための Microsoft Ignite 2018 最新情報 Data 編
Miho Yamamoto
 
BigQuery for Beginners
Better&Stronger
 
VoxxedDays Bucharest 2017 - Powering interactive data analysis with Google Bi...
Márton Kodok
 
[Cloud OnAir] Talks by DevRel Vol.4 データ管理とデータ ベース 2020年8月27日 放送
Google Cloud Platform - Japan
 
IoT NY - Google Cloud Services for IoT
James Chittenden
 
Cloud Developer Days - BigQuery
Wlodek Bielski
 
CodeCamp Iasi - Creating serverless data analytics system on GCP using BigQuery
Márton Kodok
 
With Automated ML, is Everyone an ML Engineer?
Dan Sullivan, Ph.D.
 
Introduction to Google Cloud Platform
Pradeep Bhadani
 
Ad

More from GoDataDriven (20)

PDF
Streamlining Data Science Workflows with a Feature Catalog
GoDataDriven
 
PDF
Visualizing Big Data in a Small Screen
GoDataDriven
 
PDF
Building a Scalable and reliable open source ML Platform with MLFlow
GoDataDriven
 
PDF
Training Taster: Leading the way to become a data-driven organization
GoDataDriven
 
PDF
My Path From Data Engineer to Analytics Engineer
GoDataDriven
 
PDF
dbt Python models - GoDataFest by Guillermo Sanchez
GoDataDriven
 
PDF
How to create a Devcontainer for your Python project
GoDataDriven
 
PDF
Using Graph Neural Networks To Embrace The Dependency In Your Data by Usman Z...
GoDataDriven
 
PDF
Common Issues With Time Series by Vadim Nelidov - GoDataFest 2022
GoDataDriven
 
PDF
MLOps CodeBreakfast on AWS - GoDataFest 2022
GoDataDriven
 
PDF
MLOps CodeBreakfast on Azure - GoDataFest 2022
GoDataDriven
 
PDF
Tableau vs. Power BI by Juan Manuel Perafan - GoDataFest 2022
GoDataDriven
 
PPTX
AWS Well-Architected Webinar Security - Ben de Haan
GoDataDriven
 
PDF
The 7 Habits of Effective Data Driven Companies
GoDataDriven
 
PPTX
DevOps for Data Science on Azure - Marcel de Vries (Xpirit) and Niels Zeilema...
GoDataDriven
 
PDF
Artificial intelligence in actions: delivering a new experience to Formula 1 ...
GoDataDriven
 
PDF
Smart application on Azure at Vattenfall - Rens Weijers & Peter van 't Hof
GoDataDriven
 
PDF
Democratizing AI/ML with GCP - Abishay Rao (Google) at GoDataFest 2019
GoDataDriven
 
PDF
The world runs on AI - Tony Krijnen (Microsoft) at GoDataFest 2019
GoDataDriven
 
PDF
CI/CD with Azure DevOps and Azure Databricks
GoDataDriven
 
Streamlining Data Science Workflows with a Feature Catalog
GoDataDriven
 
Visualizing Big Data in a Small Screen
GoDataDriven
 
Building a Scalable and reliable open source ML Platform with MLFlow
GoDataDriven
 
Training Taster: Leading the way to become a data-driven organization
GoDataDriven
 
My Path From Data Engineer to Analytics Engineer
GoDataDriven
 
dbt Python models - GoDataFest by Guillermo Sanchez
GoDataDriven
 
How to create a Devcontainer for your Python project
GoDataDriven
 
Using Graph Neural Networks To Embrace The Dependency In Your Data by Usman Z...
GoDataDriven
 
Common Issues With Time Series by Vadim Nelidov - GoDataFest 2022
GoDataDriven
 
MLOps CodeBreakfast on AWS - GoDataFest 2022
GoDataDriven
 
MLOps CodeBreakfast on Azure - GoDataFest 2022
GoDataDriven
 
Tableau vs. Power BI by Juan Manuel Perafan - GoDataFest 2022
GoDataDriven
 
AWS Well-Architected Webinar Security - Ben de Haan
GoDataDriven
 
The 7 Habits of Effective Data Driven Companies
GoDataDriven
 
DevOps for Data Science on Azure - Marcel de Vries (Xpirit) and Niels Zeilema...
GoDataDriven
 
Artificial intelligence in actions: delivering a new experience to Formula 1 ...
GoDataDriven
 
Smart application on Azure at Vattenfall - Rens Weijers & Peter van 't Hof
GoDataDriven
 
Democratizing AI/ML with GCP - Abishay Rao (Google) at GoDataFest 2019
GoDataDriven
 
The world runs on AI - Tony Krijnen (Microsoft) at GoDataFest 2019
GoDataDriven
 
CI/CD with Azure DevOps and Azure Databricks
GoDataDriven
 
Ad

Recently uploaded (20)

PPT
tuberculosiship-2106031cyyfuftufufufivifviviv
AkshaiRam
 
PDF
Product Management in HealthTech (Case Studies from SnappDoctor)
Hamed Shams
 
PPTX
SlideEgg_501298-Agentic AI.pptx agentic ai
530BYManoj
 
PDF
OOPs with Java_unit2.pdf. sarthak bookkk
Sarthak964187
 
PPTX
apidays Munich 2025 - Building an AWS Serverless Application with Terraform, ...
apidays
 
PDF
apidays Singapore 2025 - Surviving an interconnected world with API governanc...
apidays
 
PPTX
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
PDF
Avatar for apidays apidays PRO June 07, 2025 0 5 apidays Helsinki & North 2...
apidays
 
PDF
Simplifying Document Processing with Docling for AI Applications.pdf
Tamanna36
 
PDF
JavaScript - Good or Bad? Tips for Google Tag Manager
📊 Markus Baersch
 
PPTX
Listify-Intelligent-Voice-to-Catalog-Agent.pptx
nareshkottees
 
PDF
apidays Helsinki & North 2025 - How (not) to run a Graphql Stewardship Group,...
apidays
 
PDF
apidays Singapore 2025 - The API Playbook for AI by Shin Wee Chuang (PAND AI)
apidays
 
PDF
Context Engineering for AI Agents, approaches, memories.pdf
Tamanna36
 
PDF
apidays Singapore 2025 - Trustworthy Generative AI: The Role of Observability...
apidays
 
PDF
The European Business Wallet: Why It Matters and How It Powers the EUDI Ecosy...
Lal Chandran
 
PDF
apidays Singapore 2025 - Streaming Lakehouse with Kafka, Flink and Iceberg by...
apidays
 
PDF
Driving Employee Engagement in a Hybrid World.pdf
Mia scott
 
PPTX
Advanced_NLP_with_Transformers_PPT_final 50.pptx
Shiwani Gupta
 
PPTX
apidays Helsinki & North 2025 - Agentic AI: A Friend or Foe?, Merja Kajava (A...
apidays
 
tuberculosiship-2106031cyyfuftufufufivifviviv
AkshaiRam
 
Product Management in HealthTech (Case Studies from SnappDoctor)
Hamed Shams
 
SlideEgg_501298-Agentic AI.pptx agentic ai
530BYManoj
 
OOPs with Java_unit2.pdf. sarthak bookkk
Sarthak964187
 
apidays Munich 2025 - Building an AWS Serverless Application with Terraform, ...
apidays
 
apidays Singapore 2025 - Surviving an interconnected world with API governanc...
apidays
 
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
Avatar for apidays apidays PRO June 07, 2025 0 5 apidays Helsinki & North 2...
apidays
 
Simplifying Document Processing with Docling for AI Applications.pdf
Tamanna36
 
JavaScript - Good or Bad? Tips for Google Tag Manager
📊 Markus Baersch
 
Listify-Intelligent-Voice-to-Catalog-Agent.pptx
nareshkottees
 
apidays Helsinki & North 2025 - How (not) to run a Graphql Stewardship Group,...
apidays
 
apidays Singapore 2025 - The API Playbook for AI by Shin Wee Chuang (PAND AI)
apidays
 
Context Engineering for AI Agents, approaches, memories.pdf
Tamanna36
 
apidays Singapore 2025 - Trustworthy Generative AI: The Role of Observability...
apidays
 
The European Business Wallet: Why It Matters and How It Powers the EUDI Ecosy...
Lal Chandran
 
apidays Singapore 2025 - Streaming Lakehouse with Kafka, Flink and Iceberg by...
apidays
 
Driving Employee Engagement in a Hybrid World.pdf
Mia scott
 
Advanced_NLP_with_Transformers_PPT_final 50.pptx
Shiwani Gupta
 
apidays Helsinki & North 2025 - Agentic AI: A Friend or Foe?, Merja Kajava (A...
apidays
 

Workshop on Google Cloud Data Platform

  • 2. Agenda 1. Intro GCP for Data 09:00 - 09:30 2. Roles & tools (per role) 09:35 - 10:10 3. Build on GCP (workgroups) 10:30 - 12:30 ● Data Democratization ● Why (Google) Cloud? ● Data platforms ● Data Engineer ● Analytics Engineer ● Analyst ● clean & prep data ● build the model ● create & share insights
  • 4. data analytics & BI prescriptive predictive descriptive diagnostic
  • 8. Infrastructure Big Data and Machine Learning Application Development G Suite
  • 9. For the past 15 years, Google has been building out the fastest, most powerful, highest quality cloud infrastructure on the planet. Images by Connie Zhou
  • 10. Google Global Cache (GGC) edge nodes Points of presence (>100) Network fiber FASTER (US, JP, TW) 2016 Unity (US, JP) 2010 SJC (JP, HK, SG) 2013 Monet (US, BR) 2017 Google network More than a collection of data centers
  • 11. research.google.com/pubs/papers.html Google has been innovating data technologies 2002 2004 2006 2008 2010 2012 2014 2016 GFS MapReduce TensorFlow Bigtable Dremel Colossus Flume Megastore Spanner Millwheel Pub/Sub F1 Google needed to invent data processing methods
  • 12. 2002 2004 2006 2008 2010 2012 2014 Google has been innovating data technologies 2016 Cloud Storage Dataproc ML Engine Bigtable BigQuery Cloud Storage Dataflow Datastore Dataflow Pub/Sub Google then shared it’s innovations research.google.com/pubs/papers.html Auto ML 2018
  • 13. process & analyze transform to information ● meaningful ● usable ingest read raw data ● streaming ● batch ● ad-hoc store store in the right format ● durable ● accessible explore & visualize convert to insights ● insightful ● shareable data lifecycle – 4 steps
  • 15. the modern data platform
  • 16. data life cycle with BigQuery & Looker raw data clean sources business logic data engineer analytics engineer explorer BigQuery / dataform ERP source systems Finance HR Marketing Other data platform reports viewer Looker
  • 17. ● storage & compute ● extremely fast, very cost-efficient ● use (standard) SQL ● integrate ○ Cloud SQL ○ Data Studio ○ Connected Sheets ● BQ ML ○ machine learning “for business” ○ SQL-powered ○ brings ML to the data modern data warehouse BigQuery What is BigQuery?
  • 18. Big(!) Data with Big Query more info→ !
  • 19. Dataform & BigQuery ● Open source, SQL-based language to manage data transformations ● Fully managed, serverless orchestration for data pipelines ● Fully featured cloud development environment to develop with SQL
  • 20. Looker Looker Data Platform ● modern data technology ● modern problems
  • 21. databases then databases now maximize efficiency compensate for inefficiency database technology has changed
  • 23. Data Lake Data Storage Best practice for companies to centralise their data Data Extraction Data Analysts extracting your data into workbooks or aggregated cubes HARD TO MAINTAIN HARD TO SCALE → DATA CHAOS Data Visualisation BI tool sits on top of these siloed workbooks to present dashboards and reports LIMITED DATA MULTIPLE TRUTHS → DATA BOTTLENECK Tech team headcount legacy BI “workbook” architecture
  • 24. Looker’s universal semantic model governed metrics best-in-class APIs in-database Git version control security Cloud integrated insights modern BI & analytics data-driven workflows custom applications SQL in results back
  • 25. Agenda 1. Intro GCP for Data 09:00 - 09:30 2. Roles & tools (per role) 09:45 - 10:30 3. Build on GCP (workgroups) 10:45 - 12:30 ● Data Democratization ● Why (Google) Cloud? ● Data platforms ● Data Engineer ● Analytics Engineer ● Analyst ● clean & prep data ● build the model ● create & share insights
  • 26. Looker User • Looker Explorer • Looker Dashboarding Looker Dev • (BigQuery & Dataform) • Looker & LookML Data Engineer • BigQuery • Dataform • Terraform Tools by role Pick your preferred role → https://leend.rs/GDF-pick (with your Google-account!) Analytics Engineer Data Analyst Cloud Data Engineer
  • 27. Looker User • Looker Explorer • Looker Dashboarding Looker Dev • (BigQuery & Dataform) • Looker & LookML Data Engineer • BigQuery • Dataform • Terraform Tools by role Pick your preferred role → https://leend.rs/GDF-pick (with your Google-account!) Data Analyst Analytics Translator Machine Learning Engineer Data Architect Data Scientist Business Analyst Analytics Engineer Cloud Data Engineer
  • 28. Tools by role Looker User • Looker Explorer • Looker Dashboarding Looker Data Explorer - Qwik Start → https://leend.rs/GDF-QL-An1 Filtering and Sorting Data in Looker → https://leend.rs/GDF-QL-An2 Data Engineer • BigQuery • Dataform • Terraform console.cloud.google.com/ ... ?project=go-data-fest → https://leend.rs/GDF-project Looker Dev • (BigQuery & Dataform) • Looker & LookML Looker Developer - Qwik Start → https://leend.rs/GDF-QL-AE1 Creating Measures and Dimensions Using LookML → https://leend.rs/GDF-QL-AE2
  • 29. Agenda 1. Intro GCP for Data 2. Roles & tools (per role) 3. Build on GCP (mixed workgroups) ● Data Democratization ● Why (Google) Cloud? ● Data platforms ● Data Engineer ● Analytics Engineer ● Analyst ● clean & prep data ● build the model ● create & share insights
  • 30. Groups & roles 30 Team Google-user Data Engineer Looker Dev Looker User Arthur erik.clabbers@... 1 fcm073@... 1 haydnruthams@... 1 thomas.hantke@... 1 Ford caiofabiomc@... 1 chung.kally@... 1 debbysmit@... 1 mfharms6@... 1 spstrempel@... 1 Trillian christovvillamon@... 1 e.j.m.hamberg@... 1 saheli.de@... 1 vhverhagen@... 1
  • 31. 1. Build the dataset 2. Build the Model, Explore & Views 3. Create and Share Dashboards → leend.rs/GDF-project → gcompany.eu.looker.com Now let’s have some fun! Insights needed ● What products show yearly seasonal (sales) trends? ● How are stocks in the distribution centers doing? ● ● How are order prices related to product list prices ● Is there a relationship between a product’s events & sales