SlideShare a Scribd company logo
Neat Analytics with Pandas
A Closer Look at Pandas Indexing
Alexander C. S. Hendorf
@hendorf
Alexander C. S. Hendorf
Königsweg GmbH
Strategic data consulting for startups and the industry.
EuroPython & PyConDE 

Organisator + Programm Chair
mongoDB master, PSF managing member
Speaker mongoDB days, EuroPython, PyData…
@hendorf
Today
Closer Look at Indexes
- Catch up on Pandas indexing
- Accessing data using the index
- Index Types
- MultiIndex
- Closer look at DateTimeIndex and Resampling
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Structure: Index
-the label of a series is usually called index
-automatically created if not given
-can be reset or replaced
-immutable ndarray implementing an ordered, sliceable set
-can only contain hashable objects
-one or more dimensions
-may contain a value more than once (NOT UNIQUE!)
Index Types
-Index
-MultiIndex
-DateTimeIndex
-TimeDelta
-IntervalIndex
-CategoricalIndex
Structure
DataFrame
2D
1
2
3
4
5
6
7
1
2
3
4
5
6
7
1
2
3
4
5
6
7
1
2
3
4
5
6
7
1
2
3
4
5
6
7
1
2
3
4
5
6
7
1
2
3
4
5
6
7
1
2
3
4
5
6
7
…
Panel 3DXXXXXXXXX
pd.Series
1D
1
2
3
4
5
6
7
1
2
3
4
5
6
7
Index Data: Numpy array
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Axes
1 2 3
0
1
0
1
2
3
4
5
6
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
no match: NaN
only if index matches
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
MultiIndex
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
max value of each series (not row)
get level 0 data
get level 1 data
get level 2 data
•
•
•
•
DateTimeIndex
-index of datetime64 data
2014-09-26T03:50:00,14.0
2014-08-10T05:00:00,14
2014-08-21T22:50:00,12.0
2014-08-17T13:20:00,16.0
2014-08-06T01:20:00,14.0
2014-09-27T06:50:00,11.0
2014-08-25T21:50:00,13.0
2014-08-14T05:20:00,13.0
2014-09-14T05:20:00,16.0
2014-08-03T02:50:00,21.0
2014-09-29T03:00:00,13
2014-09-06T08:20:00,16.0
2014-08-19T07:20:00,13.0
2014-09-27T22:50:00,10.0
2014-08-28T08:20:00,12.0
2014-08-17T01:00:00,14
2014-09-27T14:00:00,17
2014-09-10T18:00:00,18
2014-09-22T23:00:00,8
2014-09-20T03:00:00,9
2014-08-29T09:50:00,16.0
2014-08-16T01:50:00,13.0
Neat Analytics with Pandas 4 3 [PyParis]
DateTime
format="%d.%m.%Y %H:%M:%S
before DateTimeIndex: unordered
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Resampling
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Neat Analytics with Pandas 4 3 [PyParis]
Resampling
-H hourly frequency
-T minutely frequency
-S secondly frequency
-L milliseonds
-U microseconds
-N nanoseconds
-D calendar day frequency
-W weekly frequency
-M month end frequency
-Q quarter end frequency
-A year end frequency
- B business day frequency
- C custom business day frequency (experimenta
- BM business month end frequency
- CBM custom business month end frequency
- MS month start frequency
- BMS business month start frequency
- CBMS custom business month start frequency
- BQ business quarter endfrequency
- QS quarter start frequency
- BQS business quarter start frequency
- BA business year end frequency
- AS year start frequency
- BAS business year start frequency
- BH business hour frequency
Extra discounts for
students & post docs
#16
180+sessions 18free
trainings
panels open
spaces
5dtalks &
trainings
2dsprints
beginners’ day
Tickets start @ 375€
Rimini
. Venice !
Bologna ! ✈ .
Florence ! . #
$
Rome ! .
Armin Rohnacher • Katharine Jarmul • Tracy Osborn
Jan Willem Tulp • Aisha Bello & Daniele Procida
interactive
sessions
25. - 27. October
ZKM, Karlsruhe
CfP is open!
Alexander C. S. Hendorf
ah@koenigsweg.com
@hendorf

More Related Content

What's hot (16)

PDF
Cache conscious index mechanism for main-memory databases
Red Over
 
PPTX
Graph db - Pramati Technologies [Meetup]
Pramati Technologies
 
PDF
simple introduction to hadoop
vishnu rao
 
PDF
Datomic rtree-pres
jsofra
 
PPT
Hw09 Hadoop Development At Facebook Hive And Hdfs
Cloudera, Inc.
 
PDF
Introduction to Hadoop : A bird eye's view | Abhishek Mukherjee
FinTechopedia
 
PPTX
Functions using stack and heap
baabtra.com - No. 1 supplier of quality freshers
 
PPT
Big Data & Hadoop
Thanakrit Lersmethasakul
 
PPT
Taste Java In The Clouds
Jacky Chu
 
DOC
Devry gsp 215 week 3 homework representing and manipulating information new
williamethan912
 
PPTX
Introduction to Map Reduce
Apache Apex
 
PPTX
GeoTuple a Framework for Web Based Geo-Analytics with R and PostGIS
Roland Hansson
 
PDF
20190909_PGconf.ASIA_KaiGai
Kohei KaiGai
 
PDF
3.5 webinar
ArangoDB Database
 
PDF
Using python to analyze spatial data
Kudos S.A.S
 
PDF
Pycon tw 2013
show you
 
Cache conscious index mechanism for main-memory databases
Red Over
 
Graph db - Pramati Technologies [Meetup]
Pramati Technologies
 
simple introduction to hadoop
vishnu rao
 
Datomic rtree-pres
jsofra
 
Hw09 Hadoop Development At Facebook Hive And Hdfs
Cloudera, Inc.
 
Introduction to Hadoop : A bird eye's view | Abhishek Mukherjee
FinTechopedia
 
Functions using stack and heap
baabtra.com - No. 1 supplier of quality freshers
 
Big Data & Hadoop
Thanakrit Lersmethasakul
 
Taste Java In The Clouds
Jacky Chu
 
Devry gsp 215 week 3 homework representing and manipulating information new
williamethan912
 
Introduction to Map Reduce
Apache Apex
 
GeoTuple a Framework for Web Based Geo-Analytics with R and PostGIS
Roland Hansson
 
20190909_PGconf.ASIA_KaiGai
Kohei KaiGai
 
3.5 webinar
ArangoDB Database
 
Using python to analyze spatial data
Kudos S.A.S
 
Pycon tw 2013
show you
 

Similar to Neat Analytics with Pandas 4 3 [PyParis] (20)

PDF
Introduction to Pandas and Time Series Analysis [PyCon DE]
Alexander Hendorf
 
PDF
Introduction to Pandas and Time Series Analysis [Budapest BI Forum]
Alexander Hendorf
 
PDF
Real-time Cassandra
Acunu
 
PDF
Steam Learn: Introduction to RDBMS indexes
inovia
 
PDF
Acunu Analytics: Simpler Real-Time Cassandra Apps
Acunu
 
PDF
Cassandra summit keynote 2014
jbellis
 
PPTX
Deployment Preparedness
MongoDB
 
PDF
Non-Relational Postgres
EDB
 
PDF
FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...
Rob Skillington
 
PDF
MongoDB as a Universal Data Store for Process Data
MongoDB
 
PDF
TechEvent Time Seriesd Databases
Trivadis
 
PDF
Mongo db improve the performance of your application codemotion2016
Juan Antonio Roy Couto
 
KEY
NOSQL101, Or: How I Learned To Stop Worrying And Love The Mongo!
Daniel Cousineau
 
PDF
Cassandra for impatients
Carlos Alonso Pérez
 
PDF
Non Relational Databases And World Domination
Jason Davies
 
PDF
Use Performance Insights To Enhance MongoDB Performance - (Manosh Malai - Myd...
Mydbops
 
PDF
Indexes in PostgreSQL (10)
Giuseppe Broccolo
 
PPTX
ParStream - Big Data for Business Users
ParStream Inc.
 
PPTX
MongoDB Days Silicon Valley: Jumpstart: The Right and Wrong Use Cases for Mon...
MongoDB
 
PDF
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Ontico
 
Introduction to Pandas and Time Series Analysis [PyCon DE]
Alexander Hendorf
 
Introduction to Pandas and Time Series Analysis [Budapest BI Forum]
Alexander Hendorf
 
Real-time Cassandra
Acunu
 
Steam Learn: Introduction to RDBMS indexes
inovia
 
Acunu Analytics: Simpler Real-Time Cassandra Apps
Acunu
 
Cassandra summit keynote 2014
jbellis
 
Deployment Preparedness
MongoDB
 
Non-Relational Postgres
EDB
 
FOSDEM 2019: M3, Prometheus and Graphite with metrics and monitoring in an in...
Rob Skillington
 
MongoDB as a Universal Data Store for Process Data
MongoDB
 
TechEvent Time Seriesd Databases
Trivadis
 
Mongo db improve the performance of your application codemotion2016
Juan Antonio Roy Couto
 
NOSQL101, Or: How I Learned To Stop Worrying And Love The Mongo!
Daniel Cousineau
 
Cassandra for impatients
Carlos Alonso Pérez
 
Non Relational Databases And World Domination
Jason Davies
 
Use Performance Insights To Enhance MongoDB Performance - (Manosh Malai - Myd...
Mydbops
 
Indexes in PostgreSQL (10)
Giuseppe Broccolo
 
ParStream - Big Data for Business Users
ParStream Inc.
 
MongoDB Days Silicon Valley: Jumpstart: The Right and Wrong Use Cases for Mon...
MongoDB
 
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Ontico
 
Ad

More from Alexander Hendorf (9)

PDF
Deep Learning for Fun and Profit [PyConDE 2018]
Alexander Hendorf
 
PDF
Databases for Data Science
Alexander Hendorf
 
PDF
Agile Datenanalsyse - der schnelle Weg zum Mehrwert
Alexander Hendorf
 
PDF
Einführung Datenanalyse mit Pandas [data2day]
Alexander Hendorf
 
PDF
Data Mangling with mongoDB the Right Way [PyData London] 2016]
Alexander Hendorf
 
PDF
NoSQL oder: Freiheit ist nicht schmerzfrei - IT Tage
Alexander Hendorf
 
PDF
Data analysis and visualization with mongo db [mongodb world 2016]
Alexander Hendorf
 
PDF
Time travel and time series analysis with pandas + statsmodels
Alexander Hendorf
 
PDF
Data mangling with mongo db the right way [pyconit 2016]
Alexander Hendorf
 
Deep Learning for Fun and Profit [PyConDE 2018]
Alexander Hendorf
 
Databases for Data Science
Alexander Hendorf
 
Agile Datenanalsyse - der schnelle Weg zum Mehrwert
Alexander Hendorf
 
Einführung Datenanalyse mit Pandas [data2day]
Alexander Hendorf
 
Data Mangling with mongoDB the Right Way [PyData London] 2016]
Alexander Hendorf
 
NoSQL oder: Freiheit ist nicht schmerzfrei - IT Tage
Alexander Hendorf
 
Data analysis and visualization with mongo db [mongodb world 2016]
Alexander Hendorf
 
Time travel and time series analysis with pandas + statsmodels
Alexander Hendorf
 
Data mangling with mongo db the right way [pyconit 2016]
Alexander Hendorf
 
Ad

Recently uploaded (20)

PDF
apidays Singapore 2025 - Trustworthy Generative AI: The Role of Observability...
apidays
 
PPTX
apidays Singapore 2025 - From Data to Insights: Building AI-Powered Data APIs...
apidays
 
PPTX
apidays Helsinki & North 2025 - API access control strategies beyond JWT bear...
apidays
 
PDF
apidays Singapore 2025 - The API Playbook for AI by Shin Wee Chuang (PAND AI)
apidays
 
PDF
apidays Singapore 2025 - Surviving an interconnected world with API governanc...
apidays
 
PPTX
Numbers of a nation: how we estimate population statistics | Accessible slides
Office for National Statistics
 
PDF
Context Engineering for AI Agents, approaches, memories.pdf
Tamanna
 
PPT
Growth of Public Expendituuure_55423.ppt
NavyaDeora
 
PDF
apidays Singapore 2025 - From API Intelligence to API Governance by Harsha Ch...
apidays
 
PPTX
ER_Model_with_Diagrams_Presentation.pptx
dharaadhvaryu1992
 
PPTX
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
PDF
apidays Singapore 2025 - How APIs can make - or break - trust in your AI by S...
apidays
 
PDF
Development and validation of the Japanese version of the Organizational Matt...
Yoga Tokuyoshi
 
PPTX
Listify-Intelligent-Voice-to-Catalog-Agent.pptx
nareshkottees
 
PDF
Research Methodology Overview Introduction
ayeshagul29594
 
PDF
JavaScript - Good or Bad? Tips for Google Tag Manager
📊 Markus Baersch
 
PDF
apidays Helsinki & North 2025 - Monetizing AI APIs: The New API Economy, Alla...
apidays
 
PPTX
b6057ea5-8e8c-4415-90c0-ed8e9666ffcd.pptx
Anees487379
 
PPTX
apidays Helsinki & North 2025 - From Chaos to Clarity: Designing (AI-Ready) A...
apidays
 
PDF
The European Business Wallet: Why It Matters and How It Powers the EUDI Ecosy...
Lal Chandran
 
apidays Singapore 2025 - Trustworthy Generative AI: The Role of Observability...
apidays
 
apidays Singapore 2025 - From Data to Insights: Building AI-Powered Data APIs...
apidays
 
apidays Helsinki & North 2025 - API access control strategies beyond JWT bear...
apidays
 
apidays Singapore 2025 - The API Playbook for AI by Shin Wee Chuang (PAND AI)
apidays
 
apidays Singapore 2025 - Surviving an interconnected world with API governanc...
apidays
 
Numbers of a nation: how we estimate population statistics | Accessible slides
Office for National Statistics
 
Context Engineering for AI Agents, approaches, memories.pdf
Tamanna
 
Growth of Public Expendituuure_55423.ppt
NavyaDeora
 
apidays Singapore 2025 - From API Intelligence to API Governance by Harsha Ch...
apidays
 
ER_Model_with_Diagrams_Presentation.pptx
dharaadhvaryu1992
 
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
apidays Singapore 2025 - How APIs can make - or break - trust in your AI by S...
apidays
 
Development and validation of the Japanese version of the Organizational Matt...
Yoga Tokuyoshi
 
Listify-Intelligent-Voice-to-Catalog-Agent.pptx
nareshkottees
 
Research Methodology Overview Introduction
ayeshagul29594
 
JavaScript - Good or Bad? Tips for Google Tag Manager
📊 Markus Baersch
 
apidays Helsinki & North 2025 - Monetizing AI APIs: The New API Economy, Alla...
apidays
 
b6057ea5-8e8c-4415-90c0-ed8e9666ffcd.pptx
Anees487379
 
apidays Helsinki & North 2025 - From Chaos to Clarity: Designing (AI-Ready) A...
apidays
 
The European Business Wallet: Why It Matters and How It Powers the EUDI Ecosy...
Lal Chandran
 

Neat Analytics with Pandas 4 3 [PyParis]