SlideShare a Scribd company logo
7
Most read
10
Most read
16
Most read
MODY UNIVERSITY OF SCIENCE
ANDTECHNOLOGY
Colloquium Presentation
CS 14.371
Submitted to-
Dr. Pervesh Kumar
Bishnoi
Ms. Sonal Shukla
Submitted by-
Akshita Kanther
B.Tech. IIIrd yr C2
Er.No.-180161
Presentation on data preparation with pandas
CONTENTS
Introduction
Why Should We Prepare Our Data
Python
Python Libraries
Pandas
Features of Pandas
Core Components Of Pandas
Pandas Operations
Typical Pipeline For Data Preparation
Common Tasks Involved In Data Preparation
Applications Of Pandas
Companies using Pandas
Summary
INTRODUCTION
Data preparation is the first step after you get your
hands on any kind of dataset. This is the step when you
pre-process raw data into a form that can be easily and
accurately analyzed. Proper data preparation allows for
efficient analysis - it can eliminate errors and
inaccuracies that could have occurred during the data
gathering process and can thus help in removing some
bias resulting from poor data quality. Therefore a lot of
an analyst's time is spent on this vital step.
WHY SHOULD
WE PREPARE
OUR DATA
Garbage in, garbage out
Reduce errors
Remove duplicate records
Fix missing values
Correct range values
Fix formatting (i.e. date, text, number)
PYTHON
 Object-oriented, high-level
programming language
 Used as a scripting language to
connect existing components
together
 Simple, easy to learn syntax
emphasizes readability
 Supports modules and
packages
PYTHON LIBRARIES
Many popular Python
toolboxes/libraries:-
• NumPy
• SciPy
• Pandas
• SciKit-Learn
Visualization libraries:-
• matplotlib
• Seaborn
Presentation on data preparation with pandas
PANDAS
• Pandas is a software library written for Python
• Pandas has so many uses that it might make sense to list
the things it can't do instead of what it can do
• This tool is essentially your data’s home. Through pandas,
you get acquainted with your data by cleaning,
transforming, and analyzing it
• Pandas is well suited for different kinds of data, such as:
 Tabular data with heterogeneously-typed columns
 Ordered and unordered time series data
 Arbitrary matrix data with row & column labels
 Unlabelled data
 Any other form of observational or statistical data sets
To use the pandas library, you need to first import it. Just type
this in your python console:
Presentation on data preparation with pandas
CORE COMPONENTS OF PANDAS
The primary two components of Pandas are:-
Dataframe
 Series
A Series is essentially column and a Dataframe is a
multidimensional Table made up of a collection of Series.
PANDAS OPERATIONS
Using Python pandas, you can perform a lot of operations with series, data frames, missing data,
group by etc. Some of the common operations for data manipulation are listed below:
TYPICAL PIPELINE FOR DATA
PREPARATION
• The first step of a data preparation pipeline is to gather data from various
sources and locations
• Before any processing is done, we wish to discover what the data is about.
At this stage, we understand the data within the context of business goals
and Visualization of the data is also helpful here
• The next stage is to cleanse the data of missing values and invalid values.
We also reformat data to standard forms
• Next we transform the data for a specific outcome or audience
• We can enrich data by merging different datasets to enable richer insights
• Finally, we store the data or directly send it out for analytics
COMMON TASKS INVOLVED IN DATA
PREPARATION
Tasks involved in
Data Preparation
Aggregation
Augmentation
Decomposing
Deletion
Blending
Anonymization
Data preparation involves one or more of the following tasks:
•Aggregation: Multiple columns are reduced to fewer columns.
Records are summarized
•Anonymization: Sensitive values are removed for the sake of
privacy
•Augmentation: Expand the dataset size without collecting more
data. For example, image data is augmented via cropping or
rotating
•Blending: Combine and link related data from various sources. For
example, combine an employee's HR data with payroll data
•Decomposing: Decompose a data column that has sub-fields. For
example, "6 ounces butter" is decomposed into three columns
representing value, unit and ingredient
•Deletion: Duplicates and outliers are removed. Exploratory Data
Analysis (EDA) may be used to identify outliers
APPLICATIONS OF
PANDAS
COMPANIES USING PANDAS
SUMMARY
Raw data is usually not suitable for direct analysis.
This is because the data might come from different
sources in different formats. Moreover, real-world
data is not clean. Some data points might be
missing. Some others might be out of range.
There could be duplicates. Data preparation is
therefore an essential task that transforms or
prepares data into a form that's suitable for
analysis.
Data preparation assumes that data has already
been collected. However, others may consider
data collection and data ingestion as part of data
preparation. Within data preparation, it's common
to identify sub-stages that might include data pre-
processing, data wrangling, and data
transformation.
Presentation on data preparation with pandas
Presentation on data preparation with pandas

More Related Content

What's hot (20)

PDF
pandas - Python Data Analysis
Andrew Henshaw
 
PPTX
Introduction to pandas
Piyush rai
 
PPTX
Python Scipy Numpy
Girish Khanzode
 
PDF
Introduction to Pandas and Time Series Analysis [PyCon DE]
Alexander Hendorf
 
PPTX
Pandas
Jyoti shukla
 
PDF
Introduction to NumPy
Huy Nguyen
 
PDF
Introduction to Python Pandas for Data Analytics
Phoenix
 
PPTX
Data Structures in Python
Devashish Kumar
 
PPTX
Python Seaborn Data Visualization
Sourabh Sahu
 
PDF
pandas: Powerful data analysis tools for Python
Wes McKinney
 
PDF
Pandas
maikroeder
 
PDF
Python Functions Tutorial | Working With Functions In Python | Python Trainin...
Edureka!
 
PPTX
DataFrame in Python Pandas
Sangita Panchal
 
PPTX
Python pandas Library
Md. Sohag Miah
 
PDF
Python NumPy Tutorial | NumPy Array | Edureka
Edureka!
 
PPTX
Python
Aashish Jain
 
PDF
Python GUI
LusciousLarryDas
 
PPTX
Data preprocessing in Machine learning
pyingkodi maran
 
PPTX
Data Analysis in Python-NumPy
Devashish Kumar
 
PDF
Data Visualization in Python
Jagriti Goswami
 
pandas - Python Data Analysis
Andrew Henshaw
 
Introduction to pandas
Piyush rai
 
Python Scipy Numpy
Girish Khanzode
 
Introduction to Pandas and Time Series Analysis [PyCon DE]
Alexander Hendorf
 
Pandas
Jyoti shukla
 
Introduction to NumPy
Huy Nguyen
 
Introduction to Python Pandas for Data Analytics
Phoenix
 
Data Structures in Python
Devashish Kumar
 
Python Seaborn Data Visualization
Sourabh Sahu
 
pandas: Powerful data analysis tools for Python
Wes McKinney
 
Pandas
maikroeder
 
Python Functions Tutorial | Working With Functions In Python | Python Trainin...
Edureka!
 
DataFrame in Python Pandas
Sangita Panchal
 
Python pandas Library
Md. Sohag Miah
 
Python NumPy Tutorial | NumPy Array | Edureka
Edureka!
 
Python
Aashish Jain
 
Python GUI
LusciousLarryDas
 
Data preprocessing in Machine learning
pyingkodi maran
 
Data Analysis in Python-NumPy
Devashish Kumar
 
Data Visualization in Python
Jagriti Goswami
 

Similar to Presentation on data preparation with pandas (20)

PPTX
data wrangling (1).pptx kjhiukjhknjbnkjh
VISHALMARWADE1
 
PPTX
this includes basics about python modules and packages introduction
ssuseree48e0
 
PPTX
Python short notes on modules and applications
ssuseree48e0
 
PDF
Data preprocessing using Machine Learning
Gopal Sakarkar
 
PPT
Preprocessing_new.ppt
YashikaSengar2
 
PPTX
To understand the importance of Python libraries in data analysis.
GurpinderSingh98
 
PPT
preproccessing level 3 for students.ppt
AhmedAlrashdy
 
PPTX
python-pandas-For-Data-Analysis-Manipulate.pptx
PLOKESH8
 
PPTX
Transformation techniques.pptx
PremaGanesh1
 
PPTX
Data Preparation.pptx
YashikaSengar2
 
PPTX
Meetup Junio Data Analysis with python 2018
DataLab Community
 
PDF
Data warehousing
Juhi Mahajan
 
PDF
BI Masterclass slides (Reference Architecture v3)
Syaifuddin Ismail
 
PPTX
Data Preprocessing&tools
Amandeep Gill
 
PDF
ML_Lec2 introduction to data processing.pdf
BeshoyArnest
 
PDF
Data Wrangling with Python_ Cleaning and Preparing Datasets for Analysis.pdf
ExcelR- Data Science, Data Analyst, Business Analyst Course Training in Delhi
 
PPTX
Data_analytics_life_cycle in big data.pptx
calvinctttafirei
 
PPT
Cssu dw dm
sumit621
 
PPTX
Introduction to data mining and data warehousing
Er. Nawaraj Bhandari
 
PPT
Data warehouse
_123_
 
data wrangling (1).pptx kjhiukjhknjbnkjh
VISHALMARWADE1
 
this includes basics about python modules and packages introduction
ssuseree48e0
 
Python short notes on modules and applications
ssuseree48e0
 
Data preprocessing using Machine Learning
Gopal Sakarkar
 
Preprocessing_new.ppt
YashikaSengar2
 
To understand the importance of Python libraries in data analysis.
GurpinderSingh98
 
preproccessing level 3 for students.ppt
AhmedAlrashdy
 
python-pandas-For-Data-Analysis-Manipulate.pptx
PLOKESH8
 
Transformation techniques.pptx
PremaGanesh1
 
Data Preparation.pptx
YashikaSengar2
 
Meetup Junio Data Analysis with python 2018
DataLab Community
 
Data warehousing
Juhi Mahajan
 
BI Masterclass slides (Reference Architecture v3)
Syaifuddin Ismail
 
Data Preprocessing&tools
Amandeep Gill
 
ML_Lec2 introduction to data processing.pdf
BeshoyArnest
 
Data Wrangling with Python_ Cleaning and Preparing Datasets for Analysis.pdf
ExcelR- Data Science, Data Analyst, Business Analyst Course Training in Delhi
 
Data_analytics_life_cycle in big data.pptx
calvinctttafirei
 
Cssu dw dm
sumit621
 
Introduction to data mining and data warehousing
Er. Nawaraj Bhandari
 
Data warehouse
_123_
 
Ad

Recently uploaded (20)

PPTX
b6057ea5-8e8c-4415-90c0-ed8e9666ffcd.pptx
Anees487379
 
PPTX
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
PPTX
SHREYAS25 INTERN-I,II,III PPT (1).pptx pre
swapnilherage
 
PPTX
05_Jelle Baats_Tekst.pptx_AI_Barometer_Release_Event
FinTech Belgium
 
PDF
The European Business Wallet: Why It Matters and How It Powers the EUDI Ecosy...
Lal Chandran
 
PPTX
Listify-Intelligent-Voice-to-Catalog-Agent.pptx
nareshkottees
 
PPTX
What Is Data Integration and Transformation?
subhashenia
 
PPTX
Aict presentation on dpplppp sjdhfh.pptx
vabaso5932
 
PDF
NIS2 Compliance for MSPs: Roadmap, Benefits & Cybersecurity Trends (2025 Guide)
GRC Kompas
 
PDF
apidays Singapore 2025 - Surviving an interconnected world with API governanc...
apidays
 
PPTX
thid ppt defines the ich guridlens and gives the information about the ICH gu...
shaistabegum14
 
PPTX
apidays Singapore 2025 - Generative AI Landscape Building a Modern Data Strat...
apidays
 
PPTX
04_Tamás Marton_Intuitech .pptx_AI_Barometer_2025
FinTech Belgium
 
PDF
OOPs with Java_unit2.pdf. sarthak bookkk
Sarthak964187
 
PDF
apidays Singapore 2025 - The API Playbook for AI by Shin Wee Chuang (PAND AI)
apidays
 
PPTX
Powerful Uses of Data Analytics You Should Know
subhashenia
 
PPT
tuberculosiship-2106031cyyfuftufufufivifviviv
AkshaiRam
 
PPTX
apidays Helsinki & North 2025 - From Chaos to Clarity: Designing (AI-Ready) A...
apidays
 
PDF
apidays Singapore 2025 - Trustworthy Generative AI: The Role of Observability...
apidays
 
PDF
A GraphRAG approach for Energy Efficiency Q&A
Marco Brambilla
 
b6057ea5-8e8c-4415-90c0-ed8e9666ffcd.pptx
Anees487379
 
apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehre...
apidays
 
SHREYAS25 INTERN-I,II,III PPT (1).pptx pre
swapnilherage
 
05_Jelle Baats_Tekst.pptx_AI_Barometer_Release_Event
FinTech Belgium
 
The European Business Wallet: Why It Matters and How It Powers the EUDI Ecosy...
Lal Chandran
 
Listify-Intelligent-Voice-to-Catalog-Agent.pptx
nareshkottees
 
What Is Data Integration and Transformation?
subhashenia
 
Aict presentation on dpplppp sjdhfh.pptx
vabaso5932
 
NIS2 Compliance for MSPs: Roadmap, Benefits & Cybersecurity Trends (2025 Guide)
GRC Kompas
 
apidays Singapore 2025 - Surviving an interconnected world with API governanc...
apidays
 
thid ppt defines the ich guridlens and gives the information about the ICH gu...
shaistabegum14
 
apidays Singapore 2025 - Generative AI Landscape Building a Modern Data Strat...
apidays
 
04_Tamás Marton_Intuitech .pptx_AI_Barometer_2025
FinTech Belgium
 
OOPs with Java_unit2.pdf. sarthak bookkk
Sarthak964187
 
apidays Singapore 2025 - The API Playbook for AI by Shin Wee Chuang (PAND AI)
apidays
 
Powerful Uses of Data Analytics You Should Know
subhashenia
 
tuberculosiship-2106031cyyfuftufufufivifviviv
AkshaiRam
 
apidays Helsinki & North 2025 - From Chaos to Clarity: Designing (AI-Ready) A...
apidays
 
apidays Singapore 2025 - Trustworthy Generative AI: The Role of Observability...
apidays
 
A GraphRAG approach for Energy Efficiency Q&A
Marco Brambilla
 
Ad

Presentation on data preparation with pandas

  • 1. MODY UNIVERSITY OF SCIENCE ANDTECHNOLOGY Colloquium Presentation CS 14.371 Submitted to- Dr. Pervesh Kumar Bishnoi Ms. Sonal Shukla Submitted by- Akshita Kanther B.Tech. IIIrd yr C2 Er.No.-180161
  • 3. CONTENTS Introduction Why Should We Prepare Our Data Python Python Libraries Pandas Features of Pandas Core Components Of Pandas Pandas Operations Typical Pipeline For Data Preparation Common Tasks Involved In Data Preparation Applications Of Pandas Companies using Pandas Summary
  • 4. INTRODUCTION Data preparation is the first step after you get your hands on any kind of dataset. This is the step when you pre-process raw data into a form that can be easily and accurately analyzed. Proper data preparation allows for efficient analysis - it can eliminate errors and inaccuracies that could have occurred during the data gathering process and can thus help in removing some bias resulting from poor data quality. Therefore a lot of an analyst's time is spent on this vital step.
  • 6. Garbage in, garbage out Reduce errors Remove duplicate records Fix missing values Correct range values Fix formatting (i.e. date, text, number)
  • 7. PYTHON  Object-oriented, high-level programming language  Used as a scripting language to connect existing components together  Simple, easy to learn syntax emphasizes readability  Supports modules and packages
  • 8. PYTHON LIBRARIES Many popular Python toolboxes/libraries:- • NumPy • SciPy • Pandas • SciKit-Learn Visualization libraries:- • matplotlib • Seaborn
  • 10. PANDAS • Pandas is a software library written for Python • Pandas has so many uses that it might make sense to list the things it can't do instead of what it can do • This tool is essentially your data’s home. Through pandas, you get acquainted with your data by cleaning, transforming, and analyzing it • Pandas is well suited for different kinds of data, such as:  Tabular data with heterogeneously-typed columns  Ordered and unordered time series data  Arbitrary matrix data with row & column labels  Unlabelled data  Any other form of observational or statistical data sets To use the pandas library, you need to first import it. Just type this in your python console:
  • 12. CORE COMPONENTS OF PANDAS The primary two components of Pandas are:- Dataframe  Series A Series is essentially column and a Dataframe is a multidimensional Table made up of a collection of Series.
  • 13. PANDAS OPERATIONS Using Python pandas, you can perform a lot of operations with series, data frames, missing data, group by etc. Some of the common operations for data manipulation are listed below:
  • 14. TYPICAL PIPELINE FOR DATA PREPARATION
  • 15. • The first step of a data preparation pipeline is to gather data from various sources and locations • Before any processing is done, we wish to discover what the data is about. At this stage, we understand the data within the context of business goals and Visualization of the data is also helpful here • The next stage is to cleanse the data of missing values and invalid values. We also reformat data to standard forms • Next we transform the data for a specific outcome or audience • We can enrich data by merging different datasets to enable richer insights • Finally, we store the data or directly send it out for analytics
  • 16. COMMON TASKS INVOLVED IN DATA PREPARATION Tasks involved in Data Preparation Aggregation Augmentation Decomposing Deletion Blending Anonymization Data preparation involves one or more of the following tasks: •Aggregation: Multiple columns are reduced to fewer columns. Records are summarized •Anonymization: Sensitive values are removed for the sake of privacy •Augmentation: Expand the dataset size without collecting more data. For example, image data is augmented via cropping or rotating •Blending: Combine and link related data from various sources. For example, combine an employee's HR data with payroll data •Decomposing: Decompose a data column that has sub-fields. For example, "6 ounces butter" is decomposed into three columns representing value, unit and ingredient •Deletion: Duplicates and outliers are removed. Exploratory Data Analysis (EDA) may be used to identify outliers
  • 19. SUMMARY Raw data is usually not suitable for direct analysis. This is because the data might come from different sources in different formats. Moreover, real-world data is not clean. Some data points might be missing. Some others might be out of range. There could be duplicates. Data preparation is therefore an essential task that transforms or prepares data into a form that's suitable for analysis. Data preparation assumes that data has already been collected. However, others may consider data collection and data ingestion as part of data preparation. Within data preparation, it's common to identify sub-stages that might include data pre- processing, data wrangling, and data transformation.