Data Management in R

SECTION 1
Importing & Exporting Data in R
2

Data Snapshot
The data has 12 rows and 6
columns
First_Name First Name
Last_Name Last Name
Grade Grade
Location Location
ba Basic
Allowance
ms Management
Supplements
Data Description
basic_salary_1_R1
3
Observations
Variables

salary_data <-read.table("C:/Users/Documents/basic_salary_1_R2.csv",
header = TRUE, sep = ",")
read.table() Function
Importing a .csv and .txt (tab delimited) file
header = TRUE (logical) indicates that the first row of the file
contains the names of the columns.
Default is header = FALSE
sep = ”,” specifies that the data is separated by commas.
Without this command, R imports data in a single column.
salary_data <-read.table("C:/Users/Documents/basic_salary_1_R2.txt",
header = TRUE, sep = "t", fill= TRUE)
First locate your data file, whether it is saved in the default working directory of R or any other
location in your system. If it is not stored in default working directory then you will have to give its
path for importing it into R. If you copy file path from the folder, ensure it uses forward slash (/).
Do not forget to accurately write the file name and extension.
*
fill= TRUE implicitly adds blank fields, in case the rows have unequal length
4

read.csv() Function
salary_data <- read.csv("C:/Users/Documents/basic_salary_1_R2.csv")
Here we haven’t given additional arguments for header= and sep=
because the function is specifically designed to import .csv - Comma
Separated Files and header=TRUE is assumed
• Apart from the functions in the base package, R also has several additional packages
to carry out efficient data import like readr, data.table, readxl.
5

Package readxl
install.packages("readxl")
library(readxl)
salary_data<-read_excel("C:/Users/Documents/basic_salary_1_R2.xlsx")
• readxl is an R package that provides function to read Excel worksheets in both .xls and
.xlsx formats.
# Install and load readxl
# Import file using read_excel
6

Importing Files Interactively
salary_data <- read.csv(file.choose())
• To manually select the directory and file where your dataset is located use
file.choose() function .
• file.choose() can be used as an independent function or can be put inside a
function with or without other arguments.
This opens a dialog box that allows you to choose the file interactively.
7

Exporting CSV, TXT and XLSX Files
write.csv(mydata, file = "MyData.csv")
• Methods for Exporting R objects into CSV, TXT and XLSX formats are given below:
# To a CSV
write.table(mydata, "mydata.txt", sep="t")
# To a Tab Delimited Text File
install.packages("xlsx")
library(xlsx)
write.xlsx(mydata, "mydata.xlsx")
# To an Excel Spreadsheet
package xlsx is used to import .xlsx files
8
mydata is the name of the object which
we want to save in csv format with
filename MyData in default working
directory.
mydata object will be saved as tab
delimited txt file as we have
specified the sep= “t”. It will be
saved in your working directory. You
can specify the path if you don’t
want to save in the currently loaded
working directory.

SECTION 2
Checking Data Features in R
9

Dimension of Data and Names of the
Columns
dim(salary_data)
[1] 12 6
dim() displays the no. of rows and columns in data.:
# Retrieve the Dimension of your data using dim()
Data contains 12 rows and 6 columns
OR
nrow() and ncol() can be used separately to know no. of rows
and columns.
names(salary_data)
[1] "First_Name" "Last_Name" "Grade" "Location"
[5] "ba" "ms"
# Get the Names of the columns using names()
10
names() displays the names of the columns of data

Internal Structure of Data
str(salary_data)
'data.frame': 12 obs. of 6 variables:
$ First_Name: Factor w/ 12 levels "Aaron","Adela",..: 4 3 10 5
$ Last_Name : Factor w/ 12 levels "Brown","Chavan",..: 1 12 5
$ Grade : Factor w/ 2 levels "GR1","GR2": 1 2 1 2 1 2 1 2 1
$ Location : Factor w/ 2 levels "DELHI","MUMBAI": 1 2 2 1 2 2
$ ba : int 17990 12390 19250 14780 19235 13390 23280
$ ms : int 16070 6630 14960 9300 15200 6700 13490 10760 NA
13220 ...
When R reads data, it treats different variable types in different ways. str() is the
easiest way to inspect how R treats variables in our dataframe. It compactly
displays a dataframe’s internal structure like this:
Structure gives the following information:
− Class of the object like in this case ‘salary_data’ is of ‘data.frame’ class
− Type of the variable.
− Number of levels of each factor.
− In case of numeric/integer it shows the values. In case of factor variable it shows which
entry is for which factor level.
11
Character variables are entered into a dataframe as factors in R

Check the Levels
levels(salary_data$Grade)
[1] "GR1" "GR2"
Our data has 4 factor variables. A factor is a categorical variable that can take only one
of a fixed, finite set of possibilities. Those possible categories are the levels. We can
check the levels using levels() function
12
You can assign individual levels, for example:
levels(salary_data$Grade)[1]<-"G1"
or as a group, for example:
levels(salary_data$Grade)<-c("G1", "G2")

Number of Missing Observations
nmiss<-sum(is.na(salary_data$ms))
nmiss
[1] 1
In R missing data are usually recorded as NA.
# Check the number of missing values in the data.
$ operator after a data frame object helps in selecting a column
from the data frame.
is.na() returns a logical vector giving information about missing
values.
Here sum() displays the sum of missing observations.
13

First n Rows of Data
head(salary_data)
First_Name Last_Name Grade Location ba ms
1 Alan Brown GR1 DELHI 17990 16070
2 Agatha Williams GR2 MUMBAI 12390 6630
3 Rajesh Kolte GR1 MUMBAI 19250 14960
4 Ameet Mishra GR2 DELHI 14780 9300
5 Neha Rao GR1 MUMBAI 19235 15200
6 Sagar Chavan GR2 MUMBAI 13390 6700
head() displays first 6 rows.
14
head(salary_data, n=2)
The no. of rows to be displayed can be customised to n

Last n Rows of data
tail(salary_data)
7 Aaron Jones GR1 MUMBAI 23280 13490
8 John Patil GR2 MUMBAI 13500 10760
9 Sneha Joshi GR1 DELHI 20660 NA
10 Gaurav Singh GR2 DELHI 13760 13220
11 Adela Thomas GR2 DELHI 13660 6840
12 Anup Save GR2 MUMBAI 11960 7880
tail() displays last 6 rows.
15
The no. of rows to be displayed can be customised to n by specifying the number of
rows.

Summarising Data
summary(salary_data)
First_Name Last_Name Grade Location
Aaron :1 Brown :1 GR1:5 DELHI :5
Adela :1 Chavan :1 GR2:7 MUMBAI:7
Agatha :1 Jones :1
Alan :1 Joshi :1
Ameet :1 Kolte :1
Anup :1 Mishra :1
(Other):6 (Other):6
ba ms
Min. :11960 Min. : 6630
1st Qu.:13472 1st Qu.: 7360
Median :14270 Median :10760
Mean :16155 Mean :11005
3rd Qu.:19239 3rd Qu.:14225
Max. :23280 Max. :16070
NA's :1
summary() gives summary of objects including datasets, variables, linear models,
etc
# Variables are summarised based on their type
16
When summary() is applied to a dataframe, it is
essentially applied to each column, and it summarises
all the columns.
For a continuous variable, it gives summary in the
form of : min, 1st quantile, median, mean, 3rd quantile,
max and count of NA’s (if any).
For a factor(categorical) variable, it gives the
frequency of each level or category like in this case:
Grade has 2 levels-GR1 and GR2 occurring 5 and 7
times respectively.

SECTION 3
Modifying Data in R
17

Change Variable Names and Content –
fix()
fix(salary_data)
Our data is saved as an object named salary_data.
In case we want to change the name of some variable or column and its values. We can use
the following command which lets us make changes interactively.
fix() function will open the data in a separate, editable Data
Editor like the screenshot below. You can go to the desired
column name or cell and make the changes manually.
Complete the task by closing the dialogue box File > Close.
Alternative function for editing data interactively is edit(), similar to fix(). The
difference is: edit() lets you edit an object and returns the new version and fix()
lets you edit an object and modifies the original.* 18

Change Variable Names – rename()
install.packages("reshape")
library(reshape)
salary_data<-rename(salary_data,c(ba="basic allowance"))
names(salary_data)
[1] "First_Name" "Last_Name" "Grade"
[4] "Location" "basic allowance" "ms"
Another way of renaming columns is by using package reshape.
rename() uses name of the data object and (old name="new name").
The result need to be saved in an object because rename() doesn’t modify the object directly.
You can rename multiple column names like this:
salary_data<-rename(salary_data, c(ba="basic allowance", ms="management
supplements"))
Alternative way to change column names:
names(salary_data)[names(salary_data)=="ba"] <- "basic allowance"
Note that this modifies salary_data directly; i.e. you don’t have to save the result
back into salary_data.
*
# Install and load package reshape
# Use rename() from package reshape to change the column names
19

Derive a New Variable
salary_data$newvariable<-salary_data$ba*0.05
head(salary_data,n=3)
First_Name Last_Name Grade Location ba ms newvariable
1 Alan Brown GR1 DELHI 17990 16070 899.50
2 Agatha Williams GR2 MUMBAI 12390 6630 619.50
3 Rajesh Kolte GR1 MUMBAI 19250 14960 962.50
Add a new variable to salary_data containing values as 5% of ba.
Specify new variable name after $ operator and assign values to it
We can recode the existing variables in same way like this:
salary_data$ba<-salary_data$ba + salary_data$newvariable* 20

Recode a Categorical Variable – ifelse()
salary_data$Location<-ifelse(salary_data$Location=="MUMBAI", 1, 2)
head(salary_data)
1 Alan Brown GR1 2 17990 16070
2 Agatha Williams GR2 1 12390 6630
3 Rajesh Kolte GR1 1 19250 14960
4 Ameet Mishra GR2 2 14780 9300
5 Neha Rao GR1 1 19235 15200
6 Sagar Chavan GR2 1 13390 6700
Recode Location ‘MUMBAI’ as 1 and ‘DELHI’ as 2
Location column is recoded using ifelse statement.
21Make a note of this syntax. It’s great for recoding within R programs
*

Recode a Continuous Variable into
Categorical Variable – ifelse()
salary_data$category<-ifelse(salary_data$ba <14000, "Low",
ifelse(salary_data$ba <19000, "Medium", "High"))
head(salary_data)
First_Name Last_Name Grade Location ba ms category
1 Alan Brown GR1 2 17990 16070 Medium
2 Agatha Williams GR2 1 12390 6630 Low
3 Rajesh Kolte GR1 1 19250 14960 High
4 Ameet Mishra GR2 2 14780 9300 Medium
5 Neha Rao GR1 1 19235 15200 High
6 Sagar Chavan GR2 1 13390 6700 Low
class(salary_data$category)
[1] "character"
Categorise the employees on the basis of their ba in three categories, namely, Low,
Medium and High
22
Nested ifelse statement is used. Alternate command using cut() function
salary_data$category<-
cut(salary_data$ba,breaks=c(0,13999,18999,Inf),labels=c("Low","Medium","High"))
Note that column category is of character type. You
will have to convert it to a factor using as.factor()

Remove Columns from a Data Frame
salary_data$Last_Name<-NULL
head(salary_data)
First_Name Grade Location ba ms newvariable
1 Alan GR1 DELHI 17990 16070 899.50
2 Agatha GR2 MUMBAI 12390 6630 619.50
3 Rajesh GR1 MUMBAI 19250 14960 962.50
4 Ameet GR2 DELHI 14780 9300 739.00
5 Neha GR1 MUMBAI 19235 15200 961.75
6 Sagar GR2 MUMBAI 13390 6700 669.50
To remove a column, set it to NULL
Remove a column Last_Name from salary_data.
To remove columns using integer indexing : salary_data[6]<-NULL
Index is the position of column. It starts from 1.
In this case index number of First_Name is 1, Grade is 2 and so on and so forth.* 23

Remove Rows from a Data Frame
salary_data[-(2:4),]
Remove unwanted rows from data using their index no.s. We can use the [] bracket
notation to access the indices of rows and columns like this: [R, C]. Here, the first index
is for Rows and the second is for Columns.
# Remove rows 2, 3 and 4 from salary_data .
24

Remove Rows from a Data Frame
salary_data[!(salary_data$Location=="MUMBAI"),]
25
Remove only rows which has Location as ‘MUMBAI’
Use negation (!) operator to remove the row which satisfies
the condition.
You can specify multiple conditions like this:
salary_data[!(salary_data$Location=="MUMBAI" &
salary_data$Grade=="GR2"),]

SECTION 4
Creating Subsets in R
26

Subsetting Observations
salary1<-subset(salary_data,Location=="MUMBAI" & ba>15000)
salary1
# Create a subset with all details of employees of MUMBAI with ba
more than 15000
There is no limit on how many
conditions may be combined
to achieve the desired subset.
subset() will return a data
frame and so this newly
created object can be used as
an argument in subset()
27
• Subsetting is extracting the needed rows (observations) and columns (variables)
from a dataset.
• The subset() function with conditions made from logical operators involving the
columns of the data frame will let you subset the data frame by observations.
• Our data is saved as an object named salary_data which is of class data.frame.
R assumes variable names are within the data frame being subset, so there is no
need to tell R where to look for Location and ba in the above example.
*

Subsetting Observations
salary2<-subset(salary_data,!(salary_data$Grade=="GR1") &
!(salary_data$Location=="MUMBAI"))
salary2
28
• Suppose we want all the details of employees not having GR1 and not from
MUMBAI, we will write the following command.
Negation (!) operator is used to give condition.

Subsetting Variables
salary3<-subset(salary1,select=c(Location,First_Name,ba))
salary3
Location First_Name ba
3 MUMBAI Rajesh 19250
5 MUMBAI Neha 19235
7 MUMBAI Aaron 23280
# Create a subset of only Location, First name and ba of salary1
data, created in the first example
select= takes columns to be selected from a data frame.
The order of columns in the output totally depends on how you
place the column names in select argument.
29
• We can also specify which columns we would like subset() to return by adding
a select argument

Subsetting Both Observations and
Variables
salary4<-subset(salary_data, Grade=="GR1" & ba>15000,
select= c(First_Name,Grade, Location))
salary4
First_Name Grade Location
1 Alan GR1 DELHI
3 Rajesh GR1 MUMBAI
5 Neha GR1 MUMBAI
7 Aaron GR1 MUMBAI
9 Sneha GR1 DELHI
# Select First_Name, Grade and Location of employees of GR1 with ba
more than 15000
30
• We can subset observations and variables by simply combining the previous two
methods of subsetting.

Row Subsetting
salary_data [5:10, ]
# Display rows from 5th to 10th
31
When we only want to subset rows we use the first index and leave the second index blank.
Leaving an index blank indicates that you want to keep all the elements in that dimension.
Use names() to know which column corresponds to which no. in the index. Use colon
notation(:) if rows have consecutive positions.

Row Subsetting
salary_data[c(1,3,5), ]
# Display only selected rows
32
c() can be used to give a list of
row indices if the rows are not in
sequential order

Column Subsetting
salary5<-salary_data[ ,1:4]
head(salary5)
First_Name Last_Name Grade Location
1 Alan Brown GR1 DELHI
2 Agatha Williams GR2 MUMBAI
3 Rajesh Kolte GR1 MUMBAI
4 Ameet Mishra GR2 DELHI
5 Neha Rao GR1 MUMBAI
6 Sagar Chavan GR2 MUMBAI
# Display columns 1 to 4
33
• We can subset variables also using [] bracket notation but now we use the second
index and leave the first index blank. This indicates that we want all the rows for
specific columns.

Row-Column Subsetting
salary6<-salary_data[c(1,5,8),c(1,2)]
salary6
First_Name Last_Name
1 Alan Brown
5 Neha Rao
8 John Patil
# Display rows 1,5,8 and columns 1 and 2
34
We can also subset the columns by name as in select for subset() like
this:
salary6<-salary_data[c(1,5,8),c(“First_Name”, “Last_Name”)]

SECTION 5
Sorting Data in R
35

Sorting data in Ascending Order
attach(salary_data)
ba_sorted_1<-salary_data[order(ba),]
ba_sorted_1
order() is
used to sort a
vector, matrix
or data frame.
By default, it
sorts in
ascending
order
36
# Sort salary_data by ba in Ascending order
Sorting is storage of data in sorted order, it can be in ascending or descending
order.

ba_sorted_2<-salary_data[order(-ba),]
ba_sorted_2
Sorting data in Descending Order
37
# Sort salary_data by ba in Descending order
The ‘-’ sign before a
numeric column
reverses the default
order.
Alternatively , here
you can add
decreasing=TRUE

Sorting by Factor Variable
gr_sorted<-salary_data[order(Grade),]
gr_sorted
• Sort data by column with characters / factors
Note that by default
order() sorts in
ascending order
38
# Sort salary_data by Grade

Sorting Data by Multiple Variables
grba_sorted<-salary_data[order(Grade,ba),]
grba_sorted
• Sort data by giving multiple columns; one column with characters / factors and
one with numerals
39
# Sort salary_data by Grade and ba
Here, data is first
sorted in increasing
order of Grade then
ba.

Multiple Variables & Multiple Ordering
Levels
grms_sorted<-salary_data[order(Grade,decreasing=TRUE,-ms),]
grms_sorted
• Sort data by giving multiple columns; one column with characters / factors and
one with numerals and multiple ordering levels
40
# Sort salary_data by Grade in Descending order and then by ms in
# Ascending order
Here, data is sorted by
Grade in descending
order and ms in
ascending order.
By default missing
values in data are put
last.
You can put it first by
adding an argument
na.last=FALSE in
order().

SECTION 6
Merging / Appending Data in R
41

Data Snapshot
columns
Employee_ID Employee ID
Last_Name Last Name
Basic_Salary Basic Salary
Data Descriptionsal_data_R2
bonus_data_R2 Data Description
columns
Employee_ID Employee ID
Bonus Bonus
“Employee ID” is
the common
column in both
datasets
42

Merging
sal_data <- read.csv("sal_data_R2.csv")
bonus_data <- read.csv("bonus_data_R2.csv")
Join (Merge) two datasets having one or more common columns with identical
names.
merge() function is used to join two data sets. To perform join operations import
the data sets.
43
# Import sal_data and bonus_data

Types of Joins
Consider sal_data =x and bonus_data=y
Left Join Right Join
Inner Join Outer Join
There are 4
types of
Joins
44

Left Join
leftjoin<-merge(sal_data,bonus_data,by="Employee_ID",all.x=TRUE)
leftjoin
Employee_ID First_Name Last_Name Basic_Salary Bonus
1 E-1001 Mahesh Joshi 16070 16070
2 E-1002 Rajesh Kolte 14960 NA
3 E-1004 Priya Jain 13490 13490
4 E-1005 Sneha Joshi 15660 NA
5 E-1007 Ram Kanade 15850 NA
6 E-1008 Nishi Honrao 15880 15880
7 E-1009 Hameed Singh 15120 NA
Left Join returns all rows from the left table, and any rows with matching keys from
the right table.
# Display all the information(including bonus) of Employees from sal_data
To include all rows from the data set x(i.e. first data frame) and only those from y(i.e.
second dataframe) that match, specify all.x= TRUE
by= is used to specify common columns.
45

Right Join
rightjoin<-merge(sal_data,bonus_data,by="Employee_ID",all.y=TRUE)
rightjoin
1 E-1001 Mahesh Joshi 16070 16070
2 E-1004 Priya Jain 13490 13490
3 E-1008 Nishi Honrao 15880 15880
4 E-1003 <NA> <NA> NA 15200
5 E-1006 <NA> <NA> NA 14200
6 E-1010 <NA> <NA> NA 15120
Right Join returns all rows from the right table, and any rows with matching keys
from the left table.
# Display all the information of employees who are receiving bonus
To keep all rows from the data set y and only those from x that match, specify
all.y= TRUE
46

Inner Join
innerjoin<-merge(sal_data,bonus_data,by=("Employee_ID"))
innerjoin
1 E-1001 Mahesh Joshi 16070 16070
2 E-1004 Priya Jain 13490 13490
3 E-1008 Nishi Honrao 15880 15880
Inner Join returns only the rows in which the x have matching keys in the y.
# Display all the information about employees which are common in both
# the tables
47
By default all.x
and all.y is
equal to FALSE

Outer Join
outerjoin<-merge(sal_data,bonus_data,by=c("Employee_ID"),all=TRUE)
outerjoin
1 E-1001 Mahesh Joshi 16860 16070
2 E-1002 Rajesh Kolte 14960 NA
3 E-1004 Priya Jain 12670 13490
4 E-1005 Sneha Joshi 15660 NA
5 E-1007 Ram Kanade 15850 NA
6 E-1008 Nishi Honrao 15950 15880
7 E-1009 Hameed Singh 15120 NA
8 E-1003 <NA> <NA> NA 15200
9 E-1006 <NA> <NA> NA 14200
10 E-1010 <NA> <NA> NA 15120
Outer Join returns all rows from x and y, join records from x which have matching
keys in the y.
merge() merges
only common
rows to both
datasets.
To include all
rows from both
data sets, specify
all= true.
# Combine sal_data and bonus_data
48

Appending Data-
Data Snapshot
basic salary – 1 data has 5 rows
and 6 columns
basic salary – 2 data has 7 rows
and 6 columns
Last_Name Last Name
Grade Grade
Location Location
ba Basic Allowance
ms Management
Supplements
Data Descriptionbasic_salary_2_R2
basic_salary_3_R2
49

Appending Data Sets
salary_1<-read.csv("basic_salary_2_R2.csv",header=TRUE)
salary_2<-read.csv("basic_salary_3_R2.csv",header=TRUE)
rbindsalary<-rbind(salary_1,salary_2)
rbindsalary
# Import the data sets and append them using rbind() function
rbind() combines the
vector, matrix or data
frame by rows
50
• Append means adding cases/observations to a dataset.

SECTION 7
Aggregating Data in R
51

Aggregating Single Variable by Single
Factor
A<-aggregate(ms~Location,data=salary_data,FUN=sum)
A
Location ms
1 DELHI 45430
2 MUMBAI 75620
# Calculate sum of variable ‘ms’ by variable ‘Location’
aggregate() carries out the process of taking numerous records and
collapsing them into a single summary record.
~ is used to give a formula such as y ~ x, where y are the numeric type
variables to be split into groups according to the grouping x variables.
FUN= argument takes the function to compute summary statistics.
aggregate() by default ignores the missing data values.
* 52
Aggregating data means splitting data into subsets, computing summary statistics
on each subset and displaying the results in a conveniently summarised form.

Aggregating Multiple Variables by Single
Factor
> B<-aggregate(cbind(ba,ms)~Location,data=salary_data,FUN=sum)
> B
Location ba ms
1 DELHI 60190 45430
2 MUMBAI 113005 75620
# Calculate sum of variables ‘ba’ and ‘ms’ by variables ‘Location’
and ‘Grade’
cbind() combines vector, matrix or data frame by
columns
When aggregated with cbind(), it ignores the rows corresponding to the row
having missing values.
So, we recommend not to use cbind()(in case of missing values) with aggregate(),
you can aggregate these variables separately.
*
53

Aggregating Single Variable by Multiple
Factors
C<-aggregate(ms~Location+Grade,data=salary_data,FUN=sum)
C
Location Grade ms
1 DELHI GR1 16070
2 MUMBAI GR1 43650
3 DELHI GR2 29360
4 MUMBAI GR2 31970
# Calculate sum of variable ‘ms’ by variables ‘Location’ and ‘Grade’
Multiple factors can be added using ‘+’ operator.
54

Data Management in R

More Related Content

What's hot (20)

Similar to Data Management in R (20)

More from Sankhya_Analytics (7)

Recently uploaded (20)

Data Management in R

Editor's Notes