数据分析处理库-Pandas基础-下篇

数据分析处理库-Pandas基础-下篇

1 SAC*Pandas 分组模式及其对象

• SAC 指的是分组操作中的split-apply-combine过程，其中
  • split 基于某一些规则，将数据拆成若干组
  • apply 对每一组独立地使用函数
  • combine 指将每一组的结果组合成某一类数据结构

1.1groupby对象

• 学生分组

import pandas as pd
df = pd.read_csv('./data/learn_pandas.csv')
df.head(4)

df.groupby(['School','Gender'])['Height'].mean()

School                         Gender
Fudan University               Female    158.776923
                               Male      174.212500
Peking University              Female    158.666667
                               Male      172.030000
Shanghai Jiao Tong University  Female    159.122500
                               Male      176.760000
Tsinghua University            Female    159.753333
                               Male      171.638889
Name: Height, dtype: float64

df.groupby(['School','Grade','Gender'])['Gender'].count()
#根据学校和年纪及性别,统计男女人数

School                         Grade      Gender
Fudan University               Freshman   Female     8
                                          Male       1
                               Junior     Female     9
                                          Male       3
                               Senior     Female     9
                                          Male       2
                               Sophomore  Female     4
                                          Male       4
Peking University              Freshman   Female     7
                                          Male       6
                               Junior     Female     6
                                          Male       2
                               Senior     Female     5
                                          Male       3
                               Sophomore  Female     4
                                          Male       1
Shanghai Jiao Tong University  Freshman   Female     8
                                          Male       5
                               Junior     Female    15
                                          Male       2
                               Senior     Female    14
                                          Male       8
                               Sophomore  Female     4
                                          Male       1
Tsinghua University            Freshman   Female    17
                               Junior     Female    13
                                          Male       9
                               Senior     Female    10
                                          Male       4
                               Sophomore  Female     8
                                          Male       8
Name: Gender, dtype: int64

#根据学生的身高是否超过总体的均值来分组,最后计算体重的平均值
condition = df['Height'] > df["Height"].mean()
#身高超过均值True
#身高没有超过均值False
df.groupby(condition)['Weight'].mean()

Height
False    47.672414
True     66.684932
Name: Weight, dtype: float64

df .groupby([ 'School','Gender' ]).agg(
    {
   
   "Height":[
        ('height_max',"max"),
        ('height_min','min'),
        ("height_mean","mean")
    ],
"Weight" :[
        ("weight_ptp",lambda x:x.max()-x.min()),
        ("weight_median" , "median")
    ]
    }
)

• 北京不同aqiInfo下,北京的白天最高温度,均值,白天最低温迪,夜晚最高,夜晚最低,均值,以及aqiInfo的数量,aqi的,最大值,最小值,均值,空气质量指数大于50的天数有多少

import pandas as pd
import numpy as np
data = pd.read_csv('./data/beijing_tianqi_2018.csv')
data.head(4)

data['bWendu'] = data['bWendu'].str.replace("℃",'').astype(np.int0)
data['yWendu'] = data['yWendu'].str.replace("℃",'').astype(np.int0)

df = data.groupby('aqiInfo').agg(
    {
   
   
        "bWendu":[
            ("bwendu_max","max"),
            ('bwendu_min','min'),
            ('bwendu_mean',"mean")
        ],
        "yWendu":[
            ("ywendu_max","max"),
            ('ywendu_min','min'),
            ('ywendu_mean',"mean")
        ],
        'aqiInfo':[
            ("aqi_info_num",'count')
        ],
        'aqi':[
            ('aqi_min','min'),
            ('aqi_max','max'),
            ('aqi_mean','mean'),
            ('aqi_50',lambda x: sum(x>50))
        ]
    }
)

• set_index(‘label’,inplace=True) # 设置索引
• reset_index(inplace=True) # 取消索引

df.reset_index(inplace=True)
df

2 数据合并相关

2.1 concat函数

• 不加参数直接合并,默认是按照轴0进行合并,不会忽略索引,缺失值用NaN填充
• ignore_index= True, 忽略索引
• axis=0 ,指定合并的方向
• join=‘outer’ # 取并集,如果设置为"inner"则取交集
• keys() # 连接的时候,加上一个层次的Key,用于判定数据来源于那个表

df1 = pd.DataFrame({
   
   'A': ['A0', 'A1', 'A2', 'A3'],
'B': ['B0', 'B1', 'B2', 'B3'],
'C': ['C0', 'C1', 'C2', 'C3'],
'D': ['D0', 'D1', 'D2', 'D3'],
'E': ['E0', 'E1', 'E2', 'E3']})
df2 = pd.DataFrame({
   
   'A': ['A4', 'A5', 'A6', 'A7'],
'B': ['B4', 'B5', 'B6', 'B7'],
'C': [