卷积神经网络图像分类与迁移学习

# 卷积神经网络图像分类与迁移学习 ## 1. 构建猫狗图像分类的卷积神经网络 ### 1.1 准备工作 当输入为特定值时，全连接层中的某些值会被突出显示。在了解了卷积神经网络（CNNs）的工作原理以及滤波器在其中的作用后，我们将应用这些知识来对猫和狗的图像进行分类。 首先，我们需要导入必要的包： ```python import torchvision import torch.nn as nn import torch import torch.nn.functional as F from torchvision import transforms,models,datasets from PIL import Image from torch import optim device = 'cuda' if torch.cuda.is_available() else 'cpu' import cv2, glob, numpy as np, pandas as pd import matplotlib.pyplot as plt %matplotlib inline from glob import glob !pip install torch_summary ``` ### 1.2 下载数据集 下载数据集的步骤如下： 1. 上传Kaggle认证文件： ```python !pip install -q aggle from google.colab import files files.upload() ``` 2. 移动到Kaggle文件夹并复制`kaggle.json`文件： ```python !mkdir -p ~/.kaggle !cp kaggle.json ~/.kaggle/ !ls ~/.kaggle !chmod 600 /root/.kaggle/kaggle.json ``` 3. 下载猫狗数据集并解压： ```python !kaggle datasets download -d tongpython/cat-and-dog !unzip cat-and-dog.zip ``` ### 1.3 提供训练和测试数据集文件夹 ```python train_data_dir = '/content/training_set/training_set' test_data_dir = '/content/test_set/test_set' ``` ### 1.4 构建数据加载类 我们需要构建一个类来从上述文件夹中获取数据，并为狗图像提供标签1，为猫图像提供标签0。同时，确保获取的图像被归一化到0到1的范围，并将通道维度放在前面。 ```python from torch.utils.data import DataLoader, Dataset class cats_dogs(Dataset): def __init__(self, folder): cats = glob(folder+'/cats/*.jpg') dogs = glob(folder+'/dogs/*.jpg') self.fpaths = cats + dogs from random import shuffle, seed; seed(10); shuffle(self.fpaths) self.targets=[fpath.split('/')[-1].startswith('dog') for fpath in self.fpaths] # dog=1 def __len__(self): return len(self.fpaths) def __getitem__(self, ix): f = self.fpaths[ix] target = self.targets[ix] im = (cv2.imread(f)[:,:,::-1]) im = cv2.resize(im, (224,224)) return torch.tensor(im/255).permute(2,0,1).to(device).float(), torch.tensor([target]).float().to(device) ``` ### 1.5 检查随机图像 ```python data = cats_dogs(train_data_dir) im, label = data[200] plt.imshow(im.permute(1,2,0).cpu()) print(label) ``` ### 1.6 定义模型、损失函数和优化器 1. 定义卷积层函数： ```python def conv_layer(ni,no,kernel_size,stride=1): return nn.Sequential( nn.Conv2d(ni, no, kernel_size, stride), nn.ReLU(), nn.BatchNorm2d(no), nn.MaxPool2d(2) ) ``` 2. 定义获取模型的函数： ```python def get_model(): model = nn.Sequential( conv_layer(3, 64, 3), conv_layer(64, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), nn.Flatten(), nn.Linear(512, 1), nn.Sigmoid(), ).to(device) loss_fn = nn.BCELoss() optimizer=torch.optim.Adam(model.parameters(), lr= 1e-3) return model, loss_fn, optimizer ``` 3. 获取模型、损失函数和优化器，并总结模型： ```python from torchsummary import summary model, loss_fn, optimizer = get_model() summary(model, torch.zeros(1,3, 224, 224)); ``` ### 1.7 创建数据加载器 ```python def get_data(): train = cats_dogs(train_data_dir) ```