python机器学习人脸识别

COMP24112 - Exercise 3: Face Recognition

1. Task description

You will work on the face recognition and face completion tasks. The provided dataset includes face images of 40 subjects. There are a total of 400 images, with 10 images per subject. Each image contains 32×32=1024 pixels, with 256 grey levels per pixel.
你将学习人脸识别和人脸完成任务。所提供的数据集包括40个受试者的面部图像。总共有400张图片，每个受试者10张。每个图像包含32×32=1024像素，每个像素有256个灰度级。

2. Preparation

Setting up

The functions that you need are defined in the source file we have provided you with. To use them you must import it first:
您需要的函数在我们提供的源文件中定义。要使用它们，你必须先导入它:
from lab3lib import load_data, show_single_face, show_faces, partition_data, split_left_right, join_left_right, show_split_faces
import numpy as np
import matplotlib.pyplot as plt
Now you can load the data:
data, labels = load_data()

Visualising the dataset

You can observe the faces stored in a data matrix by using the functions show_single_face() and show_faces().
图形化数据集,
您可以使用show_single_face()和show_faces()函数来观察存储在数据矩阵中的脸。

Display a single example.

show_single_face(data[0, :])

Display the first 16 examples in a 4x4 grid.

show_faces(data[:16, :], num_per_row=4)

Splitting into the train and test sets

Normalize your pixel values by 1/255 so that the feature values used in later experiments are between 0 and 1. In general, the normalization trick can make it easier for your to tune parameters in machine learning experiments. A data partition function is supplied to facilitate your experiments. It randomly splits the provided data into the training and test sets, where the training set contains $N$ samples per class and the test set contains
the remaining samples. It takes the labels and the specified number $N$ as the input. It returns the sample indices in the training and test set. For example, the following code divides a given set of 400 samples into a training set containing 3 samples per subject and a test set containing 7 samples per subject:
分成训练集和测试集
将像素值归一化1/255，这样以后实验中使用的特征值就在0到1之间。一般来说，归一化技巧可以让你在机器学习实验中更容易调整参数。一个数据划分函数提供方便您的实验。它随机地将提供的数据分成训练集和测试集，其中训练集包含每个类的𝑁样本，测试集包含其余的样本。它以标签和指定的编号𝑁作为输入。它返回训练和测试集中的样本索引。例如，下面的代码将给定的400个样本集分成每个受试者包含3个样本的训练集和每个受试者包含7个样本的测试集:

normalization

data = data / 255

partition

tr_ind, te_ind = partition_data(labels, num_per_class=3)
tr_data = data[tr_ind, :]
tr_label = labels[tr_ind]
te_data = data[te_ind, :]
te_label = labels[te_ind]

3. $L_2$-regularised least squares solution

3.1 Implementation

Now, you will need to implement the function l2_rls_train below. It should train a linear model by minimising the $L_2$-regularised sum of squares loss through zeroing the loss gradient. It should support both single-output and multi-output cases. This function should take a set of training samples and a user-specified regularisation parameter $\lambda$ as the input, and return the predicted weights. When $\lambda =0$, use pseudo inverse to implement the solution.
现在，您需要在下面实现l2_rls_train函数。它应该通过将损失梯度归零来最小化𝐿2 -正则化平方和损失来训练一个线性模型。
它应该同时支持单输出和多输出情况。这个函数应该以一组训练样本和用户指定的正则化参数𝜆作为输入，并返回预测的权重。
当𝜆=0时，使用伪逆来实现解决方案
def l2_rls_train(data, labels, lamda# Add any other arguments here
):
“”"
A summary of your function goes here.

data: type and description of "data"
labels: type and description of "labels"

Returns: type and description of the returned variable(s).
"""

# This is just to be consistent with the lecture notes.
X, y = data, labels

# Expand X with a column of ones.
X_tilde = np.c_[np.ones(X.shape[0]), X]

# Compute the coefficient vector. 根据公式计算即可
if lamda == 0: #如果lamda=0，无约束
    w = np.linalg.pinv(X_tilde) @ y
else:          #L2约束
    w = np.linalg.inv(np.transpose(X_tilde) @ X_tilde + np.dot(lamda, np.identity(X.shape[1] + 1))) @ (np.transpose(X_tilde) @ y)
# Return model parameters.
return w

Now, implement the function l2_rls_predict below. It should take the trained weights and your query data as the input, and return the corresponding prediction.
现在，实现下面的l2_rls_predict函数。它应该以训练过的权重和查询数据作为输入，并返回相应的预测。
def l2_rls_predict(w, data):
“”"
A summary of your function goes here.

data: type and description of "data"
w: type and description of "w"

Returns: type and description of the returned variable(s).
"""
X = data
X_tilde = np.c_[np.ones(X.shape[0]), X]

Compute the prediction.

y = X_tilde @ w
return y

3.2 Experiments

Start to use your implemented functions to perform the following experiments.

Experiment 1: Binary Classification

Start with a binary classification task to classify face images for subjects “1” and “30”.

Train three classifiers:

1. Set $y=-1$ for samples from “1” and $y=+1$ for samples from “30” for training a linear model, and use the threshold $T=0$ to obtain the class prediction.
2. Set $y=0$ for samples from “1” and $y=1$ for samples from “30” for training a linear model, and use the threshold $T=0.5$ to obtain the class prediction.
3. Set $y=1$ for samples from “1” and $y=30$ for samples from “30” for training a linear model, and use the threshold $T=15.5$ to obtain the class prediction.
   Randomly select 3 images per subject for training, and use the remaining images for testing. Train your classifier 1 and test it with a fixed regularisation parameter $\lambda$=0. Repeat this process 50 times and record the training and test errors.

开始使用您实现的函数来执行以下实验。

实验一:二分类
从二值分类任务开始，对受试者“1”和“30”的人脸图像进行分类。

训练三个分类:

1.对“1”的样本设置𝑦=−1，对“30”的样本设置𝑦=+1，训练线性模型，使用阈值𝑇=0获得类预测。
2.设置“1”的样本𝑦=0，“30”的样本𝑦=1训练一个线性模型，使用阈值𝑇=0.5获得类预测。
3.对“1”的样本设置𝑦=1，对“30”的样本设置𝑦=30来训练线性模型，使用阈值𝑇=15.5来获得类预测。
每个受试者随机选择3张图像进行训练，使用剩余的图像进行测试。训练你的分类器1，用一个固定的正则化参数𝜆=0来测试它。重复这个过程50次，并记录训练和测试错误。

Your code goes here

训练函数

def train_fun(sample1, sample2, T):
# 分配训练和测试数据
#选出“1”和“30”样本的数据
tr_labels_all = np.concatenate((labels[0: 10], labels[290: 300]))
tr_data_all = np.concatenate((data[0: 10, : ], data[290: 300, : ]))
#分出训练集和测试集
tr_ind, te_ind = partition_data(tr_labels_all, num_per_class=3)
tr_data = tr_data_all[tr_ind, :]
tr_labels = tr_labels_all[tr_ind]
te_data = tr_data_all[te_ind, :]
te_labels = tr_labels_all[te_ind]
#进行分类
#训练集
tr_labels_new = []
for la in tr_labels:
if (la == 1):
tr_labels_new.append(sample1)
if (la == 30):
tr_labels_new.append(sample2)
#测试集
te_labels_new = []
for la in te_labels:
if (la == 1):
te_labels_new.append(sample1)
if(la == 30):
te_labels_new.append(sample2)
# 训练模型w
w = l2_rls_train(tr_data, tr_labels_new, 0) #𝜆=0
#训练集预测值
tr_predicted = l2_rls_predict(w, tr_data)
#根据阈值进行二分类
tr_predicted_int = []
for i in tr_predicted:
if (i <= T):
tr_predicted_int.append(sample1)
else:
tr_predicted_int.append(sample2)
#计算训练误差
tr_err = 1 - (np.sum(np.asarray(tr_predicted_int) == np.asarray(tr_labels_new)) / 6)
#测试集预测值
te_predicted = l2_rls_predict(w, te_data)
#根据阈值进行二分类
te_predicted_int = []
for i in te_predicted:
if (i <= T):
te_predicted_int.append(sample1)
else:
te_predicted_int.append(sample2)
#计算测试误差
te_err = 1 - (np.sum(np.asarray(te_predicted_int) == np.asarray(te_labels_new)) / 14)
return tr_err, te_err
#进行第一个分类
tr_err1 = []
te_err1 = []
#迭代50次
for i in range(50):
#对“1”的样本设置𝑦=−1，对“30”的样本设置𝑦=+1，训练线性模型，使用阈值𝑇=0获得类预测。
tr_err, te_err = train_fun(-1,1,0)
tr_err1.append(tr_err)
te_err1.append(te_err)
Repeat the same experiment for classifier 2.

Your code goes here

tr_err2 = []
te_err2 = []
#迭代50次
for i in range(50):
#2.设置“1”的样本𝑦=0，“30”的样本𝑦=1训练一个线性模型，使用阈值𝑇=0.5获得类预测。
tr_err, te_err = train_fun(0, 1, 0.5)
tr_err2.append(tr_err)
te_err2.append(te_err)
Repeat the same experiment for classifier 3.

Your code goes here

tr_err3 = []
te_err3 = []
#迭代50次
for i in range(50):
#3.对“1”的样本设置𝑦=1，对“30”的样本设置𝑦=30来训练线性模型，使用阈值𝑇=15.5来获得类预测。
tr_err, te_err = train_fun(1,30, 15.5)
tr_err3.append(tr_err)
te_err3.append(te_err)
Print out the mean and the std values of the test errors of the three classifiers.

Your code goes here

te_err1_mean = np.mean(te_err1)
te_err1_std = np.std(te_err1)
print(‘class1_mean:’, te_err1_mean)
print(‘class1_std:’, te_err1_std)

te_err2_mean = np.mean(te_err2)
te_err2_std = np.std(te_err2)
print(‘class2_mean:’, te_err2_mean)
print(‘class2_std:’, te_err2_std)

te_err3_mean = np.mean(te_err3)
te_err3_std = np.std(te_err3)
print(‘class3_mean:’, te_err3_mean)
print(‘class3_std:’, te_err3_std)

Experiment 2: Multi-class Classification

Now, proceed to the multi-class classification task to classify face images for all the 40 subjects.

Split your data to two sets: the training set contains 5 randomly selected images per subject and the test set contains the remaining images. The training set is used to train the model and select hyper-parameter $\lambda$, while the test set is only for final assessment of the trained classifier with the selected $\lambda$.

Design an appropriate and complete machine learning experiment, which should include the training, hyper-parameter selection and evaluation stages. You can choose from the random subsampling, $k$-fold CV and LOO approaches for hyper-parameter selection. In the evaluation, you should construct a $40\times 40$ classification error matrix (confusion matrix) for test samples.

Write your code below and make sure it compiles.
实验2:多类分类
现在，继续进行多类分类任务，对所有40个被试的人脸图像进行分类。

将数据分成两组:训练集包含每个受试者随机选择的5张图像，测试集包含剩余的图像。训练集用于训练模型并选择超参数𝜆，而测试集仅用于使用选择的𝜆对训练的分类器进行最终评估。

设计一个合适、完整的机器学习实验，包括训练、超参数选择和评估阶段。您可以从随机子抽样中选择，𝑘-fold CV和LOO方法进行超参数选择。在评估中，您应该为测试样本构建一个40×40分类错误矩阵(混淆矩阵)。

编写代码，并确保它能够编译。

Your code goes here

from sklearn.metrics import confusion_matrix
#计算出最合适的超参数𝜆
def lamda_selection(tr_data, tr_label, data_splits):
#设置超参数范围
lamda_range = []
for i in range(-6, 10, 1):
lamda_range.append(10**i)
#分配训练集和测试集
ind = []
for i in range(data_splits):
ind.append(list(partition_data(tr_label, num_per_class=2)))
#存储误差平均值
lamda_errors = []
for lamda in lamda_range:
#每次的试验误差
each_errors = []
for i in range(data_splits):
tr_data_ = tr_data[ind[i][0], :]
te_data_ = tr_data[ind[i][1], :]
te_label_ = tr_label[ind[i][1]]
err = classify_err(tr_data_, te_data_, lamda, -1, 1, 2, 3, te_label_, 40,False)
each_errors.append(err)
lamda_errors.append(np.mean(np.array(each_errors)))
#最合适的lamda是误差最小的
best_lambda = lamda_errors.index(min(lamda_errors))
print('lamda_errors: ', lamda_errors)
return lamda_range[best_lambda]
#soft max用来获得预测类的概率
def softMax(w, X, predict):
X_new = np.append(np.array([1]), X)
exp = np.exp(np.transpose(w) @ X_new)
softmax = exp[predict - 1] / np.sum(exp)
return softmax
#获得训练标签
def train_label(class1, class2, num_per_class, kinds):
tr_label = []
for i in range(kinds):
for j in range(num_per_class):
array = [class1 for k in range(kinds)]
array[i] = class2
tr_label.append(array)
tr_label_array = np.array(tr_label)
return tr_label_array
#计算分类误差
def classify_err(X, te_data, lamda, a, b, num_per_class_tr, num_per_class_te, truth, kinds, flag):
tr_labels = train_label(a, b, num_per_class_tr, 40)
#训练
w = l2_rls_train(X, tr_labels, lamda)
#预测
predictedY = l2_rls_predict(w, te_data)
#输出标签
lb_out = []
total_pic = 40 * num_per_class_te
for j in range(total_pic):
#存储prob
prob = []
for i in range(kinds):
prob.append(softMax(w, te_data[j,:], i+1))
max_prob = max(prob)
#分类结果
result = prob.index(max_prob) + 1
lb_out.append(result)
#存储格式变换
ans = np.array(lb_out)
#如果预测标签和真实标签不一样就记为误差
err = (ans != truth)
every_err = []
err_mean = 0
for i in range(40):
tmp = 0
for j in range(num_per_class_te):
if err[j + i * num_per_class_te]:
tmp += 1
err_mean += 1
every_err.append(tmp/num_per_class_te)
if flag:
print(confusion_matrix(ans, truth))
print(‘every_err:’,every_err)
return err_mean / total_pic #平均误差
#分配训练集和测试集
tr_ind, te_ind = partition_data(labels, num_per_class=5)
tr_label = labels[tr_ind]
tr_data = data[tr_ind, :]
te_data = data[te_ind, :]
te_label = labels[te_ind]
best_lamda = lamda_selection(tr_data, tr_label, 4)
print(‘best_lamda:’, best_lamda)
#分类误差
err = classify_err(tr_data, te_data, best_lamda, -1, 1, 5, 5, te_label, 40, True)
print(‘err_mean:’, err)
Pick some subjects that are considered the easist to recognise by your classifier, and print the test images from these subjects below.

选择一些你的分类器认为最容易识别的主题，并从这些主题中打印测试图像如下。
show_faces(data[10:20, :], num_per_row=5)
show_faces(data[20:30, :], num_per_row=5)

Pick some subjects that are considered the most difficult to recognise by your classifier, and print the misclassified test images from these subjects below.

选择一些你的分类器认为最难识别的主题，并从下面这些主题中打印出分类错误的测试图像。

show_faces(data[0:10, :], num_per_row=5)
show_faces(data[30:40, :], num_per_row=5)

Experiment 3: Face Completion

The task is to predic the 512 pixels of the right face from the 512 pixels of the left face by formulating a regression task.

The following command can be used to extract the left and right faces from the input data.
实验三:脸部完成
该任务是通过制定一个回归任务，从左脸的512个像素预测右脸的512个像素。

下面的命令可以用来从输入数据中提取左面和右面。
left, right = split_left_right(data)
And you can visualise them with the function show_split_faces():
show_split_faces(left[:6, :], num_per_row=3)
Build a face completion model using your functions implemented earlier. Design an experiment to assess your model, which should include appropriate training and testing. Use mean absolute percentage error to evaluate the regression performance. You do not need to bother with hyper-parameter selection, but simply set $\lambda$=0.

Write your code below and make sure it compiles.
使用前面实现的函数构建人脸完成模型。设计一个实验来评估你的模型，这应该包括适当的训练和测试。

使用平均绝对百分比误差来评估回归性能。您不需要使用超参数选择，只需设置𝜆=0即可。

编写下面的代码，并确保它能够编译。

Your code goes here

#分配训练集和测试集
tr_ind3, te_ind3 = partition_data(labels, num_per_class = 5)
tr_data3 = data[tr_ind3, :]
te_data3 = data[te_ind3, :]
#将图像左右分开
tr_left, tr_right = split_left_right(tr_data3)
te_left, te_right = split_left_right(te_data3)
#开始训练模型
w = l2_rls_train(tr_left, tr_right, 0)
#用左边来预测右边
tr_predicted=l2_rls_predict(w, tr_left)
te_predicted = l2_rls_predict(w, te_left)
#计算平均绝对百分比误差
#训练集
sum = 0
for i in range(200):
for j in range(512):
sum += (abs(tr_right[i, j] - tr_predicted[i, j]) / abs(tr_right[i, j]))

平均绝对百分比误差

tr_mape = sum/ (200 * 512)
print('train mean absolute percentage error: ', tr_mape)
#测试集
sum=0
for i in range(200):
for j in range(512):
sum += (abs(te_right[i,j] - te_predicted[i,j])/abs(te_right[i,j]))
#平均绝对百分比误差
te_mape = sum/(200*512)
print('test mean absolute percentage error: ', te_mape)
Visualise and compare the ground truth face (include both the left and right parts) and the completed face (include both the true left and predicted right parts) for 3 randomly chosen testing samples below. You can use join_left_right(left, right) to concatenate the left and right halves of a set of faces.

对下面随机选取的3个测试样本，可视化并比较ground truth face(包括左脸和右脸)和completed face(包括左脸和预测右脸)。

您可以使用join_left_right(left, right)来连接一组面的左右半部分。

Your code goes here

#随机选取的3个测试样本
choice = np.random.choice(np.array(range(200)), size=3, replace=False)
#连接一组面的左右半部分
#预测值
predicted = join_left_right(te_left, te_predicted)
#真值
ground_truth = join_left_right(te_left, te_right)
#显示
for i in range(3):
show_single_face(predicted[choice[i], :])
show_single_face(ground_truth[choice[i], :])

4. Gradient descent for training linear least squares model

Now, implement the function lls_gd_train below. It should train a single-output linear model by minimising the sum of squares loss using the gradient descent approach. It should take the iteration number $N$ and the learning rate $\eta$ as the input. To keep things simple, you can fix the initial guess of the model weights $\mathbf{w}$ as zeros. It should record and return the weights and costs of linear least squares model calculated in all the iterations.
4. 梯度下降法训练线性最小二乘模型
现在，实现下面的lls_gd_train函数。
它应该通过使用梯度下降法最小化平方和损失来训练一个单输出线性模型。
以迭代次数𝑁和学习速率𝜂作为输入。为了保持简单，可以将模型权重𝐰的初始猜测修正为零。
它应该记录并返回在所有迭代中计算的线性最小二乘模型的权值和代价。

def compute_mse(y, x, w):
“”“Function to compute mean square error (MSE).
Args:
y (numpy array): Matrix output of size N x 1.
x (numpy array): Matrix input of size N x D.
w (numpy array): Matrix weight (parameters of the model) of size D x 1.
Returns:
mse (float, scalar) : The mean square error (MSE) for given model w.
“””

error = y - x.dot(w)
mse         = np.inner(error, error)/(2 * y.shape[0]) 
return mse

def compute_gradient_mse(y, x, w):
“”“Function to compute gradient of mse for given model w.
Args:
y (numpy array): Matrix output of size N x 1.
x (numpy array): Matrix input of size N x D.
w (numpy array): Matrix weight (parameters of the model) of size D x 1.
Returns:
gradient (numpy array) : Matrix Gradient of size D x 1.
“””

error = y - x.dot(w)
gradient    = -(np.transpose(x).dot(error)) / (y.shape[0])
return gradient

def lls_gd_train(data, labels, learning_rate, N # Add any other arguments here if needed
):
“”"
A summary of your function goes here.

data: type and description of "data"
labels: type and description of "labels"

Returns: type and description of the returned variable(s).
"""

# This is just to be consistent with the lecture notes.
X, y = data, labels

# Expand X with a column of ones.
Xtilde = np.c_[np.ones(X.shape[0]), X]
# Weight initialization
w = np.zeros(X.shape[1]+1)

cost_all=[]
w_all=[]
# GD update of weights
for i in range(N):
    
    # cost and gradient descent of the linear least squares model
    gd = compute_gradient_mse(y, Xtilde, w)
    cost = compute_mse(y,Xtilde,w)
    
    # Weight update
    w = w - learning_rate * gd
    
    # save w and cost of each iteration in w_all and cost_all 
    cost_all.append(cost)
    w_all.append(w)

# Return model parameters.
return cost_all, w_all

Apply this function to classify images from subject 1 and subject 30. You only need to conduct the experiment for one training-test trial, where 3 images from each subject are used for training and the remaining 7 images for testing.

Set your learning rate small, e.g., $\eta =10^{-3}$, and use a reasonable iteration number, e.g., $N=200$, to train your model. Plot in three figures (1) the change of your sum-of-squares error loss that you minimise, (2) the change of the classification accuracy for training samples, and (3) the change of the classification accuracy for test samples over iterations. Write your code below and make sure it compiles.
应用此函数对受试者1和受试者30的图像进行分类。您只需要进行一次训练测试试验，其中每个受试者的3张图像用于训练，其余7张图像用于测试。

设置你的学习率较小，例如𝜂=10−3，并使用一个合理的迭代数，例如𝑁=200，来训练你的模型。

在三幅图中绘制(1)最小化的平方和误差损失的变化，(2)训练样本分类精度的变化，以及(3)测试样本在迭代过程中分类精度的变化。

编写下面的代码，并确保它能够编译。

Your code goes here

分配训练和测试数据

#选出“1”和“30”样本的数据
tr_labels_all = np.concatenate((labels[0: 10], labels[290: 300]))
tr_data_all = np.concatenate((data[0: 10, : ], data[290: 300, : ]))
#分出训练集和测试集
tr_ind, te_ind = partition_data(tr_labels_all, num_per_class=3)
tr_data = tr_data_all[tr_ind, :]
tr_labels = tr_labels_all[tr_ind]
te_data = tr_data_all[te_ind, :]
te_labels = tr_labels_all[te_ind]
N = 200 #迭代次数

训练模型

cost_all, w_all = lls_gd_train(tr_data, tr_labels, 0.001, N)
#训练集预测值
tr_err = []
te_err = []
for i in range(N):
#训练集预测值
tr_predicted=l2_rls_predict(w_all[i], tr_data)
#根据阈值进行二分类
tr_predicted_int = []
for j in tr_predicted:
if (j <= 15.5):
tr_predicted_int.append(1)
else:
tr_predicted_int.append(30)
#计算训练误差
tr_err.append(1 - (np.sum(np.asarray(tr_predicted_int) == np.asarray(tr_labels)) / 6))
#测试集预测值
te_predicted = l2_rls_predict(w_all[i], te_data)
#根据阈值进行二分类
te_predicted_int = []
for j in te_predicted:
if (j <= 15.5):
te_predicted_int.append(1)
else:
te_predicted_int.append(30)
#计算测试误差
te_err.append(1 - (np.sum(np.asarray(te_predicted_int) == np.asarray(te_labels)) / 14))

plt.title(‘the change of sum-of-squares error loss’)
plt.plot(np.arange(N),cost_all)
plt.xlabel(‘N’)
plt.ylabel(‘error loss’)
plt.show()
plt.title(‘the change of train error’)
plt.plot(np.arange(N),tr_err)
plt.xlabel(‘N’)
plt.ylabel(‘error loss’)
plt.show()
plt.title(‘the change of test error’)
plt.plot(np.arange(N),te_err)
plt.xlabel(‘N’)
plt.ylabel(‘error loss’)
plt.show()
Try a larger learning rate, e.g., $\eta =10^{-2}$, and produce the same three plots by training the model for $N=200$ iterations. Write your code below and make sure it compiles.

尝试一个更大的学习率，例如𝜂=10−2，并通过训练模型𝑁=200次迭代产生相同的三个图。编写下面的代码，并确保它能够编译。

Your code goes here

分配训练和测试数据

#选出“1”和“30”样本的数据
tr_labels_all = np.concatenate((labels[0: 10], labels[290: 300]))
tr_data_all = np.concatenate((data[0: 10, : ], data[290: 300, : ]))
#分出训练集和测试集
tr_ind, te_ind = partition_data(tr_labels_all, num_per_class=3)
tr_data = tr_data_all[tr_ind, :]
tr_labels = tr_labels_all[tr_ind]
te_data = tr_data_all[te_ind, :]
te_labels = tr_labels_all[te_ind]
N = 200 #迭代次数

训练模型

cost_all, w_all = lls_gd_train(tr_data, tr_labels, 0.01, N)
#训练集预测值
tr_err = []
te_err = []
for i in range(N):
#训练集预测值
tr_predicted=l2_rls_predict(w_all[i], tr_data)
#根据阈值进行二分类
tr_predicted_int = []
for j in tr_predicted:
if (j <= 15.5):
tr_predicted_int.append(1)
else:
tr_predicted_int.append(30)
#计算训练误差
tr_err.append(1 - (np.sum(np.asarray(tr_predicted_int) == np.asarray(tr_labels)) / 6))
#测试集预测值
te_predicted = l2_rls_predict(w_all[i], te_data)
#根据阈值进行二分类
te_predicted_int = []
for j in te_predicted:
if (j <= 15.5):
te_predicted_int.append(1)
else:
te_predicted_int.append(30)
#计算测试误差
te_err.append(1 - (np.sum(np.asarray(te_predicted_int) == np.asarray(te_labels)) / 14))

plt.title(‘the change of sum-of-squares error loss’)
plt.plot(np.arange(N),cost_all)
plt.xlabel(‘N’)
plt.ylabel(‘error loss’)
plt.show()
plt.title(‘the change of train error’)
plt.plot(np.arange(N),tr_err)
plt.xlabel(‘N’)
plt.ylabel(‘error loss’)
plt.show()
plt.title(‘the change of test error’)
plt.plot(np.arange(N),te_err)
plt.xlabel(‘N’)
plt.ylabel(‘error loss’)
plt.show()

Remember that all graphs should have axis labels and a title.

5. Reflection and analysis

Finally, observe and reflect on all the results you have obtained in each experiment, and try to interpret them using your machine learning knowledge. Be ready to answer the questions asked by GTA in your face-to-face marking, regarding to your implementation, results, as well as analysis and interpretation of your results.
5. 思考和分析

最后，观察并反思你在每次实验中得到的所有结果，并尝试用你的机器学习知识来解释它们。

准备好回答GTA在你的面对面评分中提出的问题，关于你的执行，结果，以及对你的结果的分析和解释