08-12 周一 搭建vllm0.5.0和lm-eval评测环境

简介

 本文档主要演示搭建vllm0.5.0的评测环境的过程。这个环境问题，真的很费劲。

使用的镜像

(lmdeploy042) yuzailiang@ubuntu:~$ docker run --name vllm050 --gpus all -v /mnt/self-define/:/mnt/self-define -it 10.101.12.128/schen-zhejianglab.com/vllmcusparselt:1.0-dev-nvidia12.4-cudnn8-jupyter-ssh

注： 挂载共享目录，是为了方便，在共享目录中，可以有一些配置信息，自己常用的，进行保存。如缓存目录。

注： --gpus all 则是为了使用GPU。

 这样，在环境构建过程中，就不用每次下载同样的一个而包，花费很长的等待时间了。如下述的torch包，779MB，可以保证下载一次之后，之后就可以一直使用缓存了。

Collecting torch==2.3.0
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/43/e5/2ddae60ae999b224aceb74490abeb885ee118227f866cb12046f0481d4c9/torch-2.3.0-cp310-cp310-manylinux1_x86_64.whl (779.1 MB)

 同事说，要先提供配置：

export TORCH_CUDA_ARCH_LIST="8.0 8.6 8.9 9.0"

下载源码

root@74d4cc1d5091:/workspace# git clone https://github.com/yanchenmochen/vllm.git
Cloning into 'vllm'...
remote: Enumerating objects: 27420, done.
remote: Counting objects: 100% (9923/9923), done.
remote: Compressing objects: 100% (1072/1072), done.
remote: Total 27420 (delta 9392), reused 8851 (delta 8851), pack-reused 17497
Receiving objects: 100% (27420/27420), 23.84 MiB | 783.00 KiB/s, done.
Resolving deltas: 100% (20780/20780), done.

root@74d4cc1d5091:/workspace# cd vllm/
root@74d4cc1d5091:/workspace/vllm# git checkout v0.5.0
Note: switching to 'v0.5.0'.

vllm v0.5.0执行编译安装

 为了查看源码编译安装的详细过程，因此使用了如下命令

root@74d4cc1d5091:/workspace/vllm#  pip install -e . --verbose -i https://pypi.tuna.tsinghua.edu.cn/simple --cache-dir /mnt/self-define/songquanheng/pip_dir/cache/
Using pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Obtaining file:///workspace/vllm
  Running command pip subprocess to install build dependencies
  Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple

 Collecting cmake>=3.21
    Downloading https://pypi.tuna.tsinghua.edu.cn/packages/69/70/242937601f9ff9e6df4c0587b5a7702be4dbfd33420b409d80e2bccc276a/cmake-3.30.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.9 MB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26.9/26.9 MB 3.5 MB/s eta 0:00:00
  Collecting ninja
    Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6d/92/8d7aebd4430ab5ff65df2bfee6d5745f95c004284db2d8ca76dcbfd9de47/ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 KB 2.7 MB/s eta 0:00:00

 这样可以得到下载缓存。

–verbose，参数为了更详细的打印安装的执行过程。

-i https://pypi.tuna.tsinghua.edu.cn/simple 使用清华源 加速构建。

–cache-dir /mnt/self-define/songquanheng/pip_dir/cache/ 方便下次构建，这样能够将下载缓存起来，提升下载效率。

问题和解决方式

源码安装lm-eval

 安装失败

root@74d4cc1d5091:/mnt/self-define/songquanheng/lm-evaluation-harness# pip install -e .
Obtaining file:///mnt/self-define/songquanheng/lm-evaluation-harness
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Installing collected packages: UNKNOWN
  Running setup.py develop for UNKNOWN
Successfully installed UNKNOWN-0.0.0

 解决方式如下：

root@74d4cc1d5091:/mnt/self-define/songquanheng/lm-evaluation-harness# python -m pip install --upgrade pip       
Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
Collecting pip
  Downloading pip-24.2-py3-none-any.whl (1.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 4.4 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.0.2
    Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'pip'. No files were found to uninstall.
Successfully installed pip-24.2

root@74d4cc1d5091:/mnt/self-define/songquanheng/lm-evaluation-harness# pip install setuptools --upgrade
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (