Ubuntu22.4部署大模型前置安装

真机环境安装

安装Pytorch等依赖

将一下内容放入到requirements.txt

torch==2.4.1
torchaudio==2.4.1
packaging
jinja2
openai-whisper
jsonlines
pandas
validators
sty
transformers
librosa
accelerate
aiohttp
colorama
omegaconf==2.3.0
sox
six==1.16.0
hyperpyyaml
conformer==0.3.2
diffusers
pillow
sentencepiece
easydict
fire
ujson
cairosvg
immutabledict
rich
wget
gdown
datasets
torchdyn==1.0.6
huggingface_hub
loguru
decord
blobfile
timm
sacrebleu==1.5.1
soundfile
tqdm

然后 

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

安装cuda12.1 

Ubuntu22.4部署及更新cuda11.8与cuda12.1-CSDN博客

安装flash_attn

 下载flash_attn依赖(这一步很重要，要不然代码程序无法执行)

pip install flash_attn

如果一直无法安装，可直接下载安装文件，然后再安装

cd /opt/workspace/
wget https://github.moeyy.xyz/https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.4cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
pip install flash_attn-2.7.4.post1+cu12torch2.4cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

以上下载地址可参考(Linux)：Releases · Dao-AILab/flash-attention · GitHub 

windows系统参考：Releases · kingbri1/flash-attention · GitHub