当前位置: 首页 > news >正文

服务器部署基于Deepseek的检索增强知识库

AI Server Info

2颗(64核128线程主频2.9G,睿频3.5G)
主板:超微X12工作站主板
内存:三星RECC DDR4 32G 3200服务器校验内存 * 4
硬盘:金士顿1T NVME PCIE4.0高速固态
显卡:英伟达(NVIDIA)GeForce RTX 4090 24G * 2

1. Server info

# see if x86_64
uname -m
# see GPU
lspci | grep VGA
# output is NVIDIA GeForce RTX 4090 not AMD GPU
# 31:00.0 VGA compatible controller: NVIDIA Corporation AD102 [GeForce RTX 4090] (rev a1)
#see code name and more, Codename:    noble
cat /etc/os-release
lsb_release -a
hostnamectl

2. anaconda

wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh
bash Anaconda3-2024.10-1-Linux-x86_64.sh
source ~/anaconda3/bin/activate
conda --version
conda update conda

3. ollama

see install doc

# remove first
# sudo rm -rf /usr/lib/ollama
# install auto
curl -fsSL https://ollama.com/install.sh | sh

# or install manual
# using NVIDIA GeForce RTX 4090, no need install ROCm
curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
scp ~/Downloads/ollama-linux-amd64.tgz lwroot0@192.168.0.20:~/instal
# unzip to /usr[/lib/ollama]
sudo tar -C /usr -xzf ollama-linux-amd64.tgz

# start
ollama serve
# statue
ollama -v
Adding Ollama as a startup service

Create a user and group for Ollama:

sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo usermod -a -G ollama $(whoami)

Create a service file in /etc/systemd/system/ollama.service:

[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"
Environment="OLLAMA_MODEL_PATH=/usr/share/ollama/.ollama/models/"
Environment="OLLAMA_HOST=0.0.0.0"

[Install]
WantedBy=default.target

Then start the service:

sudo systemctl daemon-reload
sudo systemctl enable ollama

Add to user env:

vi ~/.bashrc
# add
# export OLLAMA_MODEL_PATH=/usr/share/ollama/.ollama/models/
# export OLLAMA_HOST=0.0.0.0

source ~/.bashrc
echo $OLLAMA_MODEL_PATH
run AI model

You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
see models lib of ollama
model will saved in ~/.ollama/models/ or OLLAMA_MODEL_PATH

模型名称规模代码
deepseek-r114bollama run deepseek-r1:14b
32bollama run deepseek-r1:32b
deepseek-v216bollama run deepseek-v2
qwen2.514bollama run qwen2.5:14b
phi414b onlyollama run phi4
glm49b onlyollama run glm4
llama3.18bollama run llama3.1

4. docker

see doc

# update
sudo apt update
sudo apt upgrade

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# aliyun mirror
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo tee /etc/apt/keyrings/docker.asc
sudo sh -c 'echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker.list'
sudo apt-get update

# install latest version
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# add mirror
sudo vi /etc/docker/daemon.json
{
“registry-mirrors”:[
"https://docker.registry.cyou",
"https://docker-cf.registry.cyou",
"https://dockercf.jsdelivr.fyi",
"https://docker.jsdelivr.fyi",
"https://dockertest.jsdelivr.fyi",
"https://mirror.aliyuncs.com",
"https://dockerproxy.com",
"https://mirror.baidubce.com",
"https://docker.m.daocloud.io",
"https://docker.nju.edu.cn",
"https://docker.mirrors.sjtug.sjtu.edu.cn",
"https://docker.mirrors.ustc.edu.cn",
"https://mirror.iscas.ac.cn",
"https://docker.rainbond.cc"
]
}
MaxKB

模型概况

docker run -d --name=maxkb --restart=always -p 7861:8080 -v ~/.maxkb:/var/lib/postgresql/data -v ~/.python-packages:/opt/maxkb/app/sandbox/python-packages 1panel/maxkb

# test connect to ollama
sudo docker exec -it maxkb bash
curl http://192.168.0.20:11434/
# output Ollama is runningroot@a7c89e320e86

visit: http://your_ip/7861
默认账号信息(首次登录系统强制修改):
username: admin
password: MaxKB@123…

http://www.dtcms.com/a/26062.html

相关文章:

  • 嵌入式硬件篇---常用的汇编语言指令
  • Mini-Omni2
  • java防抖,防止表单重复提交,aop注解形式
  • deepseek帮我设计物理量采集单片机口保护电路方案
  • 什么是Baklib云内容中台?
  • chrome V3插件开发,调用 chrome.action.setIcon,提示路径找不到
  • 操作系统2.4
  • 虚拟环境下Python调用C++开发的dll如何联调?一文解答!(Python开发环境PyCharm,C++开发环境Visual Studio)
  • Python JSON的深度解析:从基础到应用
  • 【用deepseek实现快递查询】快递物流查询-快递查询-快递查询-快递物流查询-快递物流轨迹查询-快递物流查询接口-快递查询-快递物流查询
  • Flask实现高效日志记录模块
  • DeepSeek与ChatGPT:AI语言模型的全面技术解析与对比
  • linux进程间通信的posix消息队列应用记录
  • Go日期时间处理工具Carbon
  • python 如何获取文件的keys
  • 【C语言】C语言 哈夫曼编码传输(源码+数据文件)【独一无二】
  • 2025年人工智能与教育系统国际学术会议(ICAIES 2025)
  • postgres源码学习之简单sql查询
  • Java开发实习面试笔试题(含答案)
  • 【信息系统项目管理师】专业英语重点词汇大汇总
  • 数字电路中的焊接技术
  • MyBatis XML映射文件中的批量插入和更新
  • 模电知识点总结(2)
  • 信息安全管理体系认证:赋能组织应对数字时代的风险与挑战
  • 【含文档+PPT+源码】基于微信小程序的猎兔汽车保养维修美容服务平台的设计与实现
  • INA219电流、电压、功率测量芯片应用
  • Ollama 本地GUI客户端:为DeepSeek用户量身定制的智能模型管理与交互工具
  • 斐波那契数列模型:在动态规划的丝绸之路上追寻斐波那契的足迹(上)
  • 在VS-qt的程序中,后期增加PCH预编译功能,提高编译速度
  • 【练习】【二分】力扣热题100 34. 在排序数组中查找元素的第一个和最后一个位置