Doris单价和集群的部署

1 服务器环境准备

我们这里以3台服务器为列

1.1 硬件配置

Centos	7.1及以上
Ubuntu	16.04及以上
java	1.8及以上
GCC	4.8.2及以上

        每台服务器磁盘大小最小50G及时间相差不超或5秒

1.2 环境配置

        1.2.1 修改limits.conf文件

vim /etc/security/limit.conf
#在文件最后添加,*号也要添加
* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535

1.2.2 修改sysctl.conf文件

vim /etc/sysctl.conf
#在文件最后添加
vm.max_map_count = 2000000
vm.swappiness = 0

让该文件永久生效：sysctl -p

1.2.3 关闭swap(交互分区)

# 临时关闭：
swapoff -a
# 或永久关闭, 修改fstab文件, 注册带swap的行
vim /etc/fstab

1.2.4 关闭THP

echo "never" >> /sys/kernel/mm/transparent_hugepage/enabled
echo "never" >> /sys/kernel/mm/transparent_hugepage/defrag

将上面两句命令加入到开机启动中, 防止重启失效

vi /etc/rc.d/rc.local
# 给文件授权
chmod +x /etc/rc.d/rc.local

1.2.5 开发防火墙

如果你是直接关闭防火墙可以跳过该步骤

# 使用root用户, 开放Doris数据库的端口
firewall-cmd --zone=public --add-port=9001/tcp --permanent
firewall-cmd --zone=public --add-port=9020/tcp --permanent
firewall-cmd --zone=public --add-port=9030/tcp --permanent
firewall-cmd --zone=public --add-port=9010/tcp --permanent
firewall-cmd --zone=public --add-port=9060/tcp --permanent
firewall-cmd --zone=public --add-port=8040/tcp --permanent
firewall-cmd --zone=public --add-port=9050/tcp --permanent
firewall-cmd --zone=public --add-port=8060/tcp --permanent
# 参考命令 ,重启防火墙 
关闭防火墙服务：systemctl stop firewalld 
开启防火墙服务：systemctl start firewalld
查看防火墙服务状态：systemctl status firewalld 
自启动防火墙服务：systemctl enable firewalld
查看防火墙端口：firewall-cmd --list-ports

 1.3 安装JAVA JDK

将JDK(版本1.8)压缩包拷贝到 /usr/local目录下(存放位置不强制要求)

# cd 到存放目录
cd /usr/local
#解压
tar -xzvf jdk-8u391-linux-x64.tar.gz

配置java环境

#修改profile文件
vim /etc/profile
#在文件最后添加如下内容
export JAVA_HOME=/usr/local/jdk1.8.0_391
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

#刷新配置文件
source /etc/profile
#验证是否安装成功
java -version

出现上面图片则表示成功

 2 安装Doris数据库

2.1安装数据包

#判断PCU是否支持avx2
lscpu | grep -o 'avx2'
#如果输出为avx2则表示支持，否则表示不支持
#支持avx2下载不带noavx2的如：apache-doris-2.1.8-bin-x64.tar.gz
#不支持avx2下载带noavx2的如：apache-doris-2.1.4-bin-x64-noavx2.tar.gz

doris下载地址：Apache Doris - Download | Easily deploy Doris anywhere - Apache Doris

将安装包上传到对应的目录下,我这里以 /usr/local目录下

# cd到对应的目录
cd /usr/local
# 解压安装包
tar -xzvf apache-doris-2.1.8-bin-x64.tar.gz
#可以修改一个名称(可选)
mv apache-doris-2.1.8-bin-x64 ./doris-2.1.8
#可以配置环境变量 
vim /etc/profile
#将下面内容添加到profile文件的最后
export DORIS_HOME=/usr/local/doris-2.1.8
export PATH=$DORIS_HOME/fe/bin:$DORIS_HOME/be/bin:$PATH

2.2 修改FE与BE的配置参数

2.2.1 修改FE配置参数

#进入doris的根目录
cd /usr/local/doris-2.1.8
#修改fe.conf配置文件
vim fe/conf/fe.conf

CUR_DATE=`date +%Y%m%d-%H%M%S`

# Log dir
LOG_DIR = ${DORIS_HOME}/log #可修改日志文件存放地址

# CMS JAVA OPTS
# JAVA_OPTS="-Dsun.security.krb5.debug=true -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$CUR_DATE"

# G1 JAVA OPTS
JAVA_OPTS="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -Xloggc:$LOG_DIR/fe.gc.log.$CUR_DATE -Dlog4j2.formatMsgNoLookups=true"

# For jdk 9+, this JAVA_OPTS_FOR_JDK_9 will be used as default CMS JVM options
# JAVA_OPTS_FOR_JDK_9="-Dsun.security.krb5.debug=true -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$CUR_DATE:time"

# For jdk 9+, this JAVA_OPTS_FOR_JDK_9 will be used as default G1 JVM options
JAVA_OPTS_FOR_JDK_9="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time -Dlog4j2.formatMsgNoLookups=true"

# For jdk 17+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_17="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -XX:+UseG1GC -Xmx8192m -Xms8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR/ -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time"

##
## the lowercase properties are read by main program.
##

# store metadata, must be created before start FE.
# Default value is ${DORIS_HOME}/doris-meta
# meta_dir = ${DORIS_HOME}/doris-meta

# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers

http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
arrow_flight_sql_port = -1

# Choose one if there are more than one ip except loopback address. 
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.88.101 #本机ip地址
# Advanced configurations 
# log_roll_size_mb = 1024
# INFO, WARN, ERROR, FATAL
sys_log_level = WARN         #可选 ,日志文件级别
# NORMAL, BRIEF, ASYNC
sys_log_mode = BRIEF         #可选 ,日志文件级别
# sys_log_roll_num = 10
# sys_log_verbose_modules = org.apache.doris
# audit_log_dir = $LOG_DIR
# audit_log_modules = slow_query, query
# audit_log_roll_num = 10
# meta_delay_toleration_second = 10
# qe_max_connection = 1024
# qe_query_timeout_second = 300
# qe_slow_log_ms = 5000

#保存退出ESC
:wq

2.2.2 修改be参数

#修改be.conf配置文件
vim be/conf/be.conf

CUR_DATE=`date +%Y%m%d-%H%M%S`

# Log dir
LOG_DIR="${DORIS_HOME}/log/"  #日志存放目录,可修改

# For jdk 8 根据实际可以是不是需要添加内存大小
JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx8192m -DlogPath=$LOG_DIR/jni.log -Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives  -Darrow.enable_null_check_for_get=false"

# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Dfile.encoding=UTF-8 -Xmx2048m -DlogPath=$DORIS_HOME/log/jni.log -Xlog:gc:$LOG_DIR/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives --add-opens=java.base/java.nio=ALL-UNNAMED  -Darrow.enable_null_check_for_get=false"

# For jdk 17+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_17="-Dfile.encoding=UTF-8 -Xmx2048m -DlogPath=$LOG_DIR/jni.log -Xlog:gc:$LOG_DIR/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED  -Darrow.enable_null_check_for_get=false"

# since 1.2, the JAVA_HOME need to be set to run BE process.
# JAVA_HOME=/path/to/jdk/

# https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
# https://jemalloc.net/jemalloc.3.html
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:5000,dirty_decay_ms:5000,oversize_threshold:0,prof:true,prof_active:false,lg_prof_interval:-1"
JEMALLOC_PROF_PRFIX="jemalloc_heap_profile_"

# ports for admin, web, heartbeat service 
be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060
arrow_flight_sql_port = -1

# HTTPS configures
enable_https = false
# path of certificate in PEM format.
ssl_certificate_path = "$DORIS_HOME/conf/cert.pem"
# path of private key in PEM format.
ssl_private_key_path = "$DORIS_HOME/conf/key.pem"

# Choose one if there are more than one ip except loopback address. 
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.88.101 #ip地址
# data root path, separate by ';'
# You can specify the storage type for each root path, HDD (cold data) or SSD (hot data)
# eg:
# storage_root_path = /home/disk1/doris;/home/disk2/doris;/home/disk2/doris
# storage_root_path = /home/disk1/doris,medium:SSD;/home/disk2/doris,medium:SSD;/home/disk2/doris,medium:HDD
# /home/disk2/doris,medium:HDD(default)
# 
# you also can specify the properties by setting '<property>:<value>', separate by ','
# property 'medium' has a higher priority than the extension of path
#
# Default value is ${DORIS_HOME}/storage, you should create it by hand.
# storage_root_path = ${DORIS_HOME}/storage

# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers

# Advanced configurations
# INFO, WARNING, ERROR, FATAL
sys_log_level = WARNING  #日志级别可选择
# sys_log_roll_mode = SIZE-MB-1024
# sys_log_roll_num = 10
# sys_log_verbose_modules = *
# log_buffer_level = -1
# palo_cgroups 

# aws sdk log level
#    Off = 0,
#    Fatal = 1,
#    Error = 2,
#    Warn = 3,
#    Info = 4,
#    Debug = 5,
#    Trace = 6
# Default to turn off aws sdk log, because aws sdk errors that need to be cared will be output through Doris logs
aws_log_level=0
## If you are not running in aws cloud, you can disable EC2 metadata
AWS_EC2_METADATA_DISABLED=true
string_type_length_soft_limit_bytes = 10485760  #修改单字段数据大小为10M,可选

2.3 主节点启动

2.3.1 启动fe

#没有配doris环境, 在doris根目录下
fe/bin/start_fe.sh --daemon
be/bin/start_be.sh --daemon
#如果配置了doris环境 ,二选一执行即可
start_fe.sh --daemon
start_be.sh --daemon

#查看是否启动成功
jps

如果出现这个图片的内容,则代表成功。

可以前往游览器查看：

#FE的web端,默认用户root,密码空

http://IP:8030

#BE的web端

http://IP:8040/

通过FE、BE的网址查看，如果正常，则安装成功。

2.4 数据库初始化

#如果服务器安装有mysql,可以mysql客服端连接FE
mysql -hhadoop102 -P9030 -uroot
#设置密码
SET PASSWORD FOR 'root' = PASSWORD('密码');


#如果没有客户端,可以通过DBeaver连接,端口为9030

#配置高可用
#扩容FE节点，新节点添加为follower
ALTER SYSTEM ADD FOLLOWER "192.168.88.102:9010";
#或新节点添加为observer
ALTER SYSTEM ADD OBSERVER "192.168.88.103:9010";


#将BE关联到FE,如果只添加一个本机的BE,那么只就是个单价
ALTER SYSTEM ADD BACKEND "192.168.88.101:9050";
ALTER SYSTEM ADD BACKEND "192.168.88.102:9050";
ALTER SYSTEM ADD BACKEND "192.168.88.103:9050";

#修改密码
alter user 'root' identified by '密码';

下面配置可选

开启 SQL 跟踪日志

set global enable_profile=true;

#查看跟踪是否开启

show variables like '%enable_profile%';

#关闭跟踪日志

set global enable_profile = false;

注：为避免影响性能日常运行中可以关闭跟踪日志，查问题时再打开。

​​​​​​​开启审计日志

set global enable_audit_plugin = true;

其它命令

#查看审计日志是否开启

show variables like '%enable_audit_plugin%';

#关闭审计日志

set global enable_audit_plugin = false;

注：为避免影响性能日常运行中可以关闭审计日志，查问题时再打开。

3 配置高可用

#将主节点的be和fe分发其他两个节点
# 1.先到另外两个节点创建对应目录
mkdir -r /usr/local/doris-2.1.8.1
# 2.把主节点配置分发给另外两个节点
rsync -avz /usr/local/doris-2.1.8.1/* root@192.168.88.102:/usr/local/doris-2.1.8
rsync -avz /usr/local/doris-2.1.8.1/* root@192.168.88.103:/usr/local/doris-2.1.8

 分发完成后到另外两个节点的doris根目录下修改各自的ip地址

vim fe/conf/fe.conf
vim be/conf/be.conf
#修改如下内容, 修改为本机对应的ip
priority_networks = 192.168.88.102

#删除doris的fe目录下的元数据
rm -rf doris-meta

另一个节点与上面步骤一致。

第一次启动时候需要在两个从节点执行该命令

#进入Doris的根目录, 第一次启动FE执行下面命令 主节点ip地址
fe/bin/start_fe.sh --helper 192.168.88.101:9010 --daemon

#进入Doris的根目录,启动BE
be/bin/start_be.sh --daemon

如果状态都为true则代表成功。