网站规划与建设大作业谷歌流量代理代理
ClickHouse安装
版本要求
23.8及之后的版本
硬件要求
三台机器
建议配置
- 磁盘 ssd 500G
- 内存 32g
- cpu 16c
最低配置
- 磁盘 机械硬盘 50G
- 内存 4g
- cpu 4c
容量规划
一亿条数据大约使用1TB磁盘容量
参考官方容量推荐
安装包准备
zookeeper安装
zookeeper需要java启动，8版本以上，三台机器都需要执行
解压安装：
yum install -y javatar -zxvf apache-zookeeper-3.5.9-bin.tar.gz -C /app
配置环境变量：
vi /etc/profile
添加我们需要的配置信息：
ZOOKEEPER_HOME 为你安装的zookeeper目录
export ZOOKEEPER_HOME=/app/apache-zookeeper-3.9.3-bin
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH
检测生效
source /etc/profile
echo $ZOOKEEPER_HOME
配置zookeeper
创建data目录
mkdir /app/apache-zookeeper-3.9.3-bin/data
修改配置
cd /app/apache-zookeeper-3.9.3-bin/conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/app/apache-zookeeper-3.9.3-bin/data
clientPort=2181server.1=ip1:2888:3888
server.2=ip2:2888:3888
server.3=ip3:2888:3888
为每台机器配置节点id
在机器一上执行：
echo 1 >/app/apache-zookeeper-3.9.3-bin/data/myid
在机器二上执行：
echo 2 >/app/apache-zookeeper-3.9.3-bin/data/myid
在机器三上执行：
echo 3 >/app/apache-zookeeper-3.9.3-bin/data/myid
启动zookeeper
cd /app/apache-zookeeper-3.9.3-bin/bin
sh zkServer.sh start
验证
启动成功后可以随便选择一个节点尝试连接另一个节点验证：
sh zkCli.sh -server ip:2181
clickhouse安装
检查cpu指令集是否支持
执行grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"，返回SSE 4.2 supported
表示检查通过
数据盘挂载
2.1.挂载目录
:::info
clickhouse数据库数据存储目录建议使用挂载目录
挂载目录需要客户提供并完成挂载
示例挂载命令：mount /dev/sdb1 /app
:::
2.2.创建数据目录
mkdir -p /app/clickhouse/data
mkdir -p /app/clickhouse/tmp
2.3.授予目录权限
chown -R clickhouse:clickhouse /app/clickhouse
chmod 755 /app/clickhouse
下载安装包
http://192.168.0.107/clickhouse/rpm
修改挂载目录（可选，无挂载目录可不用）
修改数据目录
vim /etc/clickhouse-server/config.xml
路径改为挂载路径即可
6.1 用户管理配置
- 修改default用户密码为econage123，全部权限，只允许本地登录
- 添加用户user，密码为econage123，读写权限，bpi服务使用
- 添加管理用户admin，密码为econage123，读写权限，ddl权限
- 添加只读用户read，密码为econage123
添加/etc/clickhouse-server/users.d/users.xml，自行修改相关密码
<?xml version="1.0"?>
<clickhouse><!-- See also the files in users.d directory where the settings can be overridden. --><!-- Profiles of settings. --><profiles><!-- Default settings. --><default><max_bytes_before_external_group_by>268435456</max_bytes_before_external_group_by><max_bytes_before_external_sort>268435456</max_bytes_before_external_sort><max_download_threads>1</max_download_threads><input_format_parallel_parsing>0</input_format_parallel_parsing><output_format_parallel_formatting>0</output_format_parallel_formatting><optimize_trivial_insert_select>1</optimize_trivial_insert_select></default><!-- Profile that allows only read queries. --><read><readonly>1</readonly><max_bytes_before_external_group_by>268435456</max_bytes_before_external_group_by><max_bytes_before_external_sort>268435456</max_bytes_before_external_sort><max_download_threads>1</max_download_threads><input_format_parallel_parsing>0</input_format_parallel_parsing><output_format_parallel_formatting>0</output_format_parallel_formatting><optimize_trivial_insert_select>1</optimize_trivial_insert_select></read><write><readonly>0</readonly><allow_ddl>0</allow_ddl><max_bytes_before_external_group_by>268435456</max_bytes_before_external_group_by><max_bytes_before_external_sort>268435456</max_bytes_before_external_sort><max_download_threads>1</max_download_threads><input_format_parallel_parsing>0</input_format_parallel_parsing><output_format_parallel_formatting>0</output_format_parallel_formatting><optimize_trivial_insert_select>1</optimize_trivial_insert_select></write></profiles><!-- Users and ACL. --><users><read><password>econage123</password><networks><ip>::/0</ip></networks><profile>read</profile><quota>default</quota></read><admin><password>econage123</password><networks><ip>::/0</ip></networks><profile>default</profile><quota>default</quota></admin><user><password>econage123</password><networks><ip>::/0</ip></networks><profile>write</profile><quota>default</quota></user><!-- If user name was not specified, 'default' user is used. --><default><password>econage123</password><networks><ip>::1</ip></networks><!-- Settings profile for user. --><profile>default</profile><!-- Quota for user. --><quota>default</quota><!-- User can create other users and grant rights to them. --><access_management>1</access_management></default></users><!-- Quotas. --><quotas><!-- Name of quota. --><default><!-- Limits for time interval. You could specify many intervals with different limits. --><interval><!-- Length of interval. --><duration>3600</duration><!-- No limits. Just calculate resource usage for time interval. --><queries>0</queries><errors>0</errors><result_rows>0</result_rows><read_rows>0</read_rows><execution_time>0</execution_time></interval></default></quotas></clickhouse>启动服务
systemctl start clickhouse-server
或者
clickhouse start
校验服务
curl localhost:8123
#返回'Ok.'则服务启动正常
clickhouse-client --user=user --password=econage123 --host=192.168.7.18 --query="select version()"
#返回版本号则用户创建正常clickhouse-client --user=admin --password=econage123 --host=192.168.7.18 --query="select version()"
#返回版本号则用户创建正常
clickhosue集群版本配置
三台机器都需要执行上述clickouse安装步骤
添加zookeeper配置
vim /etc/clickhouse-server/config.xml
host是zookeeper的三个地址
port默认2181
修改配置<zookeeper><node index="1"><host>192.168.7.141</host><port>2181</port></node><node index="2"><host>192.168.7.142</host><port>2181</port></node><node index="3"><host>192.168.7.143</host><port>2181</port></node></zookeeper>
配置分片
host是clickhouse的三个地址
port默认9000
<replica><host>192.168.7.141</host><port>9000</port></replica><replica><host>192.168.7.142</host><port>9000</port></replica><replica><host>192.168.7.143</host><port>9000</port></replica>
配置macros
第一台机器
<macros><shard>01</shard><replica>01</replica></macros>
第二台机器
<macros><shard>01</shard><replica>02</replica></macros>
第三台机器
<macros><shard>01</shard><replica>03</replica></macros>
启动服务
systemctl start clickhouse-server
或者
clickhouse start
验证集群同步
登录任意一台机器
clickhouse-client --user admin --password econage123
select * from system.clusters;
集群名称default
创建 ReplicatedMergeTree 测试表：
任选一台机器，创建一个 ReplicatedMergeTree 引擎的测试表，测试 ZooKeeper 同步功能
CREATE TABLE test_ck ON CLUSTER default (EventDate DateTime, Number UInt32, id UInt32 )ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/test_ck', '{replica}') PARTITION BY toYYYYMM(EventDate) ORDER BY (Number, EventDate, intHash32(id)) SAMPLE BY intHash32(id);
在其他机器节点show tables查看表结构是否同步成功;
创建 Distributed 引擎测试表：
创建一个分布式测试表测试数据分片是否正常。因为已经配置了zookeeper，所以创建表的DDL语句也会同步到其他节点上。
CREATE TABLE dis_test ON CLUSTER default AS test_ck
ENGINE = Distributed(default, default, test_ck, rand())
show table
文件清单
- clickhouse-common-static-*.rpm
- clickhouse-server-*.rpm
- clickhouse-client-*.rpm
- /etc/clickhouse-server/users.d/users.xml
- /etc/clickhouse-server/config.d/config.xml
clickhouse目录说明
- 数据文件目录 /var/lib/clickhouse/
- 日志文件目录 /var/log/clickhouse-server/
可视化管理工具
- dbeaver
- http://localhost:8123/play
- 更多参考 https://clickhouse.com/docs/zh/interfaces/third-party/gui