当前位置： 首页 > news >正文

【Sharding-JDBC】​Spring/Spring Boot 集成 Sharding-JDBC，分表策略与 API、YAML 配置实践​

news 2025/9/7 19:29:51

在这里插入图片描述

文章目录

  • 环境准备
  • Spring框架
    • Sharding-JDBC 4.x版本api实现
    • Sharding-JDBC 5.4.x版本yaml实现
  • Springboot框架
    • Sharding-JDBC 5.4.x版本yaml实现
  • 分库、加密、读写分离基于yaml的配置示例

更多相关内容可查看

需求：按月分区，按年分表，找不到对应年份表走备份表,动态从linux获取账户密码用于创建数据库链接

备份表：table
已经提前建表语句创建好未来年份的表:table_2026,table_2027

环境准备

maven
此依赖会引入很多有漏洞的包，如果有漏洞扫描的情况下，请替换相关包的其他版本或加白名单

 <dependency><groupId>org.apache.shardingsphere</groupId><artifactId>shardingsphere-jdbc-core</artifactId><version>${sharding-jdbc.version}</version></dependency>

Spring框架

Sharding-JDBC 4.x版本api实现

注意：4.x版本，在用api实现，如果想要非分片表走相同的数据源的情况下
需要用setDefaultdatasource赋默认数据源，在5.x版本已经移除了该方法，所以仅支持低版本实现，如果不加的话shading会无法路由其他表，会报表不存在的情况

datasource

    @Bean(name = "dataSource")public DataSource shardingDataSource() throws SQLException, HKEException {DataSource dataSource = YamlShardingSphereDataSourceFactory.createDataSource();// 1. 配置数据源Map<String, DataSource> dataSourceMap = new HashMap<>();dataSourceMap.put("ds", dataSource());// 2. 配置分片规则ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();//        shardingRuleConfig.setDefaultDatabaseShardingStrategy(new NoneShardingStrategyConfiguration());// 配置table表的分片规则ShardingTableRuleConfiguration signInfoRule = new ShardingTableRuleConfiguration("table","ds.table_2025"  // 包含备份表和未来几年的表);signInfoRule.setTableShardingStrategy(new StandardShardingStrategyConfiguration("operate_time",  // 分片字段"year-sharding-algorithm"  // 自定义分片算法));
//        signInfoRule.setKeyGenerateStrategy(
//                new KeyGenerateStrategyConfiguration("row_id", "snowflake")
//        );// 添加其他表的规则（tablexxx）// 添加表规则到分片配置shardingRuleConfig.getTables().add(signInfoRule);// 3. 配置分片算法shardingRuleConfig.getShardingAlgorithms().put("year-sharding-algorithm",new AlgorithmConfiguration("CLASS_BASED", new Properties() {{setProperty("strategy", "STANDARD");setProperty("algorithmClassName", "com.package.YearShardingAlgorithm");}}));// 4. 配置主键生成器shardingRuleConfig.getKeyGenerators().put("snowflake",new AlgorithmConfiguration("SNOWFLAKE", new Properties()));// 5. 创建ShardingSphere数据源Properties props = new Properties();props.setProperty("sql-show", "true");  // 显示SQL日志return ShardingSphereDataSourceFactory.createDataSource(dataSourceMap,Collections.singleton(shardingRuleConfig),props);}

分表算法实现

注意：implements 哪个类取决于你需要用那个对应的算法，有stand，complex等

package cfca.hke.privatization.sharding;import cfca.hke.privatization.common.logging.LoggerManager;
import org.apache.shardingsphere.sharding.api.sharding.standard.PreciseShardingValue;
import org.apache.shardingsphere.sharding.api.sharding.standard.RangeShardingValue;
import org.apache.shardingsphere.sharding.api.sharding.standard.StandardShardingAlgorithm;import java.time.LocalDateTime;
import java.util.Collection;
import java.util.Collections;public class YearShardingAlgorithm implements StandardShardingAlgorithm<LocalDateTime> {@Overridepublic String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<LocalDateTime> shardingValue) {// 获取逻辑表名String logicTableName = shardingValue.getLogicTableName();Object value = shardingValue.getValue();String year = value.toString();//年份格式会有多种 所以直接截取这里if (year.length() > 4) {year = year.substring(0, 4);}//25年的存到备份表logicTableNameif (!"2025".equals(year)){logicTableName = logicTableName + "_" + year;}// 检查表是否存在if (availableTargetNames.contains(logicTableName)) {return logicTableName;} else {LoggerManager.exceptionLogger.error("The table does not exist,please check HKEPrivateConfig/sharding.yaml, tableName: {}", logicTableName);return logicTableName;}}@Overridepublic Collection<String> doSharding(Collection<String> collection, RangeShardingValue<LocalDateTime> rangeShardingValue) {return Collections.emptyList();}
}

Sharding-JDBC 5.4.x版本yaml实现

此版本有漏洞，当你配置不分表的表时候，配置的表明会区分大小写，此漏洞在5.4.1版本已经兼容了，所以可以尽量用最新版

注意：ds.*不会影响你加分片规则的表，优先级是如果有分片规则则认为是需分片的表，这里配单表不配ds.*的时候，必须升级到5.4.1版本，或未来出现新版之后，尽量使用高版本

mode:
#设置为​​单机模式​​。此模式下，配置信息存储在本地，适用于开发测试或小型应用，无需额外的协调服务（如 ZooKeeper）type: Standalonerepository:#指定使用内嵌的 JDBC 数据库来存储 ShardingSphere 的元数据（如逻辑库、表结构等）type: JDBC
# 数据库标识 随便写
databaseName: zhongxin
# 数据库配置，连接池配置
dataSources:ds:dataSourceClassName: com.zaxxer.hikari.HikariDataSourcedriverClassName: com.mysql.cj.jdbc.Driverurl: jdbc:mysql://192.168.xx.xx:3306/zhongxin_new?rewriteBatchedStatements=true&allowMultiQueries=true&useLocalSessionState=true&useUnicode=true&characterEncoding=utf-8&socketTimeout=3000000&connectTimeout=60000username: rootpassword:connectionTimeout: 50000minimumIdle: 5maximumPoolSize: 10idleTimeout: 600000maxLifetime: 1800000connectionTestQuery: SELECT 1
# 5.x版本的配法
rules:
#分片规则标签
- !SHARDING
#需要进行分片的​​逻辑表​​及其规则tables:table:#分片的表名actualDataNodes: ds.table,ds.table_$->{2026..2028}tableStrategy:standard:#分片字段shardingColumn: operate_time#分片算法shardingAlgorithmName: infoAlgorithmtable1:actualDataNodes: ds.table1,ds.table1_$->{2026..2028}tableStrategy:standard:shardingColumn: operate_timeshardingAlgorithmName: infoAlgorithmtable2:actualDataNodes: ds.table2,ds.table2_$->{2026..2028}tableStrategy:standard:shardingColumn: operate_timeshardingAlgorithmName: infoAlgorithmshardingAlgorithms:infoAlgorithm:#分片算法类型为​​基于自定义类type: CLASS_BASEDprops:#指定自定义算法实现的​​策略类型​​为 STANDARD（标准）strategy: STANDARD#自定义分片算法的完整类名algorithmClassName: com.package.YearShardingAlgorithmkeyGenerators:snowflake:type: SNOWFLAKE
# 不分区的表
- !SINGLEtables:- ds.*props:
#​​是否在日志中打印 SQL​​sql-show: true

动态修改账号密码
注意：这里要用sharding提供的解析yaml的方法去解析，如果你是低版本可以自己去写流读取，高版本在读取sharding特殊的配置会报错，在解析完后也要用sharding提供的方法转回string或者bytes

    @Bean(name = "dataSource")public DataSource shardingDataSource() throws Exception {//获取用户名密码String username = credentials[0];String password = credentials[1];File yamlFile = new File("./xxx/sharding.yaml");YamlJDBCConfiguration rootConfig = YamlEngine.unmarshal(yamlFile, YamlJDBCConfiguration.class);Map<String, Object> dsMap = rootConfig.getDataSources().get("ds");dsMap.put("username",username);dsMap.put("password",password);String marshal = YamlEngine.marshal(rootConfig);DataSource dataSource = YamlShardingSphereDataSourceFactory.createDataSource(marshal.getBytes());printDBInfo(username,password);return dataSource;}

分表算法实现

注意：implements 哪个类取决于你需要用那个对应的算法，有stand，complex等

package cfca.hke.privatization.sharding;import cfca.hke.privatization.common.logging.LoggerManager;
import org.apache.shardingsphere.sharding.api.sharding.standard.PreciseShardingValue;
import org.apache.shardingsphere.sharding.api.sharding.standard.RangeShardingValue;
import org.apache.shardingsphere.sharding.api.sharding.standard.StandardShardingAlgorithm;import java.time.LocalDateTime;
import java.util.Collection;
import java.util.Collections;public class YearShardingAlgorithm implements StandardShardingAlgorithm<LocalDateTime> {@Overridepublic String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<LocalDateTime> shardingValue) {// 获取逻辑表名String logicTableName = shardingValue.getLogicTableName();Object value = shardingValue.getValue();String year = value.toString();//年份格式会有多种 所以直接截取这里if (year.length() > 4) {year = year.substring(0, 4);}//25年的存到备份表logicTableNameif (!"2025".equals(year)){logicTableName = logicTableName + "_" + year;}// 检查表是否存在if (availableTargetNames.contains(logicTableName)) {return logicTableName;} else {LoggerManager.exceptionLogger.error("The table does not exist,please check HKEPrivateConfig/sharding.yaml, tableName: {}", logicTableName);return logicTableName;}}@Overridepublic Collection<String> doSharding(Collection<String> collection, RangeShardingValue<LocalDateTime> rangeShardingValue) {return Collections.emptyList();}
}

Springboot框架

Sharding-JDBC 5.4.x版本yaml实现

在springboot的application.yml文件里面，直接加入分片的配置即可,springboot会自动解析该yaml文件不需要手动读取

mode:type: Standalonerepository:type: JDBC
# 数据库标识 随便写
databaseName: zhongxin
# 数据库配置，连接池配置
dataSources:ds:dataSourceClassName: com.zaxxer.hikari.HikariDataSourcedriverClassName: com.mysql.cj.jdbc.Driverurl: jdbc:mysql://192.168.xx.xx:3306/zhongxin_new?rewriteBatchedStatements=true&allowMultiQueries=true&useLocalSessionState=true&useUnicode=true&characterEncoding=utf-8&socketTimeout=3000000&connectTimeout=60000username: rootpassword:connectionTimeout: 50000minimumIdle: 5maximumPoolSize: 10idleTimeout: 600000maxLifetime: 1800000connectionTestQuery: SELECT 1
# 5.x版本的配法
rules:
- !SHARDINGtables:table:actualDataNodes: ds.table,ds.table_$->{2026..2028}tableStrategy:standard:shardingColumn: operate_timeshardingAlgorithmName: infoAlgorithmtable1:actualDataNodes: ds.table1,ds.table1_$->{2026..2028}tableStrategy:standard:shardingColumn: operate_timeshardingAlgorithmName: infoAlgorithmtable2:actualDataNodes: ds.table2,ds.table2_$->{2026..2028}tableStrategy:standard:shardingColumn: operate_timeshardingAlgorithmName: infoAlgorithmshardingAlgorithms:infoAlgorithm:type: CLASS_BASEDprops:strategy: STANDARDalgorithmClassName: com.package.YearShardingAlgorithmkeyGenerators:snowflake:type: SNOWFLAKE
# 不分区的表
- !SINGLEtables:- ds.*props:sql-show: true

分库、加密、读写分离基于yaml的配置示例

mode:type: Standalonerepository:type: JDBC
databaseName: demo_db
dataSources:ds_basic:dataSourceClassName: com.alibaba.druid.pool.DruidDataSourcedriverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://127.0.0.1:3306/demo_basic?characterEncoding=utf-8&allowPublicKeyRetrieval=true&useSSL=falseusername: rootpassword: Ph9ep971fm14nYaZsLl9LY+MCqX9uJSozYRNgP2VVSj/hbmokn5OC6kpiAA1I0okA9GiDHEo7qHUvRQYYUNZvQ==initialSize: 1minIdle: 1maxActive: 64maxWait: 20000validationQuery: SELECT 1 FROM DUALvalidationQueryTimeout: 30000minEvictableIdleTimeMillis: 300000maxEvictableIdleTimeMillis: 600000timeBetweenEvictionRunsMillis: 300000testOnBorrow: truetestWhileIdle: truefilters: config, stat, wallconnectProperties:connectTimeout: 5000socketTimeout: '20000'config.decrypt: 'true'config.decrypt.key: MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALZRYgsnvVKPqZTfMOWmmj6OuupFRSk7+Vtqv70cG3y6T3bm+DcQU3zOC993ozbHpmqeODtuLzURhIuXDMyTKW8CAwEAAQ==ds0000:dataSourceClassName: com.alibaba.druid.pool.DruidDataSourcedriverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://127.0.0.1:3306/demo_0000?characterEncoding=utf-8&allowPublicKeyRetrieval=true&useSSL=falseusername: rootpassword: Ph9ep971fm14nYaZsLl9LY+MCqX9uJSozYRNgP2VVSj/hbmokn5OC6kpiAA1I0okA9GiDHEo7qHUvRQYYUNZvQ==initialSize: 1minIdle: 1maxActive: 64maxWait: 20000validationQuery: SELECT 1 FROM DUALvalidationQueryTimeout: 30000minEvictableIdleTimeMillis: 300000maxEvictableIdleTimeMillis: 600000timeBetweenEvictionRunsMillis: 300000testOnBorrow: truetestWhileIdle: truefilters: config, stat, wallconnectProperties:connectTimeout: 5000socketTimeout: '20000'config.decrypt: 'true'config.decrypt.key: MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALZRYgsnvVKPqZTfMOWmmj6OuupFRSk7+Vtqv70cG3y6T3bm+DcQU3zOC993ozbHpmqeODtuLzURhIuXDMyTKW8CAwEAAQ==ds0001:dataSourceClassName: com.alibaba.druid.pool.DruidDataSourcedriverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://127.0.0.1:3306/demo_0001?characterEncoding=utf-8&allowPublicKeyRetrieval=true&useSSL=falseusername: rootpassword: Ph9ep971fm14nYaZsLl9LY+MCqX9uJSozYRNgP2VVSj/hbmokn5OC6kpiAA1I0okA9GiDHEo7qHUvRQYYUNZvQ==initialSize: 1minIdle: 1maxActive: 64maxWait: 20000validationQuery: SELECT 1 FROM DUALvalidationQueryTimeout: 30000minEvictableIdleTimeMillis: 300000maxEvictableIdleTimeMillis: 600000timeBetweenEvictionRunsMillis: 300000testOnBorrow: truetestWhileIdle: truefilters: config, stat, wallconnectProperties:connectTimeout: 5000socketTimeout: '20000'config.decrypt: 'true'config.decrypt.key: MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALZRYgsnvVKPqZTfMOWmmj6OuupFRSk7+Vtqv70cG3y6T3bm+DcQU3zOC993ozbHpmqeODtuLzURhIuXDMyTKW8CAwEAAQ==ds0000_slave:dataSourceClassName: com.alibaba.druid.pool.DruidDataSourcedriverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://192.168.1.88:3306/demo_0000?characterEncoding=utf-8&allowPublicKeyRetrieval=true&useSSL=falseusername: rootpassword: Ph9ep971fm14nYaZsLl9LY+MCqX9uJSozYRNgP2VVSj/hbmokn5OC6kpiAA1I0okA9GiDHEo7qHUvRQYYUNZvQ==initialSize: 1minIdle: 1maxActive: 64maxWait: 20000validationQuery: SELECT 1 FROM DUALvalidationQueryTimeout: 30000minEvictableIdleTimeMillis: 300000maxEvictableIdleTimeMillis: 600000timeBetweenEvictionRunsMillis: 300000testOnBorrow: truetestWhileIdle: truefilters: config, stat, wallconnectProperties:connectTimeout: 5000socketTimeout: '20000'config.decrypt: 'true'config.decrypt.key: MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALZRYgsnvVKPqZTfMOWmmj6OuupFRSk7+Vtqv70cG3y6T3bm+DcQU3zOC993ozbHpmqeODtuLzURhIuXDMyTKW8CAwEAAQ==ds0001_slave:dataSourceClassName: com.alibaba.druid.pool.DruidDataSourcedriverClassName: com.mysql.jdbc.Driverurl: jdbc:mysql://192.168.1.88:3306/demo_0001?characterEncoding=utf-8&allowPublicKeyRetrieval=true&useSSL=falseusername: rootpassword: Ph9ep971fm14nYaZsLl9LY+MCqX9uJSozYRNgP2VVSj/hbmokn5OC6kpiAA1I0okA9GiDHEo7qHUvRQYYUNZvQ==initialSize: 1minIdle: 1maxActive: 64maxWait: 20000validationQuery: SELECT 1 FROM DUALvalidationQueryTimeout: 30000minEvictableIdleTimeMillis: 300000maxEvictableIdleTimeMillis: 600000timeBetweenEvictionRunsMillis: 300000testOnBorrow: truetestWhileIdle: truefilters: config, stat, wallconnectProperties:connectTimeout: 5000socketTimeout: '20000'config.decrypt: 'true'config.decrypt.key: MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALZRYgsnvVKPqZTfMOWmmj6OuupFRSk7+Vtqv70cG3y6T3bm+DcQU3zOC993ozbHpmqeODtuLzURhIuXDMyTKW8CAwEAAQ==rules:
# 数据分片
- !SHARDINGtables:t_claim_case_mdtrt:actualDataNodes: ds$->{['0000','0001']}.t_claim_case_mdtrt_000$->{0..9}tableStrategy:standard:shardingColumn: transaction_noshardingAlgorithmName: t_claim_case_mdtrt_inlinekeyGenerateStrategy:column: idkeyGeneratorName: snowflaket_claim_case_info:actualDataNodes: ds$->{['0000','0001']}.t_claim_case_info_000$->{0..9}tableStrategy:standard:shardingColumn: transaction_noshardingAlgorithmName: t_claim_case_info_inlinekeyGenerateStrategy:column: idkeyGeneratorName: snowflakedefaultShardingColumn: transaction_nobindingTables:- t_claim_case_mdtrt, t_claim_case_infodefaultDatabaseStrategy:standard:shardingColumn: transaction_noshardingAlgorithmName: database_inlinedefaultTableStrategy:none:shardingAlgorithms:database_inline:type: INLINEprops:algorithm-expression: ds$->{transaction_no[-8..-5]}t_claim_case_mdtrt_inline:type: INLINEprops:algorithm-expression: t_claim_case_mdtrt_$->{transaction_no[-4..-1]}t_claim_case_info_inline:type: INLINEprops:algorithm-expression: t_claim_case_info_$->{transaction_no[-4..-1]}keyGenerators:snowflake:type: SNOWFLAKE#数据加密
- !ENCRYPTtables:t_claim_case_info:columns:appl_mobile:cipher:name: appl_mobileencryptorName: sm4_encryptoropsnId_no:cipher:name: opsnId_noencryptorName: sm4_encryptorrpter_id_no:cipher:name: rpter_id_noencryptorName: sm4_encryptorrpter_mobile:cipher:name: rpter_mobileencryptorName: sm4_encryptorencryptors:sm4_encryptor:type: SM4props:sm4-key: 86C63180C2806ED1F43A859DE501215Csm4-mode: ECBsm4-padding: PKCS5Padding
# 单表
- !SINGLEtables:- ds_basic.*# 读写分离
- !READWRITE_SPLITTINGdataSources:ds0000:writeDataSourceName: ds0000readDataSourceNames:- ds0000_slavetransactionalReadQueryStrategy: PRIMARYloadBalancerName: randomds0001:writeDataSourceName: ds0001readDataSourceNames:- ds0001_slavetransactionalReadQueryStrategy: PRIMARYloadBalancerName: randomloadBalancers:random:type: RANDOMprops:sql-show: truemax-connections-size-per-query: 5

文章转载自：

http://ZJQvQeDj.hwbmn.cn
http://9STftwCR.hwbmn.cn
http://FLIUe570.hwbmn.cn
http://lWxrR7Tn.hwbmn.cn
http://7wfv3paE.hwbmn.cn
http://prQtDeVH.hwbmn.cn
http://m1tScC75.hwbmn.cn
http://xIFO4tFc.hwbmn.cn
http://hOA4l3Xn.hwbmn.cn
http://naryGqn2.hwbmn.cn
http://kNQ5p7K3.hwbmn.cn
http://ZrvCOTC1.hwbmn.cn
http://1yu4PChq.hwbmn.cn
http://77pbxZvR.hwbmn.cn
http://8RrHAZoX.hwbmn.cn
http://LGXEmBYc.hwbmn.cn
http://5lp4VzBi.hwbmn.cn
http://uwVtcx06.hwbmn.cn
http://NXensuLV.hwbmn.cn
http://a5F5oI7H.hwbmn.cn
http://nHfPzYEA.hwbmn.cn
http://aVGttseF.hwbmn.cn
http://6SpmLk3C.hwbmn.cn
http://fzpaWJt5.hwbmn.cn
http://ZDrs0kBz.hwbmn.cn
http://Qne3ZJOx.hwbmn.cn
http://xzQamXY4.hwbmn.cn
http://ytM1kYFP.hwbmn.cn
http://hF1ijKd2.hwbmn.cn
http://NL7BOpMY.hwbmn.cn
http://www.dtcms.com/a/367096.html

相关文章：

  • C++：类和对象（中）
  • 25高教社杯数模国赛【C题国一亲授思路+问题解析】第四弹
  • 【数学建模学习笔记】机器学习回归：K邻近回归
  • JavaEE 进阶第二期：开启前端入门之旅(二)
  • 准确率可达99%！注意力机制+UNet，A会轻松收割！
  • SpringBoot 项目一些语法记录
  • 单通道ADC采集实验（单次非扫描软件触发）
  • 同步安卓手机的照片到NAS的方案（完美）
  • 嵌入式设备的外设驱动优化
  • 51单片机---硬件学习（跑马灯、数码管、外部中断、按键、蜂鸣器）
  • 嵌入式 - 硬件：51单片机（3）uart串口
  • 深度剖析：智能驾驶到底给2025带来了什么
  • MTK Linux DRM分析（三十六）- MTK mtk_cec.c
  • mysql分页SQL
  • JavaAI炫技赛：电商系统商品管理模块的智能化设计与高效实现
  • Web安全：你所不知道的HTTP Referer注入攻击
  • JS本地存储
  • python包管理神器Miniconda
  • 表达式引擎工具比较选型
  • linux thread 线程一
  • SurfaceFlinger SurfaceContol(一) SurfaceComposerClient
  • 高级RAG策略学习（二）——自适应检索系统原理讲解
  • Python快速入门专业版（三）：print 格式化输出：% 占位符、format 方法与 f-string（谁更高效？）
  • 2025打磨机器人品牌及自动化打磨抛光设备技术新版分析
  • 只会git push？——git团队协作进阶
  • Ubuntu系统配置镜像源
  • RTSP H.265 与 RTMP H.265 的差异解析：标准、扩展与增强实现
  • Vue基础知识-脚手架开发-子传父（props回调函数实现和自定义事件实现）
  • 九、数据库技术基础
  • Roo Code之自定义指令（Custom Instructions），规则（Rules）