分布式短链接系统设计方案

分布式短链接系统设计方案

1. 系统架构设计

1.1 整体系统架构图

                                    [客户端]|[CDN]|[负载均衡器]/    \[API Gateway]  [Web Server]|             |┌─────────┴─────────────┴─────────┐|         应用服务层              ||  ┌─────────┬─────────┬────────┐ ||  |短链生成 |URL重定向|统计服务| ||  |服务     |服务     |       | ||  └─────────┴─────────┴────────┘ |└─────────┬─────────────┬─────────┘|             |┌─────────┴─────────┐   ||    缓存层         |   || ┌─────┬─────────┐ |   || |Redis|Memcached| |   || |集群 |         | |   || └─────┴─────────┘ |   |└─────────┬─────────┘   ||             |┌─────────┴─────────────┴─────────┐|          数据存储层              || ┌──────────┬──────────┬────────┐ || |MySQL主库 |MySQL从库|MongoDB| || |分片集群  |读写分离  |日志存储| || └──────────┴──────────┴────────┘ |└─────────────────────────────────┘

1.2 核心组件说明

1.2.1 接入层

• CDN: 全球分布式缓存，提升访问速度
• 负载均衡器: Nginx/HAProxy，支持多种负载均衡算法
• API Gateway: 统一入口，提供限流、鉴权、监控功能

1.2.2 应用服务层

• 短链生成服务: 负责将长URL转换为短链接
• URL重定向服务: 处理短链接访问，重定向到原始URL
• 统计服务: 收集和分析访问数据
• 管理服务: 提供短链接的增删改查功能

1.2.3 缓存层

• Redis集群: 热点数据缓存，支持主从复制和哨兵模式
• 本地缓存: 应用层缓存，减少网络开销

1.2.4 数据存储层

• MySQL分片集群: 存储URL映射关系
• MongoDB: 存储访问日志和统计数据
• 消息队列: 异步处理统计数据

1.3 核心业务流程设计

1.3.1 短链接生成流程

用户提交长URL → 参数校验 → 检查缓存 → 生成短链接ID → 
存储映射关系 → 更新缓存 → 返回短链接

1.3.2 短链接访问流程

用户访问短链接 → CDN查找 → 缓存查找 → 数据库查找 → 
记录访问日志 → 重定向到原始URL

2. 核心算法设计

2.1 短链接生成算法对比

2.1.1 Base62编码方案

public class Base62Encoder {private static final String BASE62 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";private static final int BASE = BASE62.length();public static String encode(long num) {StringBuilder sb = new StringBuilder();while (num > 0) {sb.append(BASE62.charAt((int)(num % BASE)));num /= BASE;}return sb.reverse().toString();}public static long decode(String str) {long num = 0;for (char c : str.toCharArray()) {num = num * BASE + BASE62.indexOf(c);}return num;}
}

优点:

• 算法简单，性能高
• 生成的短链接较短
• 无需额外存储

缺点:

• 可预测性高，存在安全风险
• 需要全局唯一ID生成器

2.1.2 雪花算法 + Base62方案

public class SnowflakeIdGenerator {private final long epoch = 1640995200000L; // 2022-01-01 00:00:00private final long workerIdBits = 10L;private final long sequenceBits = 12L;private final long maxWorkerId = ~(-1L << workerIdBits);private final long maxSequence = ~(-1L << sequenceBits);private final long workerIdShift = sequenceBits;private final long timestampShift = sequenceBits + workerIdBits;private long workerId;private long sequence = 0L;private long lastTimestamp = -1L;public SnowflakeIdGenerator(long workerId) {if (workerId > maxWorkerId || workerId < 0) {throw new IllegalArgumentException("Worker ID out of range");}this.workerId = workerId;}public synchronized long nextId() {long timestamp = System.currentTimeMillis();if (timestamp < lastTimestamp) {throw new RuntimeException("Clock moved backwards");}if (timestamp == lastTimestamp) {sequence = (sequence + 1) & maxSequence;if (sequence == 0) {timestamp = waitNextMillis(lastTimestamp);}} else {sequence = 0L;}lastTimestamp = timestamp;return ((timestamp - epoch) << timestampShift) |(workerId << workerIdShift) |sequence;}private long waitNextMillis(long lastTimestamp) {long timestamp = System.currentTimeMillis();while (timestamp <= lastTimestamp) {timestamp = System.currentTimeMillis();}return timestamp;}
}

优点:

• 全局唯一，无重复
• 性能高，单机可达百万QPS
• 包含时间信息，便于排序

缺点:

• 依赖机器时钟
• 需要机器ID管理

2.1.3 Hash + 冲突检测方案

public class HashBasedGenerator {private static final String SALT = "your_salt_here";public String generateShortUrl(String longUrl) {String combined = longUrl + SALT + System.currentTimeMillis();long hash = MurmurHash.hash64(combined.getBytes());return Base62Encoder.encode(Math.abs(hash));}public String generateWithCollisionDetection(String longUrl) {String shortUrl;int attempts = 0;do {String input = longUrl + SALT + System.currentTimeMillis() + attempts;long hash = MurmurHash.hash64(input.getBytes());shortUrl = Base62Encoder.encode(Math.abs(hash));attempts++;} while (exists(shortUrl) && attempts < 5);if (attempts >= 5) {// 降级到雪花算法return Base62Encoder.encode(snowflakeGenerator.nextId());}return shortUrl;}
}

2.2 分布式ID生成方案

2.2.1 数据库自增ID + 步长

-- 节点1: 起始值1，步长3
ALTER TABLE url_mapping AUTO_INCREMENT = 1;
SET @@auto_increment_increment = 3;-- 节点2: 起始值2，步长3  
ALTER TABLE url_mapping AUTO_INCREMENT = 2;
SET @@auto_increment_increment = 3;-- 节点3: 起始值3，步长3
ALTER TABLE url_mapping AUTO_INCREMENT = 3;
SET @@auto_increment_increment = 3;

2.2.2 Redis计数器方案

public class RedisIdGenerator {private RedisTemplate<String, String> redisTemplate;private String keyPrefix = "short_url_id:";public long nextId(int shardId) {String key = keyPrefix + shardId;return redisTemplate.opsForValue().increment(key, 1);}public String generateShortUrl(int shardId) {long id = nextId(shardId);return Base62Encoder.encode(id);}
}

3. 数据库设计

3.1 数据表结构设计

3.1.1 URL映射表

CREATE TABLE `url_mapping` (`id` bigint(20) NOT NULL AUTO_INCREMENT,`short_url` varchar(10) NOT NULL COMMENT '短链接标识',`long_url` text NOT NULL COMMENT '原始长URL',`user_id` bigint(20) DEFAULT NULL COMMENT '用户ID',`expire_time` datetime DEFAULT NULL COMMENT '过期时间',`status` tinyint(1) DEFAULT 1 COMMENT '状态：1-有效，0-无效',`created_time` datetime DEFAULT CURRENT_TIMESTAMP,`updated_time` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,PRIMARY KEY (`id`),UNIQUE KEY `uk_short_url` (`short_url`),KEY `idx_user_id` (`user_id`),KEY `idx_created_time` (`created_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='URL映射表';

3.1.2 访问统计表

CREATE TABLE `url_statistics` (`id` bigint(20) NOT NULL AUTO_INCREMENT,`short_url` varchar(10) NOT NULL,`access_date` date NOT NULL,`pv` bigint(20) DEFAULT 0 COMMENT '页面访问量',`uv` bigint(20) DEFAULT 0 COMMENT '独立访客数',`ip_count` bigint(20) DEFAULT 0 COMMENT '独立IP数',`created_time` datetime DEFAULT CURRENT_TIMESTAMP,`updated_time` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,PRIMARY KEY (`id`),UNIQUE KEY `uk_short_url_date` (`short_url`, `access_date`),KEY `idx_access_date` (`access_date`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='URL访问统计表';

3.1.3 访问日志表（MongoDB）

// MongoDB集合结构
{"_id": ObjectId("..."),"shortUrl": "abc123","longUrl": "https://example.com/very/long/url","clientIp": "192.168.1.1","userAgent": "Mozilla/5.0...","referer": "https://google.com","accessTime": ISODate("2023-01-01T12:00:00Z"),"responseTime": 50,"statusCode": 302,"country": "CN","city": "Beijing","device": "mobile"
}

3.2 分库分表策略

3.2.1 水平分表策略

public class ShardingStrategy {private static final int SHARD_COUNT = 1024;public String getShardTable(String shortUrl) {int hash = shortUrl.hashCode();int shardId = Math.abs(hash) % SHARD_COUNT;return "url_mapping_" + String.format("%04d", shardId);}public String getShardDatabase(String shortUrl) {int hash = shortUrl.hashCode();int dbId = Math.abs(hash) % 8; // 8个数据库return "shorturl_db_" + dbId;}
}

3.2.2 分库分表配置

# ShardingSphere配置
spring:shardingsphere:datasource:names: ds0,ds1,ds2,ds3,ds4,ds5,ds6,ds7ds0:type: com.zaxxer.hikari.HikariDataSourcedriver-class-name: com.mysql.cj.jdbc.Driverjdbc-url: jdbc:mysql://192.168.1.10:3306/shorturl_db_0# ... 其他数据源配置rules:sharding:tables:url_mapping:actual-data-nodes: ds$->{0..7}.url_mapping_$->{0000..1023}database-strategy:standard:sharding-column: short_urlsharding-algorithm-name: database_inlinetable-strategy:standard:sharding-column: short_urlsharding-algorithm-name: table_inlinesharding-algorithms:database_inline:type: INLINEprops:algorithm-expression: ds$->{Math.abs(short_url.hashCode()) % 8}table_inline:type: INLINEprops:algorithm-expression: url_mapping_$->{String.format('%04d', Math.abs(short_url.hashCode()) % 1024)}

3.3 索引设计

3.3.1 主要索引策略

-- 短链接唯一索引（最重要）
CREATE UNIQUE INDEX uk_short_url ON url_mapping(short_url);-- 用户ID索引（用户查询自己的短链接）
CREATE INDEX idx_user_id ON url_mapping(user_id);-- 创建时间索引（按时间范围查询）
CREATE INDEX idx_created_time ON url_mapping(created_time);-- 过期时间索引（清理过期数据）
CREATE INDEX idx_expire_time ON url_mapping(expire_time);-- 状态索引（查询有效链接）
CREATE INDEX idx_status ON url_mapping(status);-- 复合索引（用户查询自己的有效链接）
CREATE INDEX idx_user_status ON url_mapping(user_id, status);

3.3.2 MongoDB索引

// 短链接索引
db.access_logs.createIndex({"shortUrl": 1});// 时间范围查询索引
db.access_logs.createIndex({"accessTime": 1});// 复合索引（按短链接和时间查询）
db.access_logs.createIndex({"shortUrl": 1, "accessTime": 1});// IP地址索引（防刷分析）
db.access_logs.createIndex({"clientIp": 1});// TTL索引（自动删除过期日志）
db.access_logs.createIndex({"accessTime": 1}, {expireAfterSeconds: 7776000}); // 90天

4. 关键技术方案

4.1 缓存策略

4.1.1 Redis集群配置

spring:redis:cluster:nodes:- 192.168.1.10:7000- 192.168.1.10:7001- 192.168.1.11:7000- 192.168.1.11:7001- 192.168.1.12:7000- 192.168.1.12:7001max-redirects: 3password: your_passwordtimeout: 3000mslettuce:pool:max-active: 200max-idle: 20min-idle: 5max-wait: 3000ms

4.1.2 多级缓存策略

@Service
public class UrlMappingService {@Autowiredprivate RedisTemplate<String, String> redisTemplate;@Autowiredprivate UrlMappingMapper urlMappingMapper;// 本地缓存private final Cache<String, String> localCache = Caffeine.newBuilder().maximumSize(10000).expireAfterWrite(5, TimeUnit.MINUTES).build();public String getLongUrl(String shortUrl) {// 1. 本地缓存String longUrl = localCache.getIfPresent(shortUrl);if (longUrl != null) {return longUrl;}// 2. Redis缓存longUrl = redisTemplate.opsForValue().get("url:" + shortUrl);if (longUrl != null) {localCache.put(shortUrl, longUrl);return longUrl;}// 3. 数据库查询UrlMapping mapping = urlMappingMapper.selectByShortUrl(shortUrl);if (mapping != null && mapping.getStatus() == 1) {longUrl = mapping.getLongUrl();// 更新缓存redisTemplate.opsForValue().set("url:" + shortUrl, longUrl, Duration.ofHours(24));localCache.put(shortUrl, longUrl);return longUrl;}return null;}public void invalidateCache(String shortUrl) {localCache.invalidate(shortUrl);redisTemplate.delete("url:" + shortUrl);}
}

4.1.3 缓存预热策略

@Component
public class CacheWarmupService {@Autowiredprivate UrlMappingService urlMappingService;@Autowiredprivate RedisTemplate<String, String> redisTemplate;@Scheduled(fixedRate = 3600000) // 每小时执行一次public void warmupHotUrls() {// 获取热点短链接List<String> hotUrls = getHotUrlsFromStatistics();for (String shortUrl : hotUrls) {String longUrl = urlMappingService.getLongUrlFromDb(shortUrl);if (longUrl != null) {redisTemplate.opsForValue().set("url:" + shortUrl, longUrl, Duration.ofHours(24));}}}private List<String> getHotUrlsFromStatistics() {// 从统计数据中获取热点URLreturn urlStatisticsMapper.selectHotUrls(1000);}
}

4.2 数据一致性保证

4.2.1 分布式事务处理

@Service
public class UrlCreationService {@Autowiredprivate UrlMappingMapper urlMappingMapper;@Autowiredprivate RedisTemplate<String, String> redisTemplate;@Autowiredprivate RocketMQTemplate rocketMQTemplate;@Transactional(rollbackFor = Exception.class)public String createShortUrl(CreateUrlRequest request) {try {// 1. 生成短链接String shortUrl = generateShortUrl();// 2. 数据库插入UrlMapping mapping = new UrlMapping();mapping.setShortUrl(shortUrl);mapping.setLongUrl(request.getLongUrl());mapping.setUserId(request.getUserId());mapping.setExpireTime(request.getExpireTime());urlMappingMapper.insert(mapping);// 3. 发送异步消息更新缓存CacheUpdateMessage message = new CacheUpdateMessage();message.setShortUrl(shortUrl);message.setLongUrl(request.getLongUrl());message.setOperation("CREATE");rocketMQTemplate.convertAndSend("cache-update-topic", message);return shortUrl;} catch (Exception e) {log.error("创建短链接失败", e);throw new BusinessException("创建短链接失败");}}
}@RocketMQMessageListener(topic = "cache-update-topic", consumerGroup = "cache-consumer")
@Component
public class CacheUpdateConsumer implements RocketMQListener<CacheUpdateMessage> {@Overridepublic void onMessage(CacheUpdateMessage message) {try {switch (message.getOperation()) {case "CREATE":case "UPDATE":redisTemplate.opsForValue().set("url:" + message.getShortUrl(), message.getLongUrl(), Duration.ofHours(24));break;case "DELETE":redisTemplate.delete("url:" + message.getShortUrl());break;}} catch (Exception e) {log.error("更新缓存失败", e);// 重试机制throw e;}}
}

4.2.2 最终一致性保证

@Component
public class ConsistencyChecker {@Scheduled(fixedRate = 300000) // 每5分钟检查一次public void checkDataConsistency() {// 1. 检查数据库和缓存的一致性List<String> inconsistentUrls = findInconsistentUrls();for (String shortUrl : inconsistentUrls) {// 以数据库为准，更新缓存UrlMapping mapping = urlMappingMapper.selectByShortUrl(shortUrl);if (mapping != null && mapping.getStatus() == 1) {redisTemplate.opsForValue().set("url:" + shortUrl, mapping.getLongUrl(), Duration.ofHours(24));} else {redisTemplate.delete("url:" + shortUrl);}}}private List<String> findInconsistentUrls() {// 采样检查策略，避免全量检查List<String> sampleUrls = getSampleUrls(1000);List<String> inconsistentUrls = new ArrayList<>();for (String shortUrl : sampleUrls) {String cachedUrl = redisTemplate.opsForValue().get("url:" + shortUrl);UrlMapping dbMapping = urlMappingMapper.selectByShortUrl(shortUrl);String dbUrl = (dbMapping != null && dbMapping.getStatus() == 1) ? dbMapping.getLongUrl() : null;if (!Objects.equals(cachedUrl, dbUrl)) {inconsistentUrls.add(shortUrl);}}return inconsistentUrls;}
}

4.3 高可用设计

4.3.1 服务熔断与降级

@Component
public class UrlServiceFallback {@HystrixCommand(fallbackMethod = "getLongUrlFallback",commandProperties = {@HystrixProperty(name = "circuitBreaker.enabled", value = "true"),@HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "20"),@HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50"),@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3000")})public String getLongUrl(String shortUrl) {return urlMappingService.getLongUrl(shortUrl);}public String getLongUrlFallback(String shortUrl) {// 降级策略：返回默认页面或错误页面log.warn("获取长链接失败，触发降级: {}", shortUrl);return "https://example.com/error?code=service_unavailable";}@HystrixCommand(fallbackMethod = "createShortUrlFallback")public String createShortUrl(CreateUrlRequest request) {return urlCreationService.createShortUrl(request);}public String createShortUrlFallback(CreateUrlRequest request) {// 降级策略：返回错误信息throw new ServiceUnavailableException("短链接服务暂时不可用，请稍后重试");}
}

4.3.2 限流策略

@RestController
@RequestMapping("/api/v1/url")
public class UrlController {// 基于令牌桶的限流private final RateLimiter rateLimiter = RateLimiter.create(1000.0); // 每秒1000个请求// 基于用户的限流private final LoadingCache<String, RateLimiter> userRateLimiters = Caffeine.newBuilder().maximumSize(10000).expireAfterAccess(1, TimeUnit.HOURS).build(key -> RateLimiter.create(10.0)); // 每个用户每秒10个请求@PostMapping("/create")public Result<String> createShortUrl(@RequestBody CreateUrlRequest request) {// 全局限流if (!rateLimiter.tryAcquire(100, TimeUnit.MILLISECONDS)) {return Result.error("系统繁忙，请稍后重试");}// 用户限流String userId = getCurrentUserId();RateLimiter userLimiter = userRateLimiters.get(userId);if (!userLimiter.tryAcquire(100, TimeUnit.MILLISECONDS)) {return Result.error("请求过于频繁，请稍后重试");}try {String shortUrl = urlServiceFallback.createShortUrl(request);return Result.success(shortUrl);} catch (Exception e) {log.error("创建短链接失败", e);return Result.error("创建失败，请重试");}}@GetMapping("/{shortUrl}")public void redirect(@PathVariable String shortUrl, HttpServletResponse response) {// 重定向请求的限流策略相对宽松if (!rateLimiter.tryAcquire(10, TimeUnit.MILLISECONDS)) {response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());return;}try {String longUrl = urlServiceFallback.getLongUrl(shortUrl);if (longUrl != null) {// 异步记录访问日志recordAccessLog(shortUrl, request);response.sendRedirect(longUrl);} else {response.setStatus(HttpStatus.NOT_FOUND.value());}} catch (Exception e) {log.error("重定向失败", e);response.setStatus(HttpStatus.INTERNAL_SERVER_ERROR.value());}}
}

4.3.3 监控与告警

@Component
public class SystemMonitor {private final MeterRegistry meterRegistry;private final Timer.Sample sample;@EventListenerpublic void handleUrlCreated(UrlCreatedEvent event) {// 记录创建短链接的指标meterRegistry.counter("url.created", "user_id", event.getUserId(),"status", "success").increment();}@EventListenerpublic void handleUrlAccessed(UrlAccessedEvent event) {// 记录访问指标meterRegistry.counter("url.accessed","short_url", event.getShortUrl(),"status_code", String.valueOf(event.getStatusCode())).increment();// 记录响应时间Timer.Sample sample = Timer.start(meterRegistry);sample.stop(Timer.builder("url.access.duration").description("URL access duration").register(meterRegistry));}@Scheduled(fixedRate = 60000) // 每分钟检查一次public void checkSystemHealth() {// 检查数据库连接boolean dbHealth = checkDatabaseHealth();meterRegistry.gauge("system.db.health", dbHealth ? 1 : 0);// 检查Redis连接boolean redisHealth = checkRedisHealth();meterRegistry.gauge("system.redis.health", redisHealth ? 1 : 0);// 检查服务响应时间long avgResponseTime = getAverageResponseTime();meterRegistry.gauge("system.response.time.avg", avgResponseTime);// 告警逻辑if (!dbHealth || !redisHealth || avgResponseTime > 1000) {sendAlert("系统健康检查异常");}}
}

5. 性能优化方案

5.1 读写分离优化

@Configuration
public class DataSourceConfig {@Bean@Primarypublic DataSource dataSource() {HikariDataSource masterDataSource = new HikariDataSource();masterDataSource.setJdbcUrl("jdbc:mysql://master-db:3306/shorturl");masterDataSource.setMaximumPoolSize(50);HikariDataSource slaveDataSource = new HikariDataSource();slaveDataSource.setJdbcUrl("jdbc:mysql://slave-db:3306/shorturl");slaveDataSource.setMaximumPoolSize(100);Map<Object, Object> dataSourceMap = new HashMap<>();dataSourceMap.put("master", masterDataSource);dataSourceMap.put("slave", slaveDataSource);DynamicDataSource dynamicDataSource = new DynamicDataSource();dynamicDataSource.setTargetDataSources(dataSourceMap);dynamicDataSource.setDefaultTargetDataSource(masterDataSource);return dynamicDataSource;}
}@Aspect
@Component
public class DataSourceAspect {@Around("@annotation(readOnly)")public Object around(ProceedingJoinPoint point, ReadOnly readOnly) throws Throwable {try {DataSourceContextHolder.setDataSourceType("slave");return point.proceed();} finally {DataSourceContextHolder.clearDataSourceType();}}
}

5.2 异步处理优化

@Service
public class AsyncUrlService {@Async("urlTaskExecutor")public CompletableFuture<Void> recordAccessLog(AccessLogDto logDto) {try {// 批量插入优化accessLogBatch.add(logDto);if (accessLogBatch.size() >= 100) {flushAccessLogs();}} catch (Exception e) {log.error("记录访问日志失败", e);}return CompletableFuture.completedFuture(null);}@Async("statisticsTaskExecutor")public CompletableFuture<Void> updateStatistics(String shortUrl, String clientIp) {try {// 使用Redis HyperLogLog统计UVredisTemplate.opsForHyperLogLog().add("uv:" + shortUrl + ":" + getToday(), clientIp);// 使用Redis计数器统计PVredisTemplate.opsForValue().increment("pv:" + shortUrl + ":" + getToday());} catch (Exception e) {log.error("更新统计数据失败", e);}return CompletableFuture.completedFuture(null);}@Configurationpublic class AsyncConfig {@Bean("urlTaskExecutor")public TaskExecutor urlTaskExecutor() {ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();executor.setCorePoolSize(10);executor.setMaxPoolSize(50);executor.setQueueCapacity(1000);executor.setThreadNamePrefix("url-task-");executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());executor.initialize();return executor;}}
}

6. 安全防护方案

6.1 防刷机制

@Component
public class AntiSpamService {// IP限流private final LoadingCache<String, AtomicInteger> ipCounters = Caffeine.newBuilder().maximumSize(100000).expireAfterWrite(1, TimeUnit.MINUTES).build(key -> new AtomicInteger(0));// 短链接访问频率限制private final LoadingCache<String, AtomicInteger> urlCounters = Caffeine.newBuilder().maximumSize(10000).expireAfterWrite(1, TimeUnit.MINUTES).build(key -> new AtomicInteger(0));public boolean isSpamRequest(String clientIp, String shortUrl) {// IP频率检查AtomicInteger ipCount = ipCounters.get(clientIp);if (ipCount.incrementAndGet() > 1000) { // 每分钟最多1000次log.warn("IP访问频率过高: {}", clientIp);return true;}// 短链接访问频率检查AtomicInteger urlCount = urlCounters.get(shortUrl);if (urlCount.incrementAndGet() > 10000) { // 每分钟最多10000次log.warn("短链接访问频率异常: {}", shortUrl);return true;}return false;}public boolean isBlacklistedIp(String clientIp) {// 检查IP黑名单return redisTemplate.opsForSet().isMember("blacklist:ip", clientIp);}public void addToBlacklist(String clientIp, Duration duration) {redisTemplate.opsForSet().add("blacklist:ip", clientIp);redisTemplate.expire("blacklist:ip", duration);}
}

6.2 恶意URL检测

@Service
public class UrlSecurityService {private final Set<String> maliciousDomains = loadMaliciousDomains();private final Pattern maliciousPattern = Pattern.compile(".*(phishing|malware|virus|trojan|spam).*", Pattern.CASE_INSENSITIVE);public boolean isSafeUrl(String url) {try {URL urlObj = new URL(url);String host = urlObj.getHost().toLowerCase();// 检查恶意域名if (maliciousDomains.contains(host)) {return false;}// 检查URL模式if (maliciousPattern.matcher(url).matches()) {return false;}// 调用第三方安全检测APIreturn checkWithSecurityApi(url);} catch (Exception e) {log.error("URL安全检测失败: {}", url, e);return false;}}private boolean checkWithSecurityApi(String url) {// 集成Google Safe Browsing API或其他安全检测服务// 这里简化处理return true;}
}

7. 部署架构

7.1 Docker容器化部署

# Dockerfile
FROM openjdk:11-jre-slimCOPY target/short-url-service.jar /app/app.jarEXPOSE 8080ENTRYPOINT ["java", "-jar", "/app/app.jar"]

# docker-compose.yml
version: '3.8'
services:app:build: .ports:- "8080:8080"environment:- SPRING_PROFILES_ACTIVE=prod- MYSQL_HOST=mysql- REDIS_HOST=redisdepends_on:- mysql- redisnetworks:- short-url-networkmysql:image: mysql:8.0environment:MYSQL_ROOT_PASSWORD: root123MYSQL_DATABASE: shorturlvolumes:- mysql-data:/var/lib/mysqlnetworks:- short-url-networkredis:image: redis:7-alpinevolumes:- redis-data:/datanetworks:- short-url-networknginx:image: nginx:alpineports:- "80:80"- "443:443"volumes:- ./nginx.conf:/etc/nginx/nginx.confdepends_on:- appnetworks:- short-url-networkvolumes:mysql-data:redis-data:networks:short-url-network:driver: bridge

7.2 Kubernetes部署

# k8s-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: short-url-service
spec:replicas: 3selector:matchLabels:app: short-url-servicetemplate:metadata:labels:app: short-url-servicespec:containers:- name: short-url-serviceimage: short-url-service:latestports:- containerPort: 8080env:- name: SPRING_PROFILES_ACTIVEvalue: "k8s"resources:requests:memory: "512Mi"cpu: "500m"limits:memory: "1Gi"cpu: "1000m"livenessProbe:httpGet:path: /actuator/healthport: 8080initialDelaySeconds: 30periodSeconds: 10readinessProbe:httpGet:path: /actuator/healthport: 8080initialDelaySeconds: 5periodSeconds: 5---
apiVersion: v1
kind: Service
metadata:name: short-url-service
spec:selector:app: short-url-serviceports:- port: 80targetPort: 8080type: LoadBalancer

8. 总结

本分布式短链接系统设计方案具备以下特点：

8.1 核心优势

• 高性能: 支持千万级QPS的访问量
• 高可用: 99.99%的服务可用性
• 高扩展: 支持水平扩展和弹性伸缩
• 数据安全: 多重防护机制保障数据安全

8.2 关键指标

• 响应时间: 平均响应时间 < 100ms
• 存储容量: 支持百亿级URL存储
• 并发处理: 单机支持10万+并发
• 数据一致性: 最终一致性保证

8.3 技术栈总结

• 应用层: Spring Boot + Spring Cloud
• 数据库: MySQL分片集群 + MongoDB
• 缓存: Redis集群 + 本地缓存
• 消息队列: RocketMQ
• 监控: Prometheus + Grafana
• 部署: Docker + Kubernetes

通过合理的架构设计、算法选择和技术方案，该系统能够满足大规模分布式短链接服务的需求，并具备良好的扩展性和维护性。