当前位置：首页 > news >正文

OSS文件上传错误No buffer space available

news 2025/10/29 7:10:08

错误的记录信息

2025-10-28 16:15:31 | ERROR    | app.clients.oss_client:upload_file:53 - 文件上传失败: {'status': -2, 'x-oss-request-id': '', 'details': "RequestError: ('Connection aborted.', OSError(55, 'No buffer space available'))"}
2025-10-28 16:15:31 | ERROR    | app.clients.oss_client:upload_file:53 - 文件上传失败: {'status': -2, 'x-oss-request-id': '', 'details': "RequestError: ('Connection aborted.', OSError(55, 'No buffer space available'))"}
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:_get_audio_url:119 - 上传音频文件失败: {'status': -2, 'x-oss-request-id': '', 'details': "RequestError: ('Connection aborted.', OSError(55, 'No buffer space available'))"}
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:_get_audio_url:119 - 上传音频文件失败: {'status': -2, 'x-oss-request-id': '', 'details': "RequestError: ('Connection aborted.', OSError(55, 'No buffer space available'))"}
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:transcribe_audio_file:40 - 无法获取音频文件URL: /Users/sunyuhua/PycharmProjects/mediation_check/data/mp3/Tue Oct 28 2025 11_33_49 GMT+0800 (GMT+08_00).mp3
2025-10-28 16:15:31 | ERROR    | app.clients.oss_client:upload_file:53 - 文件上传失败: {'status': -2, 'x-oss-request-id': '', 'details': "RequestError: ('Connection aborted.', OSError(55, 'No buffer space available'))"}
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:transcribe_audio_file:40 - 无法获取音频文件URL: /Users/sunyuhua/PycharmProjects/mediation_check/data/mp3/Tue Oct 28 2025 11_46_20 GMT+0800 (GMT+08_00).mp3
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:_get_audio_url:119 - 上传音频文件失败: {'status': -2, 'x-oss-request-id': '', 'details': "RequestError: ('Connection aborted.', OSError(55, 'No buffer space available'))"}
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:batch_transcribe:408 - ✗ 失败 [1/18]: Tue Oct 28 2025 11_33_49 GMT+0800 (GMT+08_00).mp3
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:transcribe_audio_file:40 - 无法获取音频文件URL: /Users/sunyuhua/PycharmProjects/mediation_check/data/mp3/Tue Oct 28 2025 11_43_59 GMT+0800 (GMT+08_00).mp3
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:batch_transcribe:408 - ✗ 失败 [2/18]: Tue Oct 28 2025 11_46_20 GMT+0800 (GMT+08_00).mp3
2025-10-28 16:15:31 | ERROR    | app.services.transcription_service:batch_transcribe:408 - ✗ 失败 [3/18]: Tue Oct 28 2025 11_43_59 GMT+0800 (GMT+08_00).mp3
2025-10-28 16:15:31 | INFO     | app.clients.oss_client:upload_file:44 - 文件上传成功，生成预签名URL: audio/Tue Oct 28 2025 11_29_09 GMT+0800 (GMT+08_00).mp3

什么问题造成的

这是一个 OSError(55, ‘No buffer space available’) 错误，这是操作系统级别的资源耗尽问题。从日志可以看出：
问题分析

并发上传过多：多个文件几乎同时开始处理，导致系统网络缓冲区耗尽
连接/文件句柄未释放：可能存在资源泄露
批量操作缺乏速率控制：没有限制并发数量和添加重试机制

解决的OSS的核心代码

import oss2
import time
from typing import BinaryIO
from app.config import get_settings
from loguru import logger
from functools import wrapsdef retry_on_failure(max_retries=3, delay=1, backoff=2):"""重试装饰器，使用指数退避策略"""def decorator(func):@wraps(func)def wrapper(*args, **kwargs):retry_delay = delayfor attempt in range(max_retries):try:return func(*args, **kwargs)except Exception as e:error_msg = str(e)# 检查是否是资源耗尽错误if 'No buffer space available' in error_msg or 'OSError' in error_msg:if attempt < max_retries - 1:logger.warning(f"尝试 {attempt + 1}/{max_retries} 失败: {error_msg}, "f"{retry_delay}秒后重试...")time.sleep(retry_delay)retry_delay *= backoffcontinue# 其他错误或最后一次尝试，直接抛出raisereturn Nonereturn wrapperreturn decoratorclass OSSClient:"""阿里云 OSS 客户端 - 增强版（带重试和资源管理）"""def __init__(self):settings = get_settings()auth = oss2.Auth(settings.OSS_ACCESS_KEY_ID,settings.OSS_ACCESS_KEY_SECRET)# 配置连接池参数以减少资源消耗self.bucket = oss2.Bucket(auth,settings.OSS_ENDPOINT,settings.OSS_BUCKET_NAME,connect_timeout=30,  # 连接超时30秒)self.audio_path = settings.OSS_AUDIO_PATH@retry_on_failure(max_retries=3, delay=2, backoff=2)def upload_file(self,file: BinaryIO,filename: str,generate_presigned: bool = True,expires: int = 10800) -> str:"""上传文件到 OSS（带重试机制）Args:file: 文件对象filename: 文件名generate_presigned: 是否生成预签名URL，默认Trueexpires: 预签名URL有效期（秒），默认3小时Returns:文件的 URL（预签名URL或公开URL）"""try:object_key = f"{self.audio_path}{filename}"# 读取文件内容（一次性读取，避免持续占用文件句柄）file_content = file.read()# 上传到 OSSself.bucket.put_object(object_key, file_content)# 根据参数决定返回预签名URL还是公开URLif generate_presigned:# 生成预签名URL（用于私有bucket）url = self.bucket.sign_url('GET', object_key, expires)logger.info(f"文件上传成功，生成预签名URL: {object_key}")else:# 生成公开访问的 URL（用于公开bucket）endpoint = self.bucket.endpoint.replace('http://', '').replace('https://', '')url = f"https://{self.bucket.bucket_name}.{endpoint}/{object_key}"logger.info(f"文件上传成功: {url}")return urlexcept Exception as e:logger.error(f"文件上传失败: {str(e)}")raise@retry_on_failure(max_retries=3, delay=1, backoff=2)def generate_presigned_url(self, object_key: str, expires: int = 10800) -> str:"""生成预签名 URL（默认3小时有效期，带重试）Args:object_key: OSS 对象键expires: 有效期（秒），默认 3 小时Returns:预签名 URL"""try:url = self.bucket.sign_url('GET', object_key, expires)logger.info(f"生成预签名URL: {object_key}")return urlexcept Exception as e:logger.error(f"生成预签名URL失败: {str(e)}")raise@retry_on_failure(max_retries=2, delay=1, backoff=2)def delete_file(self, object_key: str) -> bool:"""删除 OSS 文件（带重试）Args:object_key: OSS 对象键Returns:是否删除成功"""try:self.bucket.delete_object(object_key)logger.info(f"文件删除成功: {object_key}")return Trueexcept Exception as e:logger.error(f"文件删除失败: {str(e)}")return False