当前位置：首页 > news >正文

Git LFS 操作处理Github上传大文件操作记录

news 2025/7/15 7:04:44

1、 GitHub上传大文件报错

2、查看大文件提交记录

3、安装 BFG 工具（高效清理 Git 历史大文件）

4、搜索仓库中大于100MB的文件

5、清理历史中的超大文件

6、CentOS 上安装 Git LFS

6.1、Git LFS安装并初始化

7、执行操作

8、不提交大文件

1、 GitHub上传大文件报错

git push -u origin main
Counting objects: 1092, done.
Delta compression using up to 6 threads.
Compressing objects: 100% (975/975), done.
Writing objects: 100% (1092/1092), 504.44 MiB | 4.23 MiB/s, done.
Total 1092 (delta 288), reused 0 (delta 0)
remote: Resolving deltas: 100% (288/288), done.
remote: warning: File workspace/docker/dev-db/zookeeper/log/version-2/log.36 is 64.00 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File workspace/docker/dev-db/zookeeper/log/version-2/log.f4 is 64.00 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File cloud-demo-main/doc/seata-server/lib/rocksdbjni-8.8.1.jar is 58.43 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File workspace/docker/dev-db/zookeeper/log/version-2/log.4a is 64.00 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File workspace/docker/dev-db/zookeeper/log/version-2/log.1 is 64.00 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File workspace/docker/dev-db/zookeeper/log/version-2/log.21 is 64.00 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: error: Trace: 656793fae1dbf2796c07dfe22fc3e3ac6912e4bd81d0283227a3e2ab40b078b7
remote: error: See https://gh.io/lfs for more information.
remote: error: File 000000-local-windows-seata2.0.0-nacos2.3.0.tar.xz is 293.36 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage -

2、查看大文件提交记录

find . -type f -size +50M -exec ls -lh {} \;

git rev-list --all | xargs -r -l git ls-tree -r --long | sort -uk3 | sort -rnk4 | head -10

3、安装 BFG 工具（高效清理 Git 历史大文件）

BFG 比 git filter-branch 更快，适合清理大文件：

wget https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar
mv bfg-1.14.0.jar /usr/local/bin/bfg.jar
chmod +x /usr/local/bin/bfg.jar

4、搜索仓库中大于100MB的文件

git rev-list --objects --all | grep -E `git verify-pack -v .git/objects/pack/*.idx | sort -k 3 -n | tail -10 | awk '{print $1}'`

5、清理历史中的超大文件

# 备份仓库（重要！清理历史有风险）
cp -r example example-bak

# 进入仓库目录
cd example

# 使用 BFG 移除指定大文件（替换为你的文件名）java -jar /usr/local/bin/bfg.jar --delete-files "000000-local-windows-seata2.0.0-nacos2.3.0.tar.xz"
java -jar /usr/local/bin/bfg.jar --delete-files "log.1"
java -jar /usr/local/bin/bfg.jar --delete-files "log.f4"
java -jar /usr/local/bin/bfg.jar --delete-files "log.36"
java -jar /usr/local/bin/bfg.jar --delete-files "log.21"
java -jar /usr/local/bin/bfg.jar --delete-files "log.4a"
java -jar /usr/local/bin/bfg.jar --delete-files "rocksdbjni-8.8.1.jar"
java -jar /usr/local/bin/bfg.jar --delete-files "mysql.ibd"
java -jar /usr/local/bin/bfg.jar --delete-files "_c.cfs"
java -jar /usr/local/bin/bfg.jar --delete-files "skywalking-agent.jar"

# 清理残留的 Git 垃圾
git reflog expire --expire=now --all && git gc --prune=now --aggressive

git push -f origin main

6、CentOS 上安装 Git LFS

使用以下命令添加 Git LFS 的软件仓库到系统中。curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash

Generating yum cache for github_git-lfs...
Importing GPG key 0xDC282033:
Userid : "https://packagecloud.io/github/git-lfs (https://packagecloud.io/docs#gpg_signing) <support@packagecloud.io>"
Fingerprint: 6d39 8dbd 30dd 7894 1e2c 4797 fe2a 5f8b dc28 2033
From : https://packagecloud.io/github/git-lfs/gpgkey
Generating yum cache for github_git-lfs-source...

The repository is setup! You can now install packages.

6.1、Git LFS安装并初始化

sudo yum install git-lfs -y

初始化

git lfs install
Updated Git hooks.
Git LFS initialized.

7、执行操作

# 跟踪 .tar.xz、.jar、.log 等大文件类型（根据你的文件类型调整）
git lfs track "*.tar.xz"
git lfs track "*.jar"
git lfs track "*.log"
git lfs track "*.log.*"
git lfs track "*.ibd"
git lfs track "*.cfs"

# 将跟踪规则提交到仓库
git add .gitattributes
git commit -m "Add LFS tracking for large files"

# 确保工作区的大文件已被 LFS 跟踪（会显示 "Git LFS" 标识）
git add .
git commit -m "Fix large files with LFS"

# 强制推送（因为修改了历史，首次需要 -f，注意：协作仓库慎用！）
git push -u origin main -f

出现报错：

batch response: Git LFS is disabled for this repository.
batch response: Git LFS is disabled for this repository.
batch response: Git LFS is disabled for this repository.
batch response: Git LFS is disabled for this repository.
error: failed to push some refs to 'git

8、跟踪文件清理

git lfs track

git lfs prune

# 查看 LFS 环境信息
git lfs env

# 查看哪些文件被 LFS 跟踪
git lfs status

# 移除所有 LFS 跟踪规则（会修改 .gitattributes 文件）
git lfs untrack "*" 如果不行。换成
echo "" > .gitattributes
git add .gitattributes
git commit -m "Remove all Git LFS tracking rules"
# 3. 提交 .gitattributes 文件的更改
git add .gitattributes

# 1. 禁用当前仓库的 LFS 钩子

git lfs uninstall #

2. 验证 LFS 是否已卸载（应显示错误）

git lfs env # 输出应类似：git: 'lfs' is not a git command.

8、不提交大文件

# 1. 查看仓库中占用空间最大的文件（可选步骤，帮助你确定需要删除的大文件）
git verify-pack -v .git/objects/pack/pack-*.idx | sort -k 3 -n | tail -5

# 2. 从提交历史中删除大文件（针对你的具体文件路径）
# 注意：这会重写提交历史，影响所有协作者
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch workspace/docker/dev-db/zookeeper/log/version-2/*' --prune-empty --tag-name-filter cat -- --all

# 3. 删除其他超限文件（根据你的错误信息添加）
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch 000000-local-windows-seata2.0.0-nacos2.3.0.tar.xz' --prune-empty --tag-name-filter cat -- --all
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch cloud-demo-main/doc/seata-server/lib/rocksdbjni-8.8.1.jar' --prune-empty --tag-name-filter cat -- --all

# 4. 添加.gitignore规则，防止未来再次提交大文件
cat << EOF >> .gitignore
# 忽略Zookeeper日志文件
workspace/docker/dev-db/zookeeper/log/

# 忽略超大二进制文件
*.tar.xz
*.jar

# 可根据需要添加更多规则
EOF

# 5. 提交变更并强制推送到远程仓库
git add .gitignore
git commit -m "Remove large files and add .gitignore"
git push -f origin main