# Git Server Performance Tuning Guide to optimizing clone and fetch speeds on the git server. Especially important for large repositories like the Linux kernel. ## Why clones are slow When a client runs `git clone`, the server has to: 1. Enumerate all objects (commits, trees, blobs) in the repository 2. Compute deltas between similar objects to reduce transfer size 3. Compress everything into a pack stream and send it This is CPU-intensive, not network-intensive. A 10 Gbit network won't help if the server is spending all its time packing objects. The optimizations below reduce the work the server has to do. ## 1. Repack the repository If the repository was created with `git clone --bare` or `--mirror`, the pack files may not be optimized. Repacking creates a single, well-organized pack file that the server can stream directly instead of rebuilding on every clone. ```bash cd /var/git/repos/.git # Create a single optimized pack with all objects git repack -a -d -f --threads=0 # Or for maximum compression (slower, but smaller pack file) git gc --aggressive ``` - `-a` — pack all objects into a single pack - `-d` — remove old packs after repacking - `-f` — force recomputation of deltas (even if a pack already exists) - `--threads=0` — use all available CPU cores This is the single most impactful optimization. For the Linux kernel repo, this can take 30+ minutes but the result is permanent. ## 2. Enable bitmap index Bitmaps let git skip object traversal during clone. Without bitmaps, git must walk the entire commit graph to find which objects to send. With bitmaps, it uses a precomputed bitmap to look up objects in O(1). This can speed up clones by 10x or more on large repositories. ```bash cd /var/git/repos/.git git config repack.writeBitmaps true git config pack.writeBitmapHashCache true git repack -a -d -b ``` The `-b` flag writes the bitmap file alongside the pack. The bitmap must be rebuilt after a repack, so enabling `repack.writeBitmaps` ensures it's regenerated automatically. Only one pack file can have a bitmap. If the repo has multiple pack files, run `git repack -a -d -b` to consolidate into one. ## 3. Enable commit graph The commit graph file is a precomputed index that speeds up commit traversal (used during object enumeration on clone/fetch): ```bash cd /var/git/repos/.git git commit-graph write --reachable ``` This creates `.git/objects/info/commit-graph` (or in bare repos, `objects/info/commit-graph`). Rebuild it after significant pushes. ## 4. Git config tuning Apply these settings to each repository you want to optimize: ```bash cd /var/git/repos/.git # Packing git config pack.threads 0 # use all CPU cores for packing git config pack.deltaCacheSize 256m # cache deltas in memory during packing git config pack.windowMemory 256m # memory limit for delta search window git config pack.packSizeLimit 0 # no limit on pack file size (single pack is fastest) # Compression git config core.compression 1 # lower compression level (default is 6) # 1 = fast, larger transfer; 9 = slow, smallest transfer # for fast networks, lower is better (CPU is the bottleneck) # Transfer git config transfer.unpackLimit 1 # keep pushed objects in packs (don't unpack to loose objects) git config uploadpack.allowFilter true # allow partial clones (client requests only what it needs) # Protocol git config protocol.version 2 # protocol v2 is more efficient for negotiation ``` ### Compression level tradeoff `core.compression` controls the zlib compression level (0-9): - `0` — no compression, fastest CPU, largest transfer - `1` — minimal compression, very fast, slightly smaller - `6` — default, good balance - `9` — maximum compression, slowest CPU, smallest transfer On fast local networks (1 Gbit+), use `1` — the CPU time saved outweighs the extra bytes. On slow connections, keep the default or use `9`. ## 5. Protocol choice The protocol used for cloning affects speed: | Protocol | Encryption | Speed | Command | |----------|-----------|-------|---------| | `git://` | None | Fastest | `git clone git://swave.lol/repo.git` | | `ssh://` | Yes (SSH) | Slower (CPU overhead) | `git clone ssh://git@swave.lol/repos/repo.git` | For large repos on trusted networks, `git://` is faster because there's no encryption overhead. Use `ssh://` when you need authentication or are on untrusted networks. ## 6. Client-side speedups The client can also reduce clone time: ### Shallow clone (only latest commit) ```bash git clone --depth 1 git://swave.lol/repo.git ``` Downloads only the latest commit and its tree — no history. Ideal for builds that don't need git log. Much faster and smaller. ### Partial clone (blobless) ```bash git clone --filter=blob:none git://swave.lol/repo.git ``` Clones commits and trees but skips file contents (blobs). Blobs are fetched on demand when you checkout files. Requires `uploadpack.allowFilter=true` on the server. ### Partial clone (treeless) ```bash git clone --filter=tree:0 git://swave.lol/repo.git ``` Even more aggressive — skips both trees and blobs. Only downloads commit objects initially. Fastest initial clone, but operations like `git log --stat` will be slower (fetches trees on demand). ## 7. Container CPU limits Check that the git-server container isn't CPU-throttled: ```bash docker inspect git-server --format '{{.HostConfig.CpuQuota}}' ``` If it returns `0`, there's no limit (good). If it returns a number, the container is CPU-throttled and repacking/cloning will be slow. Remove the limit or increase it in the compose file: ```yaml services: git-server: # ... deploy: resources: limits: cpus: '4' # allow 4 CPU cores ``` ## Quick setup script Apply all optimizations to a single repository: ```bash #!/bin/bash # Usage: bash optimize-repo.sh /var/git/repos/.git REPO=$1 if [ -z "$REPO" ]; then echo "Usage: $0 /path/to/repo.git" exit 1 fi cd "$REPO" || exit 1 echo "Configuring git settings..." git config pack.threads 0 git config pack.deltaCacheSize 256m git config pack.windowMemory 256m git config pack.packSizeLimit 0 git config core.compression 1 git config transfer.unpackLimit 1 git config uploadpack.allowFilter true git config protocol.version 2 git config repack.writeBitmaps true git config pack.writeBitmapHashCache true echo "Writing commit graph..." git commit-graph write --reachable echo "Repacking with bitmaps (this may take a while)..." git repack -a -d -f -b --threads=0 echo "Done. Repository optimized." ```