1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
|
# Git Server Performance Tuning
Guide to optimizing clone and fetch speeds on the git server.
Especially important for large repositories like the Linux kernel.
## Why clones are slow
When a client runs `git clone`, the server has to:
1. Enumerate all objects (commits, trees, blobs) in the repository
2. Compute deltas between similar objects to reduce transfer size
3. Compress everything into a pack stream and send it
This is CPU-intensive, not network-intensive. A 10 Gbit network won't
help if the server is spending all its time packing objects. The
optimizations below reduce the work the server has to do.
## 1. Repack the repository
If the repository was created with `git clone --bare` or `--mirror`,
the pack files may not be optimized. Repacking creates a single,
well-organized pack file that the server can stream directly instead
of rebuilding on every clone.
```bash
cd /var/git/repos/<repo>.git
# Create a single optimized pack with all objects
git repack -a -d -f --threads=0
# Or for maximum compression (slower, but smaller pack file)
git gc --aggressive
```
- `-a` — pack all objects into a single pack
- `-d` — remove old packs after repacking
- `-f` — force recomputation of deltas (even if a pack already exists)
- `--threads=0` — use all available CPU cores
This is the single most impactful optimization. For the Linux kernel
repo, this can take 30+ minutes but the result is permanent.
## 2. Enable bitmap index
Bitmaps let git skip object traversal during clone. Without bitmaps,
git must walk the entire commit graph to find which objects to send.
With bitmaps, it uses a precomputed bitmap to look up objects in O(1).
This can speed up clones by 10x or more on large repositories.
```bash
cd /var/git/repos/<repo>.git
git config repack.writeBitmaps true
git config pack.writeBitmapHashCache true
git repack -a -d -b
```
The `-b` flag writes the bitmap file alongside the pack. The bitmap
must be rebuilt after a repack, so enabling `repack.writeBitmaps`
ensures it's regenerated automatically.
Only one pack file can have a bitmap. If the repo has multiple pack
files, run `git repack -a -d -b` to consolidate into one.
## 3. Enable commit graph
The commit graph file is a precomputed index that speeds up commit
traversal (used during object enumeration on clone/fetch):
```bash
cd /var/git/repos/<repo>.git
git commit-graph write --reachable
```
This creates `.git/objects/info/commit-graph` (or in bare repos,
`objects/info/commit-graph`). Rebuild it after significant pushes.
## 4. Git config tuning
Apply these settings to each repository you want to optimize:
```bash
cd /var/git/repos/<repo>.git
# Packing
git config pack.threads 0 # use all CPU cores for packing
git config pack.deltaCacheSize 256m # cache deltas in memory during packing
git config pack.windowMemory 256m # memory limit for delta search window
git config pack.packSizeLimit 0 # no limit on pack file size (single pack is fastest)
# Compression
git config core.compression 1 # lower compression level (default is 6)
# 1 = fast, larger transfer; 9 = slow, smallest transfer
# for fast networks, lower is better (CPU is the bottleneck)
# Transfer
git config transfer.unpackLimit 1 # keep pushed objects in packs (don't unpack to loose objects)
git config uploadpack.allowFilter true # allow partial clones (client requests only what it needs)
# Protocol
git config protocol.version 2 # protocol v2 is more efficient for negotiation
```
### Compression level tradeoff
`core.compression` controls the zlib compression level (0-9):
- `0` — no compression, fastest CPU, largest transfer
- `1` — minimal compression, very fast, slightly smaller
- `6` — default, good balance
- `9` — maximum compression, slowest CPU, smallest transfer
On fast local networks (1 Gbit+), use `1` — the CPU time saved
outweighs the extra bytes. On slow connections, keep the default or
use `9`.
## 5. Protocol choice
The protocol used for cloning affects speed:
| Protocol | Encryption | Speed | Command |
|----------|-----------|-------|---------|
| `git://` | None | Fastest | `git clone git://swave.lol/repo.git` |
| `ssh://` | Yes (SSH) | Slower (CPU overhead) | `git clone ssh://git@swave.lol/repos/repo.git` |
For large repos on trusted networks, `git://` is faster because
there's no encryption overhead. Use `ssh://` when you need
authentication or are on untrusted networks.
## 6. Client-side speedups
The client can also reduce clone time:
### Shallow clone (only latest commit)
```bash
git clone --depth 1 git://swave.lol/repo.git
```
Downloads only the latest commit and its tree — no history. Ideal for
builds that don't need git log. Much faster and smaller.
### Partial clone (blobless)
```bash
git clone --filter=blob:none git://swave.lol/repo.git
```
Clones commits and trees but skips file contents (blobs). Blobs are
fetched on demand when you checkout files. Requires
`uploadpack.allowFilter=true` on the server.
### Partial clone (treeless)
```bash
git clone --filter=tree:0 git://swave.lol/repo.git
```
Even more aggressive — skips both trees and blobs. Only downloads
commit objects initially. Fastest initial clone, but operations like
`git log --stat` will be slower (fetches trees on demand).
## 7. Container CPU limits
Check that the git-server container isn't CPU-throttled:
```bash
docker inspect git-server --format '{{.HostConfig.CpuQuota}}'
```
If it returns `0`, there's no limit (good). If it returns a number,
the container is CPU-throttled and repacking/cloning will be slow.
Remove the limit or increase it in the compose file:
```yaml
services:
git-server:
# ...
deploy:
resources:
limits:
cpus: '4' # allow 4 CPU cores
```
## Quick setup script
Apply all optimizations to a single repository:
```bash
#!/bin/bash
# Usage: bash optimize-repo.sh /var/git/repos/<repo>.git
REPO=$1
if [ -z "$REPO" ]; then
echo "Usage: $0 /path/to/repo.git"
exit 1
fi
cd "$REPO" || exit 1
echo "Configuring git settings..."
git config pack.threads 0
git config pack.deltaCacheSize 256m
git config pack.windowMemory 256m
git config pack.packSizeLimit 0
git config core.compression 1
git config transfer.unpackLimit 1
git config uploadpack.allowFilter true
git config protocol.version 2
git config repack.writeBitmaps true
git config pack.writeBitmapHashCache true
echo "Writing commit graph..."
git commit-graph write --reachable
echo "Repacking with bitmaps (this may take a while)..."
git repack -a -d -f -b --threads=0
echo "Done. Repository optimized."
```
|
