Openhub
Sign in Sign up
samjeston / openhub Public
9aa3c45
Switch branches/tags
Branches
main
Sam Jeston Working vertical 9aa3c45 · History
149 lines· 9.1 KB ·Markdown
Preview Code
Raw
1# Operations
2
3This page covers running an Openhub instance day to day: first start, reverse proxies, backups, upgrades, administration without a UI, and troubleshooting. The [project README](../README.md) has installation steps and the full [configuration table](../README.md#configuration).
4
5## First start checklist
6
71. **Set `OPENHUB_PUBLIC_URL`** to the exact URL people will use, such as `https://git.example.com`. Links, HTTP clone URLs, push hints, the cross-origin check and the cookie `Secure` flag all depend on it.
82. **Set the SSH host and port** with `OPENHUB_SSH_HOST` and `OPENHUB_SSH_PORT` if clients reach SSH on a different host or port from the one Openhub listens on. Docker port mapping is the usual reason.
93. **Start the server and register the administrator account straight away.** The first account becomes site administrator (see [Security](security.md#known-gaps)).
104. **Add an SSH key** in **Settings → SSH keys**, then check it with the command shown there (`ssh -T -p 2222 git@host`).
115. **Set `OPENHUB_DISABLE_REGISTRATION=true`** once everyone who needs an account has one, unless you intend to run an open instance.
126. **Note the SSH host key fingerprint** from the startup log (`SSH host key fingerprint: SHA256:…`) so users can verify it on first connect.
13
14## Logs
15
16Openhub logs to stdout through `tracing`. Set the level with `RUST_LOG`:
17
18| `RUST_LOG` | Shows |
19| --- | --- |
20| unset (`openhub=info`) | Startup, SSH sign-ins, git commands run over SSH, and errors |
21| `openhub=debug` | Plus SSH session errors |
22| `openhub=info,tower_http=debug` | Plus one line for every HTTP request and response |
23
24Internal errors (HTTP 500) are logged with their full error chain. The user only sees a generic message.
25
26## Reverse proxy
27
28Openhub speaks plain HTTP. Put a TLS-terminating proxy in front of it for the web port only; SSH goes straight to the SSH port.
29
30**Caddy:**
31
32```
33git.example.com {
34 reverse_proxy 127.0.0.1:3000
35}
36```
37
38**nginx:**
39
40```nginx
41server {
42 listen 443 ssl;
43 server_name git.example.com;
44 # ssl_certificate …
45
46 client_max_body_size 64m; # git fetch negotiation for large repositories
47 location / {
48 proxy_pass http://127.0.0.1:3000;
49 proxy_set_header Host $host;
50 proxy_buffering off; # stream clones and archive downloads
51 }
52}
53```
54
55Then set `OPENHUB_PUBLIC_URL=https://git.example.com`. The cross-origin check accepts form posts whose `Origin` matches either the public URL or the `Host` header, so proxies that rewrite `Host` still work.
56
57If you rate-limit at the proxy (recommended, see [Security](security.md#hardening-advice)), cover `POST /login`, `POST /register` and `/*/*/info/refs`.
58
59## Backups
60
61Everything lives in the data directory (see [Architecture](architecture.md#data-directory)):
62
63| Item | How to back it up |
64| --- | --- |
65| `openhub.db` | While running: `sqlite3 openhub.db ".backup backup.db"`, which takes a consistent snapshot. When stopped: copy the file. Don't copy a live database file without its `-wal` file |
66| `repos/` | Copying bare repositories while a push is in progress can capture a half-written state. Stop the server, use a filesystem snapshot (ZFS, Btrfs or LVM), or accept that the latest push may need repeating |
67| `ssh_host_ed25519_key` | Copy it. Without it, every client warns about a changed host key after a restore |
68
69**To restore:** stop Openhub, put all three items back in one data directory, and start it again. The database and `repos/` must come from the same point in time. A repository row without its directory, or a directory without its row, causes errors (see [Data model → Invariants](data-model.md#invariants)).
70
71## Upgrades
72
731. **Back up** the data directory.
742. **Build or pull the new version.** From source, run `git pull && cargo build --release`. With Docker, run `docker compose build && docker compose up -d`.
753. **Start it.** Pending database migrations run automatically at startup, before the servers listen.
76
77Migrations only go forward. To roll back, restore the backup taken in step 1 and run the old version.
78
79## Repository maintenance
80
81Openhub never runs `git gc`. `git receive-pack` runs `git gc --auto` after pushes by default, which keeps most repositories tidy. To compact a repository by hand, or to clean up objects left by failed web merges:
82
83```sh
84git --git-dir data/repos/<id>.git gc
85```
86
87To map a repository to its ID:
88
89```sh
90sqlite3 -readonly data/openhub.db \
91 "SELECT r.id FROM repositories r JOIN users u ON u.id = r.owner_id WHERE u.username = 'alice' AND r.name = 'project';"
92```
93
94Run maintenance as the same user that runs Openhub, so file ownership stays correct. In Docker:
95
96```sh
97docker compose exec openhub git --git-dir /data/repos/<id>.git gc
98```
99
100## Administration without a UI
101
102Openhub has no admin pages. Do these tasks with `sqlite3` against `openhub.db`, after taking a backup. Changes take effect immediately, and the server doesn't need restarting.
103
104```sql
105-- Make someone a site administrator (or set to 0 to remove)
106UPDATE users SET is_admin = 1 WHERE username = 'bob';
107
108-- Sign a user out everywhere
109DELETE FROM sessions WHERE user_id = (SELECT id FROM users WHERE username = 'bob');
110
111-- Revoke all of a user's SSH keys (stops them pushing)
112DELETE FROM ssh_keys WHERE user_id = (SELECT id FROM users WHERE username = 'bob');
113
114-- Transfer a repository to another user (the new owner must not already have a repository with that name)
115UPDATE repositories SET owner_id = (SELECT id FROM users WHERE username = 'carol')
116 WHERE id = 42;
117```
118
119**Resetting a forgotten password.** There's no reset flow. You can set a new hash made by any Argon2id tool that writes the standard PHC string format. For example, with the `argon2` command-line tool (Debian package `argon2`):
120
121```sh
122printf '%s' 'new-password' | argon2 "$(openssl rand -hex 16)" -id -e
123# → $argon2id$v=19$m=4096,t=3,p=1$…
124```
125
126```sql
127UPDATE users SET password_hash = '$argon2id$v=19$…' WHERE username = 'bob';
128```
129
130**Deleting a user.** `DELETE FROM users …` removes their sessions, keys, repositories and collaborations, through cascades. It fails if they authored or merged pull requests, or wrote pull request events, in other users' repositories (see [Data model → Deleting rows](data-model.md#deleting-rows)). Deleting their rows also doesn't remove their repositories' directories under `repos/`. Delete each repository in the web interface first, while signed in as an administrator.
131
132## Troubleshooting
133
134| Symptom | Likely cause | Fix |
135| --- | --- | --- |
136| Forms fail with **"Cross-origin request rejected"** | The browser's origin matches neither `OPENHUB_PUBLIC_URL` nor the `Host` header the server receives | Set the public URL to exactly what's in the address bar, including scheme and port |
137| Startup fails with **"could not run `git`"** | git isn't installed or isn't on `PATH` | Install git, or set `OPENHUB_GIT_BIN` |
138| Startup fails with a **migration checksum or "previously applied but has been modified"** error | An already-applied file in `migrations/` was edited | Restore the original file and put the change in a new migration |
139| **"database is locked"** errors | Another process held a write lock for more than 5 seconds, often a `sqlite3` shell with an open transaction | Close the other connection. Open shells with `-readonly` for inspection |
140| `git push` says **"Permission denied (publickey)"** | The key isn't registered, or ssh offered a different key | Add the key in Settings. Test with `ssh -i ~/.ssh/key -o IdentitiesOnly=yes -T -p 2222 git@host` |
141| `git push` says **"ERROR: Repository not found."** | The path is wrong, or the account can't read the repository | Check `owner/name` in the remote URL, and that the key belongs to the expected account (`ssh -T` greets you by username) |
142| `git push` says **"You do not have permission to push to …"** | The account can read but isn't the owner or a collaborator | The owner adds the account under repository **Settings → Collaborators** |
143| ssh warns **"REMOTE HOST IDENTIFICATION HAS CHANGED"** | `ssh_host_ed25519_key` was lost or replaced | Restore the key from backup. If it really is gone, users remove the old entry with `ssh-keygen -R '[host]:2222'` |
144| Push to `refs/pull/…` fails with **"deny updating a hidden ref"** | Expected: Openhub manages those refs | Push to a branch instead |
145| `git push` over HTTP fails with **403** | HTTP is read-only by design | Use the SSH remote URL |
146| Clone URLs show **localhost** or the wrong port | `OPENHUB_PUBLIC_URL`, `OPENHUB_SSH_HOST` or `OPENHUB_SSH_PORT` isn't set | Set them and restart |
147| Large HTTP clones fail with **413** | A proxy limits request bodies below the 64 MB Openhub allows for fetch negotiation | Raise the proxy limit (`client_max_body_size` in nginx) |
148| A repository page shows **"Something went wrong"** | An internal error, such as a missing `repos/<id>.git` directory | Check the log for `internal error:`. Restore the directory from backup, or delete the repository |
149| A new repository unexpectedly shows **old content** | A previous repository with the same ID left its directory behind (deletion logs `could not remove`) | Delete the new repository, remove the stale `repos/<id>.git`, then create it again |