Sam Jeston
History
| 1 | # Git transport |
| 2 | |
| 3 | This page explains how git clients talk to Openhub. Pushes are accepted only over SSH, by a built-in SSH server. Clones and fetches work over SSH and over read-only smart HTTP. In every case the git protocol itself is handled by the installed `git` program, and Openhub decides who may run it against which repository. |
| 4 | |
| 5 | **Source files:** |
| 6 | - `src/ssh.rs`: the SSH server. |
| 7 | - `src/web/git_http.rs`: smart HTTP. |
| 8 | - `src/sync.rs`: bookkeeping after a push. |
| 9 | - `src/web/code.rs` (`archive`): downloads. |
| 10 | - `src/config.rs`: clone URLs. |
| 11 | |
| 12 | ## Channels at a glance |
| 13 | |
| 14 | | Channel | Operations | Authentication | Runs | |
| 15 | | --- | --- | --- | --- | |
| 16 | | SSH (`--ssh-addr`, default port 2222) | clone, fetch, push | SSH public key | `git upload-pack` or `git receive-pack` | |
| 17 | | Smart HTTP (`--http-addr`, default port 3000) | clone, fetch | None for public repositories; Basic auth for private ones | `git upload-pack --stateless-rpc` | |
| 18 | | HTTP archive download | ZIP or tar.gz of a branch, tag or commit | Browser session, or anonymous for public repositories | `git archive` | |
| 19 | |
| 20 | ## Clone URLs |
| 21 | |
| 22 | `Config` builds the URLs shown in the **Clone** dropdown, in push hints and in the documentation pages: |
| 23 | |
| 24 | | URL | Built by | Example | |
| 25 | | --- | --- | --- | |
| 26 | | SSH, port 22 | `ssh_clone_url` | `[email protected]:alice/project.git` | |
| 27 | | SSH, any other port | `ssh_clone_url` | `ssh://[email protected]:2222/alice/project.git` | |
| 28 | | HTTP | `http_clone_url` | `https://git.example.com/alice/project.git` | |
| 29 | | SSH test command | `ssh_test_command` | `ssh -T -p 2222 [email protected]` | |
| 30 | |
| 31 | The SSH host is `--ssh-host`, or failing that the host of `--public-url`. The SSH port is `--ssh-port`, or failing that the port of `--ssh-addr`. Set both when Openhub runs behind port forwarding, for example container port 2222 published as host port 22. |
| 32 | |
| 33 | ## SSH server |
| 34 | |
| 35 | The server is built on russh (`ssh::serve`) and listens on `--ssh-addr`. |
| 36 | |
| 37 | ### Host key |
| 38 | |
| 39 | On first start, Openhub generates an Ed25519 host key at `<data>/ssh_host_ed25519_key`, in OpenSSH format with mode `0600`, and reuses it on every later start. Its SHA-256 fingerprint is logged at startup so users can check it. If the key is lost, every client sees a "host key changed" warning, so keep it in backups. |
| 40 | |
| 41 | ### Server settings |
| 42 | |
| 43 | | Setting | Value | |
| 44 | | --- | --- | |
| 45 | | Version string | `SSH-2.0-Openhub` | |
| 46 | | Authentication methods | `publickey` only. No passwords or keyboard-interactive | |
| 47 | | Delay after a rejected authentication | 300 ms (none for the first "none" probe) | |
| 48 | | Inactivity timeout | 1 hour | |
| 49 | | Keepalive interval | 60 s | |
| 50 | |
| 51 | ### Authentication |
| 52 | |
| 53 | 1. **`auth_publickey_offered`.** The client offers a public key, and Openhub accepts the offer only if its fingerprint is in `ssh_keys`. Clients therefore only sign with keys that will work, and move on to the next key otherwise. |
| 54 | 2. **`auth_publickey`.** russh has verified the client's signature. Openhub loads the owning user, updates `last_used_at` and logs the sign-in with the peer address. |
| 55 | |
| 56 | The SSH username is ignored. Anything works, but clone URLs use `git`. |
| 57 | |
| 58 | ### Channel requests |
| 59 | |
| 60 | | Request | Response | |
| 61 | | --- | --- | |
| 62 | | `env GIT_PROTOCOL=…` | Stored and passed to git, which enables protocol v2. Other variables are refused | |
| 63 | | `pty-req` | Refused | |
| 64 | | `shell` (for example `ssh -T`) | Prints `Hi <user>! You've successfully authenticated, but Openhub does not provide shell access.` and exits with status 1 | |
| 65 | | `subsystem` (for example `sftp`) | Refused | |
| 66 | | `exec` | Parsed as a git command. See below | |
| 67 | |
| 68 | ### Command parsing |
| 69 | |
| 70 | `parse_command` accepts exactly these forms, which are what git clients send: |
| 71 | |
| 72 | ``` |
| 73 | git-upload-pack '<path>' |
| 74 | git-receive-pack '<path>' |
| 75 | git upload-pack '<path>' |
| 76 | git receive-pack '<path>' |
| 77 | ``` |
| 78 | |
| 79 | It then processes the path: |
| 80 | |
| 81 | 1. Removes surrounding single or double quotes. |
| 82 | 2. Rejects the path if it still contains `'`, `"`, `\` or a NUL byte. |
| 83 | 3. Trims `/` from both ends, then removes a trailing `.git`. |
| 84 | 4. Requires exactly `owner/name`. |
| 85 | |
| 86 | Anything else, including `git-upload-archive` (that is, `git archive --remote`), gets `ERROR: Openhub only supports git fetch and push over SSH.` and exit status 1. |
| 87 | |
| 88 | ### Authorisation and execution |
| 89 | |
| 90 | `SshSession::exec_request` does the following: |
| 91 | |
| 92 | 1. Looks up the repository with `db::repo_by_name` and calls `AppState::access`. |
| 93 | 2. Checks permissions: |
| 94 | - No read access, or no such repository: `ERROR: Repository not found.` |
| 95 | - `receive-pack` without write access: `ERROR: You do not have permission to push to owner/name.` |
| 96 | 3. For `receive-pack`, records every branch tip (`branch_tips`), so push hints can tell what changed. |
| 97 | 4. Spawns `<git-bin> upload-pack|receive-pack <data>/repos/<id>.git`: |
| 98 | - The path comes from the database ID, never from the client's string. |
| 99 | - No shell is involved. |
| 100 | - stdin, stdout and stderr are piped. |
| 101 | - `kill_on_drop` is set. |
| 102 | - `GIT_PROTOCOL` is passed through when the client sent it. |
| 103 | 5. Relays data: |
| 104 | |
| 105 | ``` |
| 106 | client ──SSH data──▶ SshSession::data ──unbounded mpsc──▶ task ──▶ git stdin |
| 107 | client ◀─SSH data─── handle.data ◀── forward() ◀───────────── git stdout |
| 108 | client ◀─ext. data── handle.extended_data(1) ◀── forward() ◀────── git stderr |
| 109 | client EOF ─────────▶ channel_eof drops the sender ──────────────▶ git stdin closed |
| 110 | ``` |
| 111 | |
| 112 | 6. After git exits, sends git's exit status, then EOF, then closes the channel. |
| 113 | |
| 114 | **Why the `Channel` object is dropped.** `channel_open_session` drops russh's `Channel` handle on purpose and receives data through the `data` callback instead. Holding the handle without reading from it fills russh's per-channel buffer and deadlocks the session. The trade-off is that the queue to git's stdin is unbounded (see [Security](security.md#known-gaps)). |
| 115 | |
| 116 | ### Protecting `refs/pull` |
| 117 | |
| 118 | Every repository is created with `receive.hideRefs=refs/pull`. `git receive-pack` then neither advertises `refs/pull/*` nor accepts updates to it: a push there fails with `deny updating a hidden ref`. `upload-pack` still advertises these refs, so clients can fetch them. |
| 119 | |
| 120 | ## After a push |
| 121 | |
| 122 | When `receive-pack` exits with status 0, two things happen before the exit status is sent: |
| 123 | |
| 124 | 1. **`sync::after_push`** runs: |
| 125 | - `db::touch_pushed` sets `pushed_at`. |
| 126 | - `git::ensure_head` makes sure `HEAD` names an existing branch. It prefers the current default branch, then `main`, then `master`, then the first branch alphabetically. If it had to pick a different branch, `repositories.default_branch` is updated. That's how a repository's first push of a non-`main` branch becomes its default. |
| 127 | - `sync::refresh_pulls` re-evaluates every open pull request in the repository. It records new heads and marks pull requests as merged when their head is now in the base (see [Pull requests](pull-requests.md#tracking-open-pull-requests)). |
| 128 | 2. **Push hints** (`push_hints`): for each non-default branch whose tip changed in this push, Openhub prints one of the following on stderr. The client shows them prefixed with `remote:`. |
| 129 | |
| 130 | ``` |
| 131 | Create a pull request for 'feature' on Openhub by visiting: |
| 132 | https://git.example.com/alice/project/compare/main...feature |
| 133 | ``` |
| 134 | |
| 135 | ``` |
| 136 | View pull request for 'feature': |
| 137 | https://git.example.com/alice/project/pull/3 |
| 138 | ``` |
| 139 | |
| 140 | Deleted branches and the default branch get no hint. |
| 141 | |
| 142 | Errors in `after_push` are logged and don't fail the push, which by then has already been accepted. |
| 143 | |
| 144 | ## Smart HTTP (read-only) |
| 145 | |
| 146 | Routes, with `{repo}` accepting an optional `.git` suffix: |
| 147 | |
| 148 | | Request | Result | |
| 149 | | --- | --- | |
| 150 | | `GET /{owner}/{repo}/info/refs?service=git-upload-pack` | Ref advertisement from `git upload-pack --stateless-rpc --advertise-refs`. Unless protocol v2 was requested, it's prefixed with the `# service=git-upload-pack` pkt-line and a flush packet | |
| 151 | | `POST /{owner}/{repo}/git-upload-pack` | Request body (gzip decoded if `Content-Encoding: gzip`) piped to `git upload-pack --stateless-rpc`; the output is streamed back | |
| 152 | | `GET …/info/refs?service=git-receive-pack` | 403: "Pushing over HTTP is not supported. Add an SSH key in your Openhub settings and push over SSH." | |
| 153 | | `POST …/git-receive-pack` | 403 | |
| 154 | | `GET …/info/refs` with no `service` (dumb HTTP) | 403: only the smart protocol is supported | |
| 155 | |
| 156 | Details: |
| 157 | |
| 158 | - **Authorisation** (`git_http::authorize`): |
| 159 | - A missing repository gets 404. |
| 160 | - A public repository needs no credentials. |
| 161 | - A private repository needs Basic auth. Missing or wrong credentials get 401 with `WWW-Authenticate: Basic realm="Openhub"`. Valid credentials without read access get 404. |
| 162 | - **Protocol v2.** The `Git-Protocol` header is passed through as `GIT_PROTOCOL` when it contains only `[A-Za-z0-9=:.]`. |
| 163 | - **Body limit.** The `git-upload-pack` route accepts request bodies up to 64 MB (`GIT_BODY_LIMIT` in `src/web/mod.rs`). Negotiation for large repositories lists many commits. Other routes keep axum's 2 MB default. |
| 164 | - **Middleware.** These paths skip the session lookup and the cross-origin check, because git clients send neither cookies nor `Origin` (see `ctx::is_git_request`). |
| 165 | - **Errors.** git's stderr is discarded. Failures are logged and returned as 500 with a short message. |
| 166 | |
| 167 | ## Archive downloads |
| 168 | |
| 169 | `GET /{owner}/{repo}/archive/{ref}.zip` or `.tar.gz` (`code::archive`): |
| 170 | |
| 171 | 1. Checks read access with `load_repo`. |
| 172 | 2. Resolves `{ref}` with `git::resolve`, which accepts a branch, tag or commit SHA. The ref must not include a path. |
| 173 | 3. Runs `git --git-dir <path> archive --format=zip|tar.gz --prefix=<repo>-<ref>/ <commit-sha>` and streams stdout to the client as `attachment; filename="<repo>-<ref>.<ext>"`. Any `/` in the ref becomes `-`. |
| 174 | |
| 175 | `git archive` honours the repository's own `.gitattributes`, including `export-ignore` and `export-subst`. See [Security](security.md) for why that matters. |
| 176 | |
| 177 | ## Protocol features |
| 178 | |
| 179 | The installed git provides the protocol support: |
| 180 | - Protocol v0, v1 and v2. |
| 181 | - Shallow clones (`--depth`). |
| 182 | - Side-band progress. |
| 183 | - Atomic pushes. |
| 184 | |
| 185 | Openhub doesn't set `uploadpack.allowFilter`, so partial clone filters (`--filter=blob:none`) are ignored by the server. It doesn't install hooks, and nothing in the protocol can write a repository's config or hooks. Git LFS isn't supported. |
| 186 | |
| 187 | ## When you change this |
| 188 | |
| 189 | - **New SSH command:** add it to `Service` and `parse_command`, together with its unit test. Decide which `Access` level it needs. Extend the smoke test. |
| 190 | - **New post-push behaviour:** put it in `sync::after_push`, so web merges and pushes share it where it applies. |
| 191 | - **New git configuration for hosted repositories:** add it to `git::configure`. Existing repositories don't pick it up automatically, so add a startup step if they need it. |
| 192 | - **Changes to HTTP authentication:** keep private repositories indistinguishable from missing ones, and keep pushes on SSH unless that is decided otherwise. |