Sam Jeston
History
| 1 | # Architecture |
| 2 | |
| 3 | Openhub is one Rust process. It serves the web interface and git over HTTP on one port, and git over SSH on a second port. State is kept in two places: |
| 4 | |
| 5 | - A SQLite database, for accounts, keys, repository metadata and pull requests. |
| 6 | - A directory of bare git repositories, for all code and history. |
| 7 | |
| 8 | There are no background workers, queues or caches. Everything happens in response to a web request or an SSH session. |
| 9 | |
| 10 | ## Components |
| 11 | |
| 12 | ``` |
| 13 | browser git over HTTP git over SSH |
| 14 | │ │ │ |
| 15 | ▼ ▼ ▼ |
| 16 | ┌───────────────────────────┐ ┌───────────────────────────┐ |
| 17 | │ Web server (axum) │ │ SSH server (russh) │ |
| 18 | │ src/web/ │ │ src/ssh.rs │ |
| 19 | └─────────────┬─────────────┘ └─────────────┬─────────────┘ |
| 20 | │ │ |
| 21 | └───────────────┬───────────────┘ |
| 22 | ▼ |
| 23 | ┌───────────────────────────────────────────────────────────┐ |
| 24 | │ Shared core │ |
| 25 | │ AppState (src/state.rs): Config, SQLite pool, access() │ |
| 26 | │ libgit2, in process (src/git.rs, src/sync.rs) │ |
| 27 | │ git child processes: upload-pack, receive-pack, archive │ |
| 28 | └─────────────┬───────────────────────────────┬─────────────┘ |
| 29 | ▼ ▼ |
| 30 | <data>/openhub.db <data>/repos/<id>.git |
| 31 | SQLite database bare repositories |
| 32 | ``` |
| 33 | |
| 34 | | Module | Responsibility | |
| 35 | | --- | --- | |
| 36 | | `src/main.rs` | Startup and shutdown | |
| 37 | | `src/config.rs` | `Config`: every command-line flag and `OPENHUB_*` variable, plus path and clone-URL helpers | |
| 38 | | `src/state.rs` | `AppState`, shared by both servers, and the permission model `AppState::access` | |
| 39 | | `src/db.rs` | The connection pool, migrations, row types and every SQL query | |
| 40 | | `src/git.rs` | Every libgit2 operation: refs, trees, blobs, history, diffs, mergeability, merging | |
| 41 | | `src/sync.rs` | Keeps the database in step with refs after they change | |
| 42 | | `src/ssh.rs` | The SSH server: key authentication, command parsing, running git, push hints | |
| 43 | | `src/web/` | The HTTP router, middleware and page handlers, plus smart HTTP and static assets | |
| 44 | | `src/highlight.rs` | Syntax highlighting to `tok-*` CSS classes | |
| 45 | | `src/markdown.rs` | Markdown to sanitised HTML, with relative-link rewriting | |
| 46 | | `src/validate.rs` | Rules for usernames, repository names, emails, passwords and SSH public keys | |
| 47 | | `src/icons.rs` | Inline SVG icons | |
| 48 | | `templates/` | askama HTML templates, compiled into the binary | |
| 49 | | `static/` | `style.css`, `app.js` and `favicon.svg`, embedded into the binary with `include_str!` | |
| 50 | | `migrations/` | SQL migrations, embedded and applied at startup | |
| 51 | |
| 52 | ## Startup |
| 53 | |
| 54 | `main` in `src/main.rs`: |
| 55 | |
| 56 | 1. Starts logging. The level comes from `RUST_LOG` and defaults to `openhub=info`. |
| 57 | 2. Parses `Config` from flags and environment variables, then calls `Config::normalize`, which strips trailing slashes from the public URL. |
| 58 | 3. Creates `<data>/repos` if it is missing. |
| 59 | 4. Runs `git --version`. If `git` can't be run, startup fails. |
| 60 | 5. Opens the database with `db::connect`: |
| 61 | - Creates the database file if needed. |
| 62 | - Turns on WAL mode, `synchronous=NORMAL` and foreign keys, with a 5-second busy timeout and a pool of up to 8 connections. |
| 63 | - Applies any pending migrations. |
| 64 | 6. Builds `AppState` and runs `web::serve` and `ssh::serve` together with `tokio::select!`. |
| 65 | |
| 66 | If either server returns an error, the process exits. Ctrl-C shuts it down. On its first start, the SSH server generates `<data>/ssh_host_ed25519_key` (see [Git transport](git-transport.md#host-key)). |
| 67 | |
| 68 | ## Data directory |
| 69 | |
| 70 | ``` |
| 71 | <data-dir>/ default ./data, or /data in Docker |
| 72 | ├── openhub.db SQLite database (plus openhub.db-wal and openhub.db-shm while running) |
| 73 | ├── ssh_host_ed25519_key SSH host private key (mode 0600) |
| 74 | └── repos/ |
| 75 | ├── 1.git/ bare repository for repositories.id = 1 |
| 76 | └── 2.git/ |
| 77 | ``` |
| 78 | |
| 79 | A whole installation is this directory plus the binary. Back it up as a unit (see [Operations](operations.md#backups)). |
| 80 | |
| 81 | ## libgit2 and the git executable |
| 82 | |
| 83 | Openhub uses two git implementations, each for the jobs it handles best: |
| 84 | |
| 85 | | Work | Implementation | Where | |
| 86 | | --- | --- | --- | |
| 87 | | Serving fetches and clones (SSH and HTTP) | `git upload-pack`, run as a child process | `src/ssh.rs`, `src/web/git_http.rs` | |
| 88 | | Receiving pushes (SSH only) | `git receive-pack`, run as a child process | `src/ssh.rs` | |
| 89 | | ZIP and tar.gz downloads | `git archive`, run as a child process | `code::archive` in `src/web/code.rs` | |
| 90 | | Browsing trees, files, history, branches and tags | libgit2, in process | `src/git.rs` | |
| 91 | | Diffs, mergeability checks and web merges | libgit2, in process | `src/git.rs` | |
| 92 | | Creating repositories and initial commits | libgit2, in process | `git::init_bare`, `git::initial_commit` | |
| 93 | | Updating refs (`HEAD`, `refs/pull/*`, deleting branches) | libgit2, in process | `src/git.rs`, `src/sync.rs` | |
| 94 | |
| 95 | The wire protocol is left to the installed `git`, so protocol v2, shallow clones and future protocol changes come from the system package. Everything Openhub needs to inspect or compute is done with libgit2, which avoids parsing command output. |
| 96 | |
| 97 | libgit2 is compiled into the binary from bundled C sources (`git2` crate, default features off, so no OpenSSL or libssh2). The `git` executable is a runtime dependency, configurable with `--git-bin`. |
| 98 | |
| 99 | ## Concurrency model |
| 100 | |
| 101 | - **Runtime.** Openhub runs on a multi-threaded tokio runtime (`#[tokio::main]`). |
| 102 | - **libgit2 calls are synchronous.** Web handlers run them inside `web::blocking`, which wraps `spawn_blocking`, so they never stall the async workers. A `git2::Repository` is opened per request and never shared between threads. |
| 103 | - **Database access is async** through the sqlx pool. Handlers normally: |
| 104 | 1. Load the repository and check access (async). |
| 105 | 2. Do the git work (blocking). |
| 106 | 3. Render the page. |
| 107 | - **Each SSH connection** is its own russh handler (`SshSession`). Each git child process gets tasks that copy data between the SSH channel and the process's stdin, stdout and stderr. |
| 108 | - **Child processes** are started with `tokio::process::Command` and `kill_on_drop(true)`, so a dropped connection kills its git process. |
| 109 | - **Concurrent ref updates** are serialised by git's own ref locking. Web merges use a compare-and-swap ref update, so a merge racing a push fails cleanly instead of overwriting it (see [Pull requests](pull-requests.md#merging-in-the-web-interface)). |
| 110 | |
| 111 | ## How the main flows work |
| 112 | |
| 113 | ### Viewing a file in the browser |
| 114 | |
| 115 | 1. The axum router matches `/{owner}/{repo}/blob/{*rest}` and calls `code::blob`. |
| 116 | 2. `ctx::middleware` has already loaded the signed-in user from the session cookie into `Ctx`. |
| 117 | 3. `repo::load_repo` looks up the repository by owner and name and calls `AppState::access`. If the user can't read it, the handler returns 404. |
| 118 | 4. Inside `web::blocking`: |
| 119 | - `git::resolve` splits `rest` into a ref and a path. |
| 120 | - `git::read_blob` reads the file. |
| 121 | - `highlight::highlight_lines` or `markdown::render` turns it into HTML. |
| 122 | 5. The askama template renders. The middleware adds the security headers. |
| 123 | |
| 124 | ### Pushing over SSH |
| 125 | |
| 126 | 1. russh authenticates the client key against `ssh_keys.fingerprint`. |
| 127 | 2. The client runs `git-receive-pack '/alice/project.git'`. `parse_command` extracts the service, owner and name. |
| 128 | 3. Openhub looks up the repository and checks write access. |
| 129 | 4. It records the branch tips, then spawns `git receive-pack <data>/repos/<id>.git` and relays data between the client and the process. |
| 130 | 5. When the process exits successfully: |
| 131 | - `sync::after_push` updates `pushed_at`, fixes `HEAD` and refreshes open pull requests. |
| 132 | - Push hints are printed as `remote:` lines. |
| 133 | |
| 134 | ### Merging a pull request in the browser |
| 135 | |
| 136 | 1. `pulls::merge` checks write access and re-checks mergeability. |
| 137 | 2. `git::merge` builds the new commits in memory with libgit2. |
| 138 | 3. It moves the base branch with `reference_matching`, which only succeeds if the branch still points where it did when the merge started. |
| 139 | 4. The database records the merge, and `sync::refresh_pulls` re-evaluates the repository's other open pull requests. |
| 140 | |
| 141 | ## Design decisions |
| 142 | |
| 143 | | Decision | Why | |
| 144 | | --- | --- | |
| 145 | | A single binary with SQLite | Openhub targets small, self-hosted installations. One process and one data directory are easy to run, back up and move. It isn't designed for horizontal scaling. | |
| 146 | | A built-in SSH server | No system `sshd`, `authorized_keys` management or `git` Unix user is needed. Keys are managed in the web interface. | |
| 147 | | Pushes over SSH only | Key-based auth is the one credential for writing. HTTP is read-only, so passwords can never push. | |
| 148 | | Bare repositories named by database ID | Renaming a repository or user never touches the filesystem, and no filesystem path is ever built from user input. | |
| 149 | | `refs/pull/<n>/head` | A pull request's commits stay reachable, and its diff can still be shown, after its branch is deleted. | |
| 150 | | No background jobs | State is reconciled when it is next needed: after a push, and on pull request page views. There's nothing to schedule or monitor. | |
| 151 | | Server-rendered HTML, with optional JavaScript | Every page works without JavaScript. `static/app.js` only adds conveniences such as copy buttons and line highlighting. | |
| 152 | |
| 153 | ## Scope |
| 154 | |
| 155 | Openhub is git hosting only: repositories, code browsing, history, SSH keys, collaborators and pull requests. Issues, CI, wikis, releases, forks, organisations, webhooks, line-level review comments and HTTP push are out of scope. Discuss any of these before adding them. |
| 156 | |
| 157 | ## When you change this |
| 158 | |
| 159 | - **New module:** add it to the layout table in [`CLAUDE.md`](../CLAUDE.md) and to the components table above. |
| 160 | - **New runtime dependency** (another external program, say): add it to `check_git`-style startup checks, the `Dockerfile` runtime stage and the README requirements. |