Running Repo Renderer on OpenBSD

# Running Repo Renderer on OpenBSD

This guide sets up an OpenBSD 8.0 (or later) server that hosts your git
repositories and serves a website for them. People push to the server over
SSH, each push re-renders the site, and httpd serves the result, with
relayd in front of it for TLS.

You'll need two machines:

- **Your machine**, with a checkout of this repository and Go installed.
  You build the server's programs here, and run the scripts in `tools/`.
- **The server**, running OpenBSD.

Each set of commands below is labelled with the machine it runs on and the
user it runs as. The paths used here are the usual OpenBSD ones, but none
of them are built into the renderer, so you can change any of them in the
config.

When you're done, the server will have:

| Path | Contents | Owner |
|---|---|---|
| `/var/git/NAME.git` | the bare repositories | `git` |
| `/var/www/htdocs/repo-render` | the rendered website | `git` |
| `/var/spool/repo-render`, `/var/db/repo-render` | the renderer's job queue and saved state | `git` |
| `/etc/repo-render/repo-render.yaml` | the renderer's config | root |
| `/etc/repo-render/httpd.conf` | httpd's config for the site | root |
| `/usr/local/bin/repo-render`, `/usr/local/bin/repo-render-new-repo` | the programs | root |
| `/usr/local/share/repo-render/post-receive` | the git hook, which every repository links to | root |
| `/usr/local/share/examples/repo-render/` | example configs | root |

A user called `git` owns the repositories, accepts the pushes, and runs
the renderer. When someone pushes, the git hook adds the push to the
renderer's queue and starts the renderer in the background. If the
repository has mirrors (for example on GitHub), the hook also pushes the
changes on to them.

## 1. Install the programs

On your machine, in the checkout, build the server package and copy it to
the server. For an arm64 server, add `SERVER_ARCH=arm64` to the `make`
command.

```sh
make server
scp dist/repo-render-openbsd-amd64.tar.gz SERVER:
```

On the server, as root, unpack it and run the install script, which
installs the programs, the hook and the example configs under
`/usr/local`:

```sh
tar -xzf repo-render-openbsd-amd64.tar.gz
repo-render-openbsd-amd64/install.sh
```

## 2. Create the git user

On the server, as root:

```sh
pkg_add git
echo /usr/local/bin/git-shell >> /etc/shells
useradd -d /var/git -g =uid -s /usr/local/bin/git-shell git
install -d -o git -g git -m 755 /var/git
install -d -o git -g git -m 700 /var/git/.ssh
```

Add the SSH public key of everyone who should be able to push to
`/var/git/.ssh/authorized_keys`, and make the file owned by `git` with
mode 600. The `git-shell` login shell only allows git commands, so these
keys can push and fetch but can't log in.

The git user's home directory is the directory of repositories, so the
address of a repository is simply `git@code.example.com:NAME.git`.

## 3. Configure the renderer

On the server, as root, copy the example config into place and edit it:

```sh
install -d /etc/repo-render
install -m 644 /usr/local/share/examples/repo-render/repo-render.yaml /etc/repo-render/
vi /etc/repo-render/repo-render.yaml
```

The settings that aren't commented out are required. Change
`code.example.com` to your host name, and change the paths if you want
things somewhere else. The commented-out settings show their default
values, and you can uncomment any of them to change it, for example to
set the site's title or to group repositories into sections on the index
page.

The config file belongs to root, so people who can push can't edit it.

Then create the directories the renderer writes to, and check the config
by running the renderer's `check` command as the git user. It should
finish by printing `ok`.

```sh
install -d -o git -g git -m 755 /var/www/htdocs/repo-render \
    /var/spool/repo-render /var/db/repo-render
su -s /bin/sh git -c 'repo-render --config /etc/repo-render/repo-render.yaml check'
```

## 4. Configure httpd

The package includes a complete httpd config for the site, with comments
explaining each part. On the server, as root, copy it into place and edit
it:

```sh
install -m 644 /usr/local/share/examples/repo-render/httpd.conf /etc/repo-render/
vi /etc/repo-render/httpd.conf
```

In the editor:

- Change `code.example.com` in the `server` line to your host name.
- Delete the section marked as the private site, unless you're setting
  one up (step 8).
- If `/etc/httpd.conf` already has a `types` block, delete the `types`
  block in this file. httpd only allows one.

If you changed `htdocs` in the renderer's config, change `root` and
`errdocs` to match. The file explains how.

If `/etc/httpd.conf` already has a `server` block for your host name,
remove it. Then include the new file from `/etc/httpd.conf` and start
httpd:

```sh
echo 'include "/etc/repo-render/httpd.conf"' >> /etc/httpd.conf
httpd -n && rcctl enable httpd && rcctl restart httpd
```

httpd now serves the site over plain HTTP on port 8080. The next step puts
relayd in front of it to handle HTTPS.

## 5. Set up HTTPS with relayd

On the server, as root, add a relay to `/etc/relayd.conf` that accepts
HTTPS on port 443 and forwards it to httpd:

```
relay "https" {
        listen on $external_address port 443 tls
        protocol "https"
        forward to 127.0.0.1 port 8080 check tcp interval 30
}
```

You'll need a TLS certificate for relayd to use. The site's httpd config
answers acme-client(1) challenges from `/var/www/acme`, so you can get a
certificate from Let's Encrypt with acme-client.

## 6. Add repositories

The easiest way to add a repository is from your machine. In the working
copy of the project, run:

```sh
tools/create-repo -d "A one-line description" --section "Section title" SERVER
```

This creates the repository on the server, named after the working copy's
directory, adds it to the given section of the index page, adds the
server as the `origin` remote, and pushes. If you leave out `--section`,
the repository is private and doesn't appear on the public site. Add
`--github` to create a GitHub repository as well, as a mirror (see
step 7).

To copy all of your GitHub repositories to the server at once, run this
on your machine. Try it with `--dry-run` first to see what it will do.

```sh
tools/migrate-from-github --dry-run SERVER
tools/migrate-from-github SERVER
```

Both scripts connect to the server over SSH as your own user and use doas
for the parts that need root. The comment at the top of each script lists
all of its options.

You can also add a repository by hand. On the server, as root:

```sh
su -s /bin/sh git -c 'repo-render-new-repo -c /etc/repo-render/repo-render.yaml -d "A one-line description" /var/git/NAME.git'
```

Then push to `git@SERVER:NAME.git`, and add the repository to a section
in the config if you want it on the index page. If your default branch
isn't called `main`, add `-b BRANCH` to the command above.

To change a repository's description later, run the following on the
server, as root. The second command re-renders the repository, since the
description appears on every page.

```sh
su -s /bin/sh git -c 'repo-render-new-repo -d "A new description" /var/git/NAME.git'
su -s /bin/sh git -c 'repo-render --config /etc/repo-render/repo-render.yaml render NAME'
```

## 7. Mirror repositories to GitHub or elsewhere (optional)

A repository can have one or more mirrors. After every push, the hook
pushes the repository's default branch and all of its tags to each
mirror. It force-pushes the branch, so the mirror always matches the
server, but it never deletes branches or tags on the mirror. If a mirror
push fails, the error is logged to `/var/log/messages` and the next push
tries again.

Mirroring only goes one way. Anything changed on a mirror, such as a pull
request merged on GitHub, is overwritten by the next push to the server.
It's a good idea to turn off merging on your mirrors.

The server needs an SSH key that's allowed to push to the mirrors, and
the mirrors' host keys, because the hook runs without anyone there to
answer SSH's questions. On the server, as root:

```sh
ssh-keygen -t ed25519 -N '' -f /etc/repo-render/mirror_key
chown git /etc/repo-render/mirror_key
ssh-keyscan github.com codeberg.org >> /etc/ssh/ssh_known_hosts
cat /etc/repo-render/mirror_key.pub
```

Add the public key that the last command prints to each mirror host, with
permission to push. On GitHub, that's a deploy key with write access, or
a key on your account.

`tools/create-repo --github` and `tools/migrate-from-github` set up the
mirrors for you if you give them the key with
`--key /etc/repo-render/mirror_key`. To add a mirror by hand, run this on
the server, as root:

```sh
su -s /bin/sh git -c 'repo-render-new-repo -m git@github.com:me/NAME.git /var/git/NAME.git'
su -s /bin/sh git -c 'git -C /var/git/NAME.git config core.sshCommand "ssh -o BatchMode=yes -o ConnectTimeout=30 -i /etc/repo-render/mirror_key"'
```

## 8. Set up a private site (optional)

Repositories that aren't in any section of the index are private. By
default, private repositories aren't rendered at all. You can instead
render them to a separate site at `/private/` that asks for a password.

On the server, as root, create a directory for the private site, add it
to the renderer's config, and create a login:

```sh
install -d -o git -g www -m 750 /var/www/private/repo-render
echo 'private_htdocs: /var/www/private/repo-render' >> /etc/repo-render/repo-render.yaml
htpasswd /var/www/conf/repo-render.htpasswd YOURNAME
chown root:www /var/www/conf/repo-render.htpasswd
chmod 640 /var/www/conf/repo-render.htpasswd
```

Make sure `/etc/repo-render/httpd.conf` still has the private site
section from the example, with your host name in it. Then reload httpd
and render everything:

```sh
httpd -n && rcctl reload httpd
su -s /bin/sh git -c 'repo-render --config /etc/repo-render/repo-render.yaml render --all'
```

To add more logins later, run `htpasswd` again. httpd reads the file on
every request, so there's no need to reload it. Browsers send the
password with every request, so only make the private site available
over HTTPS.

## Day-to-day use

These commands run on the server, as root:

- After editing the renderer's config, check it with:

  ```sh
  su -s /bin/sh git -c 'repo-render --config /etc/repo-render/repo-render.yaml check'
  ```

  Any change to the config makes the next push re-render every repository.

- To re-render everything immediately:

  ```sh
  su -s /bin/sh git -c 'repo-render --config /etc/repo-render/repo-render.yaml render --all'
  ```

  This also removes the pages of any repository you've deleted from
  `/var/git`.

- The renderer logs to `/var/log/daemon`, and mirror pushes log to
  `/var/log/messages`.

## Upgrading

On your machine, update the checkout and build a new package:

```sh
git pull
make server
scp dist/repo-render-openbsd-amd64.tar.gz SERVER:
```

On the server, as root, unpack it and run the install script again, as
in step 1. Every repository links to the same installed hook, so this
upgrades the hook for all of them.

Then compare `/etc/repo-render/httpd.conf` with the new example at
`/usr/local/share/examples/repo-render/httpd.conf`, copy over anything
that changed, and reload httpd:

```sh
httpd -n && rcctl reload httpd
```

If you're upgrading from the older version that had a `setup-check`
command, follow [upgrading.md](upgrading.md) instead.