Forgejo / Codey

This page discusses the technical details for our two products Forgejo by VSHN and Codey: Fully managed, dedicated Forgejo instances.

Codey is a product name for a very specific configuration and product offering of Forgejo by VSHN. Codey won’t be available on the regular in-cluster VSHN AppCat offering, only Forgejo by VSHN will. An instance of Codey has to be ordered via the Codey SaaS website and is running on pre-defined "black-box" OpenShift OKE service clusters to which the customer doesn’t have access to.

The offering of Codey combines compute resources, storage and the managed service into a pre-defined package.

Main differences between Forgejo by VSHN and Codey:

	Forgejo by VSHN	Codey
Compute and Storage	not included	included
Resource Restrictions	depending on underlying infrastructure	constraint be the product plan
Instance Location	depends on the underlying cluster	pre-defined by Codey
Configuration Flexibility	most options can be configured	only few options are exposed

Forgejo by VSHN

Codey

Compute and Storage

not included

included

Resource Restrictions

depending on underlying infrastructure

constraint be the product plan

Instance Location

depends on the underlying cluster

pre-defined by Codey

Configuration Flexibility

most options can be configured

only few options are exposed

Restrictions of Forgejo Features

Due to technical limitations, we cannot support all possible features of Forgejo. This section lists some of these restrictions.

Git access: Git access is currently only possible via HTTPS, SSH is not supported. Supporting SSH would either require to expose a plain TCP service which inclines the use of a LoadBalancer type service, or making use of the PROXY protocol and configure SSH_SERVER_USE_PROXY_PROTOCOL. While not impossible, it complicates the configuration by a lot.
Forgejo Actions: Forgejo Actions are fully supported, but the Actions runner need to be provided by the user right now. We plan to add a managed runner in the future as well.
Exposed configuration options: Many features of Forgejo are configured in the app.ini file and not via the web interface. We expose configuration options through the Deployment configuration.
High availability: We currently don’t offer a highly available option of Forgejo. If there is demand, we’ll implement it.

Deployment

Deployment and configuration of Forgejo happens with the official upstream Helm Chart and the Crossplane Provider for Helm. In the Crossplane Composite Resource Definition (XRD) we expose all required parameters.

We differentiate between Forgejo by VSHN and Codey by exposing a different XRD, depending on the service.

Forgejo by VSHN

For Forgejo by VSHN we expose many configuration options. This also implies that the user has to take care of several configuration options themselves, like configuring a suitable SMTP server or choosing the right database option. We provide sensible defaults though.

Claim

apiVersion: vshn.appcat.vshn.io/v1
kind: VSHNForgejo
metadata:
  name: my-forgejo-prod
spec:
  parameters:
    service:
      majorVersion: "9"
      fqdn: poc.mydomain.com (1)
      adminEmail: admin@example.com (2)
      forgejoSettings:
        config: (3)
          APP_NAME: "My Code Forge"
          mailer:
            ENABLED: "true"
            PROTOCOL: smtps
            SMTP_ADDR: mail.example.com
            SMTP_PORT: "465"
            USER: forgejo
        additionalConfigFromEnvs:
        - name: FORGEJO__MAILER__PASSWD
          valueFrom:
            secretKeyRef:
              name: my-passwords
              key: MAILER_PASSWD
    backup:
      schedule: "30 23 * * *"
      retention: 12
    extraSecrets:
    - name: my-passwords
  writeConnectionSecretToRef:
    name: forgejo-creds

1	The FQDN is passed to the Helm values on several places, for example `.ingress.*`, `.gitea.config.server.DOMAIN` and others.
2	Used for the Helm Chart value `.gitea.admin.email`
3	This is passed to the Helm Chart at `.gitea`. Some values will be overwritten, some values will be added. To be figured out which exactly we allow.

Injected configuration values:

.gitea.admin: Handled by the Composition and exposed in the connection secret.

Codey

Codey only offers very few configuration options, compared to Forgejo by VSHN. It also has pre-defined plans which set the compute and storage requests, limits and database choices accordingly.

The XRD for Codey is implemented by a different Composition than Forgejo by VSHN. It instantiates a pre-configured Composite Resource of Forgejo by VSHN. Think of Codey being the template for a XVSHNForgejo Composite Resource.

Claim

apiVersion: codey.io/v1
kind: Instance
metadata:
  name: my-codey-prod
spec:
  parameters:
    service:
      majorVersion: "9"
      adminEmail: admin@example.com (1)
    size:
      plan: mini (2)
  writeConnectionSecretToRef:
    name: codey-creds

1	Used for the Helm Chart value `.gitea.admin.email`
2	Codey plan

Notes:

The FQDN is automatically set to .metadata.name.app.codey.ch
We use a default configuration for all Codey instances, for example we use Mailgun to send E-Mail

Codey Plans

Codey offers several plans, which define the sizing and other affecting parameters. See also Product Documentation and Codey Website.

	Mini	Small	Medium	Large
Recommended Users Max.	2	10	100	500
Database	SQLite	SQLite	PostgreSQL	PostgreSQL
Storage	10 GiB	50 GiB	200 GiB	500 GiB
Cache	TwoQueue (2Q)	TwoQueue (2Q)	Redis by VSHN	Redis by VSHN
Queue	LevelDB	LevelDB	Redis by VSHN	Redis by VSHN
Kubernetes Resources	See below	See below	See below	See below

Mini

Small

Medium

Large

Recommended Users Max.

100

500

Database

SQLite

PostgreSQL

Storage

10 GiB

50 GiB

200 GiB

500 GiB

Cache

TwoQueue (2Q)

Redis by VSHN

Queue

LevelDB

Redis by VSHN

Kubernetes Resources

See below

Plan Upgrade / Downgrade

Plans can be changed anytime, as long as the database stays the same. Changing between plans with different databases is currently not supported. It will be offered once the process of changing databases is automated.

Forgejo Configuration

In this section the Forgejo configuration decisions are documented.

For Forgejo by VSHN the user can choose various options themselves, but we set reasonable defaults, as documented in this section. In Codey, the choices are given by the plan.

Database

There are several options for the database, most prominently PostgreSQL and SQLite.

Default: PostgreSQL by VSHN
Offered Choices: PostgreSQL, SQLite

Migration between different databases can be done with the forgejo dump command.

Storage

We use local storage for everything, therefore a PVC is mounted to /data.

While we could also use object storage for most of the data, it complicates a lot. For example backup would become an additional burden, as well as storage space monitoring also won’t be that easy anymore. And it’s not possible to store Git repositories on object storage, this way we’d have multiple storage locations.

Ref: Admin documentation: Storage

Cache

There are several options for caching, most prominently Redis and TwoQueue.

Default: Redis by VSHN
Offered Choices: Redis, twoqueue

twoqueue uses the recommended configuration for the HOST parameter:

{"size":100, "recent_ratio":0.25, "ghost_ratio":0.5}

Refs:

Queue

There are several options for queuing, most prominently Redis and LevelDB.

Default: Redis by VSHN
Offered Choices: Redis, level

Ref: Configuration Cheat Sheet: Queue

Backup and Restore

We use forgejo dump as a backup command for K8up. This command includes dumps of everything.

The backup scheduled is configured to run daily in the early morning.

Restoring is currently a manual process, there is no forgejo restore command available.

Ref: Backup and Restore (Forgejo is based on Gitea and the Forgejo docs do not have a specific page, the page in the Gitea documentation also applies to Forgejo.)

Metrics and Monitoring

We enable the /metrics endpoint and scrape metrics accordingly. See example-forgejo-metrics.txt for a list of available metrics.

By using the provided Monitoring Mixin we get Grafana dashboards.

Alerting is done on the availability of the service.

For Codey, we monitor gitea_users and compare the number to the chosen plan. Should the number of users exceed the plan, an alert is raised so that we can contact the customer.

E-Mail

Forgejo needs to send E-Mails and optionally can also receive E-Mails for issue handling.

Sending via SMTP: Codey uses Mailgun to send E-Mails with noreply@codey.ch as sender address. Forgejo by VSHN exposes the configuration parameters to configure the SMTP service.
Receiving via IMAP: E-Mail receiving works by regularly getting E-Mails from an IMAP mailbox. This is a manual configuration and only supported in Forgejo by VSHN version.

Refs:

Kubernetes Resource Configuration

We set appropriate Kubernetes resources, depending on the service type.

Forgejo by VSHN

TODO

Codey

Plan	Requests	Limits	Storage
Mini	Memory: XX, CPU: XX	Memory: XX, CPU: XX	10 GiB
Small	Memory: XX, CPU: XX	Memory: XX, CPU: XX	50 GiB
Medium	Memory: XX, CPU: XX	Memory: XX, CPU: XX	200 GiB
Large	Memory: XX, CPU: XX	Memory: XX, CPU: XX	500 GiB

Plan

Requests

Limits

Storage

Mini

Memory: XX, CPU: XX

10 GiB

Small

Memory: XX, CPU: XX

50 GiB

Medium

Memory: XX, CPU: XX

200 GiB

Large

Memory: XX, CPU: XX

500 GiB

Default Configurations

Forgejo by VSHN Helm Values

WIP. Currently Redis connections don’t work yet, because failed: remote error: tls: certificate required.

gitea:
  admin:
    username: "TheAdminUser"
    password: "AGoodPassword"
    email: "root@admin.ch"
  additionalConfigFromEnvs:
    - name: FORGEJO__DATABASE__USER
      valueFrom:
        secretKeyRef:
          name: postgres-creds
          key: POSTGRESQL_USER
    - name: FORGEJO__DATABASE__PASSWD
      valueFrom:
        secretKeyRef:
          name: postgres-creds
          key: POSTGRESQL_PASSWORD
    - name: FORGEJO__DATABASE__NAME
      valueFrom:
        secretKeyRef:
          name: postgres-creds
          key: POSTGRESQL_DB
    - name: POSTGRESQL_HOST
      valueFrom:
        secretKeyRef:
          name: postgres-creds
          key: POSTGRESQL_HOST
    - name: POSTGRESQL_PORT
      valueFrom:
        secretKeyRef:
          name: postgres-creds
          key: POSTGRESQL_PORT
    - name: FORGEJO__DATABASE__HOST
      value: "$(POSTGRESQL_HOST):$(POSTGRESQL_PORT)"
    - name: REDIS_HOST
      valueFrom:
        secretKeyRef:
          name: redis-creds
          key: REDIS_HOST
    - name: REDIS_PORT
      valueFrom:
        secretKeyRef:
          name: redis-creds
          key: REDIS_PORT
    - name: REDIS_USERNAME
      valueFrom:
        secretKeyRef:
          name: redis-creds
          key: REDIS_USERNAME
    - name: REDIS_PASSWORD
      valueFrom:
        secretKeyRef:
          name: redis-creds
          key: REDIS_PASSWORD
    - name: FORGEJO__CACHE__HOST
      value: "rediss://$(REDIS_USERNAME):$(REDIS_PASSWORD)@$(REDIS_HOST):$(REDIS_PORT)/0?pool_size=100&idle_timeout=180s&skipverify=true"
  config:
    database:
      DB_TYPE: postgres
      SSL_MODE: require
    log:
      LEVEL: "info"
    server:
      ROOT_URL: https://myforgejo.example.com
      DOMAIN: myforgejo.example.com
      DISABLE_SSH: true
      LFS_START_SERVER: true
      OFFLINE_MODE: true
      LANDING_PAGE: login
      MINIMUM_KEY_SIZE_CHECK: true
    repository:
      ROOT: /data/git/repositories
    lfs:
      PATH: /data/git/lfs
    service:
      DEFAULT_KEEP_EMAIL_PRIVATE: true
      ENABLE_NOTIFY_MAIL: true
      ALLOW_ONLY_EXTERNAL_REGISTRATION: true
    admin:
      SEND_NOTIFICATION_EMAIL_ON_NEW_USER: true
    cron:
      ENABLED: true
    mailer:
      ENABLED: false
    session:
      PROVIDER: db
    cache:
      ADAPTER: twoqueue
      HOST: '{"size":100, "recent_ratio":0.25, "ghost_ratio":0.5}'
    packages:
      LIMIT_SIZE_CONTAINER: "2 GiB"
    oauth2_client:
      UPDATE_AVATAR: true
    queue:
      TYPE: level
    cron.archive_cleanup:
      SCHEDULE: "@hourly"
      OLDER_THAN: "2h"
    indexer:
      ISSUE_INDEXER_TYPE: bleve
      REPO_INDEXER_ENABLED: true
    security:
      REVERSE_PROXY_TRUSTED_PROXIES: "*"
    metrics:
      ENABLED: true

global:
  compatibility:
    openshift:
      adaptSecurityContext: force

ingress:
  enabled: true
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-production
  tls:
    - hosts:
        - myforgejo.example.com
      secretName: ingress-cert
  hosts:
    - host: myforgejo.example.com
      paths:
        - path: /
          pathType: Prefix

redis-cluster:
  enabled: false
redis:
  enabled: false
postgresql:
  enabled: false
postgresql-ha:
  enabled: false

persistence:
  enabled: true

strategy:
  type: "Recreate"

Download: example-forgejo-helm-values.yaml

Codey Composite Resource Template

WIP. Needs proper definition…

Composite Resource

apiVersion: vshn.appcat.vshn.io/v1
kind: XVSHNForgejo
metadata:
  name: my-codey-prod
spec:
  parameters:
    service:
      majorVersion: "9"
      fqdn: poc.mydomain.com
      adminEmail: admin@example.com
      forgejoSettings:
        config:
          APP_NAME: "Codey"
          mailer:
            ENABLED: "true"
            PROTOCOL: smtps
            SMTP_ADDR: mail.example.com
            SMTP_PORT: "465"
            USER: forgejo
        additionalConfigFromEnvs:
        - name: FORGEJO__MAILER__PASSWD
          valueFrom:
            secretKeyRef:
              name: my-passwords
              key: MAILER_PASSWD
    backup:
      schedule: "30 23 * * *"
      retention: 12
    extraSecrets:
    - name: my-passwords

Old Helm Values: example-forgejo-helm-values-codey.yaml

Forgejo / Codey

Restrictions of Forgejo Features

Deployment

Forgejo by VSHN

Codey

Codey Plans

Plan Upgrade / Downgrade

Forgejo Configuration

Database

Storage

Cache

Queue

Backup and Restore

Metrics and Monitoring

E-Mail

Kubernetes Resource Configuration

Forgejo by VSHN

Codey

Links

Default Configurations

Forgejo by VSHN Helm Values

Codey Composite Resource Template