CI/CD Processes
AppCat contains many repositories.
They all converge in the component-appcat
, which is responsible for deploying everything.
In order to streamline our development process, we improved the CI/CD with following steps:
-
Creating a PR in an AppCat repository automatically builds images and packages
-
Creating a PR in an AppCat repository automatically creates a PR on the component that reflects the changes, so it can instantly be deployed any SYN cluster to test
-
Proper labels on the PRs are enforced
-
Merging a PR will automatically create a release, according to the labels it will detect what version increment it needs to do
-
The auto release is not active on the component repository
How to manage the CI/CD pipelines
All the files that specify the process are managed via cruft and cookiecutter. The repository is: github.com/vshn/appcat-cookiecutter. Each of the AppCat repos contains a cronjob that checks for changes every 60 minutes. If there are changes it will open a new PR.
Continuous Integration
There are automated workflows in place in AppCat repository that should streamline our development between AppCat and Component repositories.
Color purple shows user action |
Releases from AppCat repository
Example of engineer steps:
-
Trigger the GitHub Action for a pre-release.
-
Check and merge the
component
PR that was opened in step 1. -
Tag a new version.
The release has to be manually dispatched, usually before the end of the sprint, on Tuesday. The following is the lifecycle of the release process:
Feature Requests from AppCat repository
Example of engineer steps:
-
Create a pull request (PR) with base branch
develop
. -
Label the PR with either
patch
orminor
. -
Perform development work.
-
Approve related PRs in the
component
andappcat
repositories. -
Merge the PR by commenting
/merge
in theappcat
repository. -
Check the workflow link in the next comment to ensure all jobs succeeded.
-
Confirm that the
develop
branch has been updated.
All feature requests starting from AppCat repository have the following lifecycle:
Hotfix from AppCat
Example of engineer steps:
-
Create a pull request (PR) with base branch
master
. -
Label the PR with
hotfix
. -
Perform development work.
-
Approve related PRs in the
component/appcat
repository. -
Merge the PR by commenting
/merge
in theappcat
repository. -
Check the workflow link in the next comment to ensure all jobs succeeded.
-
Confirm that the
master
branch has been updated. -
Confirm PR from
master
intodevelop
was merged. -
Tag a new version.
All hotfixes requests starting from AppCat repository have the following lifecycle:
Continuous Deployment
AppCat Deployment
Deployment of AppCat is not the same as releasing a new version of the framework to services.
With Crossplane composition functions, multiple versions can be deployed simultaneously in the cluster.
-
These functions run in parallel and remain mostly idle until a composite (service) uses them.
-
This enables a fine-grained release process at the level of each individual instance.
-
Each instance can run any of the available AppCat versions present in the cluster at that time.
A new version is deployed when a new tag is created in the component-appcat
repository.
-
In test clusters, deployment happens instantly.
-
In production, it occurs after a one-week delay.
-
The dependency is updated automatically via the
commodore-defaults
repository using Renovate.
AppCat Rollout
The rollout process refers to a service starting to use a newer composition revision with a higher revision number.
-
The upgrade is triggered automatically during a service maintenance window.
-
The latest available revision is selected during maintenance, even if intermediate versions exist.
-
An upgrade can also be triggered manually by running the maintenance job for a specific instance.
AppCat Rollback
Currently, rollback must be performed manually for each service instance.
-
This involves updating the claim’s
spec.compositionRevisionRef.name
field to the desired AppCat version. -
The process is manual.
-
In the near future, Schedar will implement automatic rollback capabilities:
-
Per service instance
-
Per service type
-
Per cluster
-
Hotfix
Hotfixes are special jobs that update the composition revision reference in composites.
-
A hotfix is triggered for a specific version from the
component-appcat
repository using the format:appcat.deploymentManagementSystem.hotfix=<component-version>-<appcat-version>
-
The specified version must match an existing composition revision.
-
If no match is found, the hotfix jobs will not apply any changes.