APPUiO Control API Billing Entity as Virtual Resource

Problem

We want to allow users to create and manage billing relevant data (billing entity) on their own using the APPUiO Control API (self-service). We use an ERP which is not Kubernetes aware for all of our billing and accounting needs. Some customers might have other products than APPUiO Cloud and our accounting team needs to edit billing entities in the ERP.

Since billing entities will be edited through both the APPUiO Portal and the ERP, we need to make sure that the billing entities are always in sync.

Relevant requirements

  • Self-Service

Proposals

Virtual Resource Stored in ERP

The billing entity is not stored in the APPUiO Control API but fetched on demand from the ERP. This is implemented as an aggregated API server which translates the APPUiO Control API API to the ERP API. The adapter respects the RBAC rules of the APPUiO Control API and only allows users to access their own billing entities.

This solves any synchronization issues as the billing entity is always up-to-date.

The APPUiO Control API might become partly unresponsive if the ERP is down or slow.

Aggregated apiserver empowers you to customize your apiserver to do following things cannot achieved by CR[D]:

  • Makes your apiserver adopt different storage APIs rather than ETCDv3

  • Extends long-running subresources/endpoints like websocket for your own resources

  • Integrates your apiserver with whatever other external systems

— Kubernetes SIG Kubebuilder
apiserver-builder-alpha Readme
Example 1. Editing Entity example
  1. The API transparently loads the billing entity from the ERP.

  2. A user edits the billing entity in the APPUiO Portal.

  3. The changes are saved in the ERP through the API.

Custom Resource Stored in Control API

The billing entity is stored in the APPUiO Control API as a custom resource. The billing entity is synchronized with the ERP periodically and on demand.

The controller synchronizing the billing entity with the ERP is implemented as a Kubernetes controller. An implementation based on a controller would create a system with multiple sources of truth. In the general case, there’s no right order for independent updates in such a system.

The controller must have a way to track changes from both the ERP and the APPUiO Control API. The sync state tracking might be inspired by Kubernetes three-way merge patching and resource versioning.

It could be possible to use Webhooks in the ERP to notify the APPUiO Control API about changes. There still needs to be backup mechanism if changes made in the ERP happen during downtime of the APPUiO Control API or vice-versa.

Since the custom resources are stored locally performance is not affected by the ERP.

We still need to implement an aggregated API server since list operations are not supported through Kubernetes RBAC.

Example 2. Editing Entity example
  1. The controller loads the billing entity from the ERP and stores it in the APPUiO Control API.

  2. A user edits the billing entity in the APPUiO Portal.

  3. The changes are saved in the control API.

  4. The controller compares versions and synchronizes the changes with the ERP.

VSHN Specific

There are plugins that implement webhooks in the ERP. Odoo Webhook marketplace search.

Since we have an ancient ERP version the plugins might not work.

Decision

Virtual Resource Stored in ERP

Rationale

The virtual resource stored in the ERP approach has been chosen because it does not have any synchronization issues. Debugging synchronization issues can be tricky.

If we go for the locally stored CRD approach we still need to implement an aggregate API server for list operations.

The following reasons favor the virtual resource stored in the ERP over the CRD approach:

Debuggability

The virtual resource stored in the ERP approach is easier to debug. The CRD approach requires a lot of synchronization logic and debugging might be tricky. The virtual resource stored in the ERP approach is easier to understand and debug.

Implementation effort

The syncing approach has more open variables and might change substantially if switching to a different ERP. We need to implement an aggregate API server for list operations anyway.