Usage profiles
Originally, the Resource limit per Namespace and Namespace quota per Zone requirements have been implemented as Kyverno policies on each zone. With this approach, overriding the default resource quotas must be done on individual namespaces (or on zones for the namespace count quota). This approach doesn’t scale, and doesn’t easily allow configuring different default quotas for different organizations or billing entities.
As APPUiO Cloud has grown, and we now allow users to sign up autonomously (self signup), we want to configure lower default quotas and additional policies (for example network policies) for organizations of billing entities which haven’t been validated yet.
Usage profiles
We’ve decided to call a set of quotas and policies a usage profile.
We’re implementing usage policies in the APPUiO Control API, so that non-technical users who have administrative privileges can easily update the usage profile for a billing entity or an organization in the APPUiO Portal.
Usage profiles are defined as a new custom resource in the APPUiO Control API.
See the UsageProfile reference documentation for details on the structure of the custom resource.
Default usage profile for a billing entity
We want to allow operators of APPUiO Cloud to set a default usage profile for a whole billing entity.
To enable this, we introduce a new field spec.usageProfileRef on the billing entity which indicates the default usage profile for organizations associated with this billing entity.
The APPUiO Control API sets field status.usageProfileRef of all organizations to field spec.usageProfileRef of the billing entity referenced in spec.billingEntityRef.
Customize usage profile for an organization
The organization virtual resource has a new optional field spec.usageProfileRef which contains the name (metadata.name) of a UsageProfile resource.
If this field is present, it overrides the default usage profile that would be assigned to the organization based on its billing entity.
If present, the value of this field is validated by the APPUiO Control API.
The APPUiO Control API rejects updates of an organization virtual resource, if the spec.usageProfileRef field is modified and the principal doesn’t have access to the usage profile referenced in the updated organization resource.
The APPUiO Control API ensures that status.usageProfileRef is updated to the new billing entity’s default usage profile when an organization’s spec.billingEntityRef is updated, if the organization doesn’t have spec.usageProfileRef set.
Applying usage profiles on zones
To apply usage profiles to organization namespaces on each zone, we extend the APPUiO Cloud agent.
The agent identifies the organization’s usage profile by looking at field status.usageProfileRef in the APPUiO Control API Organization object for the organization.
We reimplement the existing Kyverno policies which manages the resource quotas in organization namespaces and restrict the number of namespaces per zone in the agent.
The resource quota policy is changed to apply the quotas and policies defined in the UsageProfile referenced by the Organization by default, instead of the default quotas stored on the zone.
Per-namespace quota overrides are still possible through namespace annotations.
Additionally, the agent reject new namespaces if the namespace’s organization has reached its namespace count limit on the zone.
The agent reads the organization’s namespace count limit from the organization’s usage profile.
To determine the usage profile for each organization, the agent connects to the APPUiO Control API to read UsageProfile and Organization resources.
The agent watches these resource for changes to ensure changes are reflected on the zones.
Additionally, the agent reconciles all Kubernetes resources which are created to apply a usage profile. This ensures that users can’t modify the quotas and policies defined through the usage profile.