Security and Vulnerability Handling Process

Critical security vulnerabilities are announced on mailing lists from suppliers, on security newsletters, by MELANI, and by many other sources. This security advisories are analysed and if a vulnerability affects the system directly, the update is planned as soon as possible.

Urgent upgrades are assessed and planned immediately and less urgent vulnerabilities are patched according to our maintenance process.

General

Our maintenance process ensures regular system updates (there are some exceptions on customer’s systems which are maintained according to special agreements). We manage server systems based on Ubuntu and RedHat. These two big players usually provide updates rapidly. With this regular maintenance the known vulnerabilities are eliminated within one week.

Internal security incidents are reported according our ISO27001 certified incident management process.

Our Responsible Ops engineers are supervising our monitoring dashboard during office hours according to our Monitoring Concept. The Responsible Ops engineers are able to react quickly during office hours.

Customers can report vulnerabilities to VSHN according their Service Level Agreement.

We regularly discuss security issues in our vshn.chat #security channel.

Security Process

Office Hours

Our standard procedure to handle security issues or vulnerabilities.

  1. Someone (customer, employee, Responsible Ops, etc.) become alert of a vulnerability.

  2. Ticket is created in control.vshn.net.

  3. If it’s a critical vulnerability it’s announced in our chat and an instant task force starts discussing potential solution paths. Depending on the severity we are following Emergency Communication Channels.

  4. If an immediate action within 24h is required, the affected systems will be updated. Because of our flexible working philosophy there is always someone found who can handle the update.

  5. Such maintenance work is announced via:

Outside Office Hours

VSHN doesn’t actively supervise vulnerability announcements outside office hours.

Customers with a 24/7 contract who become aware of a critical vulnerability should…​

  1. Create a ticket in control.vshn.net with high priority selected

  2. AND call our 24/7 on-call number.

The on-call engineer will assess the issue and has the possibility to call 2nd level on-call engineers, if there is a bigger problem to tackle down.

Checklist for High Critical Vulnerabilities

If there are new vulnerabilities revealed, such as Remote Code Executions (RCE) which need immediate action, this checklist helps to ensure mitigation is done.

This checklist should be used usually for very urgent unauthenticated remote code execution vulnerabilities with a CVSS score 9 and higher.

Technical Checklist

  • Create a ticket per server for customer communication and steps done for mitigation

  • Put the service behind a VPN if possible

  • Ensure it is not available directly via the Internet

  • If available via Internet and no patch available yet

    • Shut the service down

    • Ensure service does not restart automatically after reboot

    • Ensure service is not restarted via configuration management (e.g. Puppet) or other means like GitOps (e.g. ArgoCD)

    • Ensure someone review your change also with this checklist

  • (?) it is in the nature of such an incident, that we cannot predict everything; discuss with your peers what should be done additionally.

  • (?) …​

Process Checklist

  • Create internal ticket to handle all separate customer / server tickets

  • Ensure an internal communication channel (e.g additional chat channel '#CVE-XXXX-YYY') is created to discuss with all involved VSHNeers

  • Ensure that VSHNeers are informed at #security and/or #technical

  • Make sure all Responsible Ops could support in handling the vulnerability if needed.

Lists

We’re subscribed to the most important security announcement lists, such as:

Individual Customer’s Needs

If you as a customer have a specific need to monitor a certain software or security list, please approach us so we can work out a process to ensure this.