Skip to main content

Neuvector Patching

This document covers a high level patching example for Neuvector.

Neuvector should be patched per environment to allow for testing and confidence in the upgraded version.

Neuvector is deployed via a Helm chart which is maintained by HMCTS, the chart itself has a dependency on the upstream Neuvector Helm chart.

If a new version of the upstream Neuvector chart is available then we can patch, if not then there is no need to patch unless bug fixes are required.

Neuvector is deployed to both CFT and SDS AKS clusters across multiple non-production and production environments.

Prepare a new chart version

Before any patching of AKS can take place we need to have an updated version of the HMCTS Neuvector chart.

To check if patching is ready, check the releases on the upstream Neuvector github

Then compare the latest version available with the current version that the HMCTS Neuvector depends on.

If the latest release is ahead then we need to patch the HMCTS Neuvector chart with an updated dependency.

We should also check for any Renovate PRs that have been raised and merge these in where no breaking changes are contained.

Using Git you need to:

Once reviewed and merged you need to check for any renovate PRs and merge these in where no breaking changes are contained. Then draft a new release with the latest code. You can find steps here on how to draft a release

The instructions on that page are specific to draft releases, Initially when creating the new release it is worth marking as “pre-release” as we will be testing on non-production before moving to production. Once tested we can edit this release and mark it as latest release before patching the production environments. This will trigger the Azure DevOps pipeline to run and validate and release a new version.

Non-Production Patching

Updating the chart version and CRDs in non-production

This section is only necessary for non-production environment patching

Part of the Neuvector setup involves the installation of CRDs (custom resource definitions) and these will need to be updated as well. With recent updates to how CRDs are handled we can now just match the version of the CRD to the Neuvector application version, So if we are on Neuvector 2.8.12 then the CRD will be on that version also.

We can update the CRDs and app in non-production on a per environment basis by updating the following files.

To update the app to use our new release of chart-neuvector we need to add the following lines to the environment level yaml file

  chart:
    spec:
      chart: neuvector-azure-keyvault
      version: x.x.x

By adding this block under the “spec:” on line 7 we can override the chart on that environment to use our new release.

We also need to update the CRDs for that environment, to do this we need to update the patching file in the environment specific crd folder. In the patching file we need to update the version to match our new neuvector version. We then add this as a patch to the kustomization file for that environment under the patches section.

Once both these changes have been made raise a PR to have them applied to the non-production environment. It is recommended you do these one at a time to allow for easier troubleshooting.

This is a PR example showing the files and locations involved in this example.

Check health of deployment and log in to the UI

Check pod health

Make sure that you are connected to the correct AKS cluster in your local setup (kubectl), then check the pods of the service are all healthy:

You can check local cluster with this command:

kubectl config current-context

Then check the pods in the neuvector namespace with this command:

kubectl get pods -n neuvector

You can also check the status of the Helm Release which will show the version of chart-neuvector that has been installed:

kubectl get hr -n neuvector

Output should look something like this:

NAME            AGE    READY   STATUS
fluentbit-log   427d   True    Helm upgrade succeeded
neuvector       84d    True    Helm upgrade succeeded for release neuvector/neuvector.v6 with chart neuvector-azure-keyvault@1.5.4

If all pods are healthy you can move on.

Log on to neuvector UI

Neuvector has a UI that should be checked to make sure its working and you can log in. In the case of Neuvector you can find a clusters UI address in the specific values files. It should look something like:

host: sds-neuvector00.ithc.platform.hmcts.net

If the UI is working, and you can log in if required then its safe to continue on with the upgrade.

Logging in should be possible by selecting the Login with SAML option which uses single sign on, however if that does not work there are admin credentials available to use directly.

These can be found in the specific Azure Key Vault for that environment (search Azure to find these):

neuvector key vaults

The secrets should be called:

  • neuvector-admin-username
  • neuvector-new-admin-password

If the UI is failing or you can’t log in, then you’ll need to check the logs of the pods we looked at earlier:

kubectl logs <pod name> -n neuvector

You may also need to ask platform-operations on Slack for input if the logs have nothing useful.

To check if its being installed you can view the description of one of the pods (pods that have a very short age have just been deployed so use one of these to test).

kubectl -n neuvector describe pod neuvector-controller-pod-<hash>

Within the description you will find the image tag and it should contain the latest version of the Neuvector application

Image: hmctspublic.azurecr.io/imported/neuvector/controller:5.3.0

This process can be used to update all non-production environments were Neuvector is currently installed and should be done per environment to allow for testing and confidence in the upgraded version before moving to production.

Production Patching

Production patching mainly involves removing some of the above configuration and updating the primary deployment files.

Remove non-production overrides and update the main deployment file

Remove the environment level overrides for the chart version:

spec:
  chart:
    spec:
      version: 1.5.4

Once non-production temporary changes have been removed you can now update the chart version in the main deployment file to the latest version of the HMCTS chart (the one you just removed from the environment level files). We also need to update the neuvector-crds yaml file to match the version we set for our non-production environments, this is the file we need to update.

Push, commit and merge this change for it to get deployed to Production.

Check everything is deployed and working as expected

To check if its being installed you can view the description of one of the pods (pods that have a very short age have just been deployed so use one of these to test).

kubectl -n neuvector describe pod neuvector-controller-pod-<hash>

Within the description you will find the image tag and it should contain the latest version of the Neuvector application

Image: hmctspublic.azurecr.io/imported/neuvector/controller:5.3.0

You can also check the status of the Helm Release which will show the version of the HMCTS chart that has been installed:

kubectl -n neuvector get hr

NAME            AGE    READY   STATUS
fluentbit-log   427d   True    Helm upgrade succeeded
neuvector       84d    True    Helm upgrade succeeded for release neuvector/neuvector.v6 with chart neuvector-azure-keyvault@1.5.4

Check the UI for each Production cluster to make sure everything is working as normal:

SDS

CFT

This page was last reviewed on 26 February 2025. It needs to be reviewed again on 26 February 2026 by the page owner platops-build-notices .
This page was set to be reviewed before 26 February 2026 by the page owner platops-build-notices. This might mean the content is out of date.