Common pipeline
To get onboarded to the common pipeline please see Jenkins onboarding.
The common pipeline is a Jenkins pipeline defined by code which implements Continuous Delivery to production while enforcing a standard set of checks on the code processed.
It enables HMCTS Reform to deploy changes to the platform in a well defined manner providing the confidence that all the test and verification stages have been executed in a structured and repeatable sequence. This allows for fast feedback to developers to improve code development velocity.
The delivery pipeline combines infrastructure and database schema changes with the application deployment, therefore taking advantage of the microservices architecture in use at HMCTS.
Code changes are subjected to a round of static tests - consisting of unit tests, static code analysis and security checks - before being deployed to a non-prod environment in a non-publicly accessible AKS (Kubernetes) deployment. Here, a range of smoke tests and non-destructive functional tests verifies the app is functioning. At this point a promotion process is started which labels the Docker image produced by the previous stages of the pipeline as production-ready. Production deployments are managed by flux which takes care of keeping an application deployment up to date with the latest production-ready image generated by the delivery pipeline.
Smoke tests are available in production as part of a flux deployment and run from a separate container.
The entire Delivery Pipeline is a hands-off, automated process, triggered at the point of change. Code merged to master is deployed to production without any further human intervention. For this reason:
- PRs need to be carefully reviewed
- Feature Flagging, to separate the deployment of changes from their activation, is a practical necessity for all apps.
More information
For an in-depth guide on how the common pipeline works and how it should be used, please refer to the cnp-jenkins-library README
Deploying applications using the common pipeline
Helm is the package manager for Kubernetes. We deploy all of our applications using a helm chart to Kubernetes. For more information about Helm, please see the related new component docs
The diagram below shows the end to end workflow of how an application is deployed throughout the SDLC, Perftest is an example of a non path to live environment that is optionally deployed to.
An application consists of the different infrastructure levels of platform, product and component as detailed in Infrastructure levels.
The product and component levels are combined to define the name of the application, denoted by app-name in the diagram below. The values for product and component should be defined in the Jenkinsfile within the github repo.
Example: cnp-plum-frontend
The environments shown in the diagram reflect CFT. Review the table below to see the corresponding SDS environment names.
| CFT Environment | SDS Environment |
|---|---|
| AAT | Staging |
| Perftest | Test |
| Preview | Dev |
| Prod | Prod |
End to end process:
- Commit changes to feature branch
- Create a pull request
- Jenkins will push an image with tag
- Jenkins will force a new Helm release to AKS
- Jenkins will run automated tests
- Merging and closing the PR will merge the feature with master
- The merge with master will trigger a webhook to start the Jenkins pipeline
- Jenkins will push an image with tag
- Jenkins will force a new Helm release to AKS
- Jenkins will run automated tests
- Jenkins will destroy the pod
- Jenkins will promote the image by:
- retagging the image in ACR
- updating the github flux repo with the new image name
- Flux will see new changes and deploy new pod
Finding your pipeline
Within Jenkins, there exists the concept of organisation folders. This enables Jenkins to scan a GitHub Organization to discover repositories and automatically create managed pipelines for them.
Your pipeline will exist within a specific organisation folder which should have been defined when the pipeline integration was first configured.
See Step 4 of Jenkins Onboarding for details.
Finding your application
Within AKS, your application pods will exist within a namespace. The namespace will correspond to what has been configured in the Jenkins team-config.yml file.
Resource locks
Resources in Azure are locked in Staging or Production to prevent their destruction.
If you need to rename or destroy a resource, you will need to temporarily remove the lock. This can be done using the link in the readme of the Azure Resource Locks repository.
SSH access to resources
To SSH into Azure resources, such as a Postgres Database, you will typically need to go via a Bastion.
- Staging and production resources use the Production Bastion
- Other resources use the Non-production Bastion
To SSH into the bastion, you must first get a temporary access pass on myaccess.microsoft.com.
Building infrastructure in the demo environment
To build infrastructure in demo, you will need to create a branch called “demo”. All merges to this branch will be deployed in the Demo environment.
Key vaults build failures
If your infrastructure pipeline requires the use of a Key Vault to build, and the Key Vault is also itself being built in the pipeline, then it is common for the first deployment to fail. Just rebuild the pipeline.