CI/CD integration
Wiring an Azure DevOps pipeline to CPP Artifactory. In nearly every case this means extending a template from cpp-azure-devops-templates rather than writing Maven or npm configuration yourself: the templates already handle credentials, the CA, caching and retries.
Extending the shared templates
A consuming repository declares the template repository as a resource and extends a pipeline from it:
resources:
repositories:
- repository: cppAzureDevOpsTemplates
type: github
name: hmcts/cpp-azure-devops-templates
endpoint: 'hmcts'
extends:
template: pipelines/library-validation.yaml@cppAzureDevOpsTemplates
parameters:
serviceName: my-service
Pick the pipeline that matches what the repository produces.
| Repository produces | Pipeline |
|---|---|
| A Java library | pipelines/library-validation.yaml |
| A context service | pipelines/context-validation.yaml |
| A UI library | pipelines/ui-library-validation.yaml |
| A UI application | pipelines/ui-validation.yaml |
| Terraform |
pipelines/terraform-ws-plan-and-apply.yaml and friends |
The steps that touch Artifactory
| Template | What it does | Takes artifactoryServer? |
|---|---|---|
steps/common/download-files.yaml |
Downloads settings.xml and settings-security.xml into ~/.m2 and the GPG key; also logs in to the non-live ACR and, unless told not to, sets up Gerrit access. Run this before anything that resolves or deploys. |
No |
steps/common/maven-deploy.yaml |
Branch-routed mvn deploy for context services, with Maven caching and retries. |
Yes |
steps/common/release-start.yaml |
jgitflow:release-start on main. |
Yes |
steps/common/release.yaml |
jgitflow:release-finish, GPG signing, snapshot check, version bump, and, separately, the release deploy to libs-release-sp-azure. See Publishing and promoting artefacts. |
Both (see below) |
steps/common/deploy-vld-stack.yaml |
Downloads the Helmsman and helm-diff binaries from the helmsman-binaries generic repository, among other validation-stack setup. |
No |
maven-deploy.yaml and release-start.yaml take a single artifactoryServer parameter, defaulting to https://libraries-internal.mdv.cpp.nonlive/artifactory. Leave the default alone unless you are doing something genuinely unusual; pipelines publish to non-live.
release.yaml is inconsistent with itself: it takes a required ARTIFACTORY_SERVER (no default, the calling pipeline must supply it: used by the main-branch deploy) and an optional artifactoryServer with the same default as above (used by the release/*-branch deploy). Both need to resolve to the same URL in practice; do not assume changing one changes the other.
download-files.yaml and deploy-vld-stack.yaml take no Artifactory URL parameter at all: they either need none, or read it from a pipeline variable set elsewhere.
Secure files
Artifactory credentials and the CA are Azure DevOps secure files, not variables. They are downloaded per run with DownloadSecureFile@1.
| Secure file | Used for |
|---|---|
settings.xml |
Maven resolution and the deploy credentials, under whatever server id the calling step’s deploy command uses (HASS in the templates above) |
settings-security.xml |
Decrypts the encrypted passwords in settings.xml
|
npmrc |
The npm registry configuration. Repository-committed .npmrc files in the crime-idam-* repositories point at api/npm/npm-virtual; the secure file’s exact content is not visible in any repository, but presumably follows the same pattern (see Consuming dependencies). |
npmrc_frontend |
The Azure Artifacts registry, for the second publish UI libraries do |
cpp-nonlive-ca.pem |
Trusting the non-live internal CA |
devops-team@hmcts.net.secret.asc |
GPG signing on the release path |
Never put any of these in a repository, a pipeline variable or a parameter.
Variable groups
| Group | Why a build needs it |
|---|---|
cpp-ghauth |
GitHub App credentials for the tag and version-bump pushes the release steps make |
cpp-nonlive-vault-admin |
HashiCorp Vault access, where deployment values are rendered |
cpp-nonlive-sonarqube-aks / -iaas
|
SonarQube, on the same jobs |
cpp-nonlive-slack-notifications |
Build notifications |
Variables the pipelines set
| Variable | Typical value |
|---|---|
ARTIFACTORY_SERVER |
https://libraries-internal.mdv.cpp.nonlive/artifactory |
artifactory_url |
https://libraries.mdv.cpp.nonlive/ (the external name, used with helmsman_artifactory_path for generic downloads) |
NPM_REGISTRY_URL |
$(ARTIFACTORY_URL)/api/npm/npm-virtual |
DEPLOY_SNAPSHOT_URL / DEPLOY_RELEASE_URL
|
$(ARTIFACTORY_URL)/libs-snapshot-local / libs-release-local (the older repository pair, used by the Rota application) |
RELEASE_BRANCHES |
main,release/8.x.x in library validation |
MAVEN_CACHE_FOLDER |
The local Maven repository the Cache@2 task restores |
Caching and retries
Maven builds use the Azure DevOps Cache@2 task keyed on maven | "$(Agent.OS)" | **/pom.xml, restoring into MAVEN_CACHE_FOLDER. The cache is skipped when mavenSharedCacheEnabled is true, because a shared agent cache is already in use. npm builds cache $(Pipeline.Workspace)/.npm keyed on the lockfile.
Every Maven deploy is wrapped in run_with_retry 3 from scripts/run_with_retry.sh. Uploads to Artifactory fail intermittently often enough that this is standard, so a build log showing a retry is not by itself a problem.
Agents
Artifactory is only reachable from the Crime network, so builds run on CPP-hosted agents: ubuntu-ado-agents-mdv for non-live, or the AKS-hosted ephemeral agents. A hosted Microsoft agent cannot resolve *.cpp.nonlive and will fail at the first dependency. If a job needs Artifactory, it needs a CPP pool.
Self-triggered builds
The release steps commit the version bump back to the branch, which would re-trigger the pipeline. The templates guard against this by checking the author of the last commit and setting shouldrun=false when it is devops-team@hmcts.net or contains azdevops. Most Artifactory-facing steps are conditioned on shouldrun.
If you add a step that pushes commits, follow the same pattern. If a pipeline mysteriously does nothing, check that variable before looking anywhere else.
Adding Artifactory to a new pipeline
- Extend an existing pipeline template if one fits. Most repositories should not need their own Artifactory steps.
- If you are writing steps, include
steps/common/download-files.yamlbefore the first Maven command. - Leave
artifactoryServerat its default. - Deploy with
-DaltDeploymentRepository=HASS::default::${ARTIFACTORY_SERVER}/${ARTIFACTORY_REPO}and let branch routing choose the repository, as Publishing and promoting artefacts describes. Do not hardcodelibs-release-sp-azure. - Wrap the deploy in
run_with_retry 3. - Run on a CPP agent pool.
- For npm, download the
npmrcsecure file and setcafilefromcpp-nonlive-ca.pem.
Things that catch people out
- Resolving works, deploying 401s. The
HASSserver id is missing from the agent’ssettings.xml, orsettings-security.xmlwas not copied alongside it. - The build resolves from Maven Central. The
securecentralprofile is not active. It overridescentralon purpose; without it you are not testing what the pipeline tests. - A snapshot dependency will not resolve. The shared
securecentralprofile enables releases only. - The image build fails on an
ADD. The artefact is not in Artifactory yet, or the version does not match. Check the deploy step, not Docker. npm auditfails. Expected: the Artifactory version does not support it. CPP pipelines run it against the public advisory database and tolerate failure.- A live deployment cannot find a package that exists on non-live. The live pull-through cache. See Publishing and promoting artefacts.
- Certificate errors. Trust
cpp-nonlive-ca.pem. Do not add-korstrict-ssl=falseto make it go away.
Related documentation
- CPP Artifactory: instances, repositories and access
- Publishing and promoting artefacts: branch routing and promotion to live
- Consuming dependencies: the client configuration these templates apply
- Tools and configuration: the wider CPP pipeline and configuration estate