CNP Artifactory
Table of Contents
Cloud Native Platform uses JFrog Artifactory OSS primarily to perform on-demand caching of upstream dependencies from public repositories consumed by Java Gradle CI builds.
This brings the artefacts closer to the build agents and significantly reduces impact of upstream issues or limitations (e.g. eliminates potential API rate limiting problems).
It can also be used to store build dependencies/inputs and outputs in a similar fashion to its counterpart in the Crime Platform but this use is currently very limited.
How does it work?
Java dependency caching
---
config:
theme: 'base'
themeVariables:
primaryColor: '#087ebf'
primaryTextColor: white
secondaryColor: 'grey'
tertiaryColor: 'white'
---
flowchart LR
subgraph caching[ ]
direction TB
artifact_cached{Is artifact requested for the first time?}
uncached[Yes]
cached[No]
uncached
upstream[Example: Maven Central]
cache[Cache on PVC Disk]
end
subgraph build[ ]
direction TB
merge[ PR Merged]
jenkins-ci-master[Jenkins CI Master]
jenkins-ci-agent[Jenkins CI Agent]
gradle-build[Gradle Build]
artifactory[Artifactory]
end
merge --> jenkins-ci-master
jenkins-ci-master -->|Use / Spawn Agent| jenkins-ci-agent
jenkins-ci-agent -->|Run build| gradle-build
gradle-build --> |Fetch dependencies| artifactory
artifact_cached --> uncached
artifact_cached --> cached
cached -->|Fetch cache| cache
uncached -->|Fetch upstream| upstream
build --> caching
Figure 1 - Flow shows when caching happens
Artifactory performs lazy / on-demand cache - this means it will only cache a dependency once it has actually been requested by a consumer at least once. Cache is saved internally by Artifactory (which works out to be an Azure disk provisioned by the PVC).
Caching functionality itself is part of a Gradle Java builder in CNP Jenkins Library.
Caching for other application builders such as Node JS or Python is not currently supported.
Where does it run?
Note: While the PVC is not ephemeral and data within it will remain for as long as the Artifactory Stateful Set exists within the AKS, redundant backup is currently not implemented. Please contact Platform Operations if you would like to start storing critical artefacts which cannot be easily recovered, re-synced or rebuilt within it.
Platform Operations runs and maintains the self-hosted Open Source (OSS) version of the artifactory within the PTL-Sandbox and PTL AKS clusters.
This is managed through the CNP-Flux-Config repository that is also used by all other AKS apps in the CNP - if you followed our onboarding guide then your application is likely configured in there as well.
---
config:
architecture:
idealEdgeLengthMultiplier: 2
---
architecture-beta
group platform(cloud)[Cloud Native Platform (AKS / Flux CD)]
group ptl-sandbox-ci(cloud)[PTL Sandbox CI] in platform
group ptl-ci(cloud)[PTL CI] in platform
group upstream-repos(internet)[Upstream]
align column upstream-maven artifactory-sbox
align row upstream-maven upstream-gradle-plugins upstream-other
service upstream-maven(database)[Maven Central] in upstream-repos
service upstream-gradle-plugins(database)[Gradle Plugins] in upstream-repos
service upstream-other(database)[Other upstreams...] in upstream-repos
align row artifactory-prod artifactory-sbox
service artifactory-prod(disk)[Production Artifactory] in platform
service artifactory-sbox(disk)[Sandbox Artifactory] in platform
service sbox-agent(server)[Jenkins Agents] in ptl-sandbox-ci
service ptl-agent(server)[Jenkins Agents] in ptl-ci
upstream-maven{group}:B --> T:artifactory-sbox
upstream-maven{group}:B --> T:artifactory-prod
artifactory-sbox:B --> T:sbox-agent
artifactory-prod:B --> T:ptl-agent
Figure 2 - Diagram shows which components artifactory interacts with
| Which Jenkins | Build URL | Which Artifactory | Artifactory URL |
|---|---|---|---|
| CNP Jenkins | https://build.hmcts.net/* | Production Artifactory | https://artifactory.platform.hmcts.net/* |
| CNP Sandbox Jenkins | https://sandbox-build.hmcts.net/* | Sandbox Artifactory | https://artifactory.sandbox.platform.hmcts.net/* |
| SDS Jenkins | https://sds-build.hmcts.net/* | Production Artifactory | https://artifactory.platform.hmcts.net/* |
| SDS Sandbox Jenkins | https://sds-sandbox-build.hmcts.net/* | Sandbox Artifactory | https://artifactory.sandbox.platform.hmcts.net/* |
How do I use it?
Artifactory cache is enabled by default / mandatory for all Java Gradle builds within Jenkins through CNP Jenkins Library. If you would like to use Artifactory cache when building on your local development machine you can configure Gradle to do this with:
pluginManagement {
repositories {
maven {
url "https://artifactory.platform.hmcts.net/artifactory/maven-remotes"
}
}
}
Keep in mind that configuring gradle repositories will only have an effect locally, within Jenkins this is always overridden to use Artifactory cache anyway.
Accessing Artifactory
Note: Local repositories are not visible unless logged in as an admin and Platform Operations does not currently offer non-anonymous accounts / login to teams. If you require a local repository to be added and / or authenticated access please contact us.
You can access Artifactory using URLs in the table above as long as you are connected to the F5 VPN.
This does not require a login and will allow you to browse artifacts available within the mirrors.
Caching issues and other build problems
If there is a genuine issue with the Artifactory which is causing your builds to fail Platform Operations can temporarily disable Artifactory caching in order to unblock your team while the issue is being fixed.
If you are having issues with your build check these first:
- Jenkins - Frequently Asked Questions
- Jenkins - Troubleshooting
If the issue is specifically related to Artifactory caching - for example, if you think Artifactory is down check #cloud-native-announce to see if maintenance / regular patching is underway.
You can also request support by creating a support ticket with the Platform Operations BAU team - simply use /Platops Help Request to summon the support bot which will guide you through the process.