Case studiesDevSecOpsRegulated sector, Hong Kong
DevSecOps case study: Secure DevSecOps with GitOps for software supply chain to On-Premise Kubernetes
We helped a regulated organisation in Hong Kong check every open-source package against policy before it reaches a build, and gave its applications one standard path to on-premise Kubernetes: Jenkins to build, approval, then FluxCD to deploy from Git.
From commit to cluster
- 01Source repo
- 02Jenkins CI
- 03Scans
- 04Nexus
- 05Jenkins CD
- 06GitOps repo
- 07FluxCD
- 08Kubernetes
- Client
- Regulated organisation, Hong Kong
- Environment
- On-premise Kubernetes, designed to run air-gapped
- Duration
- 9 months
- Service
- DevSecOps
Where they started
Each team built and released software its own way.
The organisation builds software in highly varied technologies, such as Java, Node.js, Python, Go, C# and native mobile, and runs it as containers on on-premise Kubernetes.
Before
- Each team copied and changed its own pipeline script.
- Teams started builds by hand, with no automated tests or quality gates.
- Teams uploaded open-source packages by hand, with no security check.
- Security review came late, just before user acceptance testing.
- Teams deployed to production by hand with cluster commands.
- Image tags showed a branch and a timestamp, not a version number.
Every team kept its own copy of the pipeline script, so nobody could fix or improve the build in one place. Security review close to user acceptance testing made release dates hard to predict. Image tags carried a branch name and a timestamp instead of a version, so tracing a production issue back to the code meant digging through build logs.
Starting from a DevSecOps gap analysis
The work started from a DevSecOps gap analysis of the whole delivery lifecycle, based on workshops with the teams, process documents and a hands-on review of the pipeline configuration. The analysis mapped each step of the current flow to a proposed one and kept what already worked, such as the teams' familiar branching model. For each tool category it compared three or four products on features, resource needs and the skills needed to run them.
Two goals
A secure software supply chain and one standard CI/CD pipeline.
Secure the software supply chain
Fetch, cache and check every open-source component in one place.
One standard CI/CD pipeline
Give every Kubernetes application one pipeline that the organisation's DevSecOps governance can build on.
The organisation chose BSOL because our vast experience let us get up to speed quickly.
The platform must:
- run on-premise with high availability and no single point of failure in the application and storage tiers;
- keep the source control, CI and GitOps tools the teams already use;
- fit the organisation's change-management process, with strict approvals and a full audit trail before anything reaches production.
High availability
High availability on-premise, ready for an air-gapped network.
Our engineers built every tier as a pair of nodes behind a virtual IP, so the platform keeps running when a node fails.
We tested failover for every tier before acceptance and wrote the results into a recovery manual.
We started with the infrastructure, working alongside the organisation's platform operations team. Each component runs as a pair of nodes behind a virtual IP: the repository servers, the policy servers and the connection poolers in front of a replicated database, with shared replicated storage for the artifacts. Database, application and storage tiers run separately, and the whole design works in an air-gapped network.
The operations and recovery manual tells the in-house team what to do when a node goes down.
Supply chain
How we secured open-source packages with Sonatype Repository Firewall.
package ecosystems, one address
- Maven
- Gradle
- npm
- PyPI
- Go
- NuGet
- CocoaPods
- Docker
Our engineers put one Nexus proxy between the organisation's teams and every public package source. Repository Firewall checks each package against policy as it arrives and quarantines any package that breaks it before it can reach a build.
Sonatype Nexus Repository Pro is now the single place for every artifact. Hosted repositories hold the organisation's own images and packages. Proxy repositories fetch and cache approved public sources for eight package ecosystems. A group repository puts both behind one address, so developers, build agents and clusters all pull from the same place.
The assessment suggested several open-source scanners, but the organisation preferred fewer tools with commercial support. So we integrated Checkmarx One for application security testing and kept SonarQube for code quality.
Jenkins pipeline
Each new application joins the Jenkins shared-library pipeline with one YAML file.
Our engineers moved all pipeline logic into one Jenkins shared library that the DevOps team maintains.
Application teams now onboard a new application with one YAML file, without writing pipeline code.
The DevOps team keeps all pipeline logic in one Jenkins shared library, and application repositories carry no Jenkinsfile. A short YAML file describes each application: its name, source branch and the steps it needs, such as security scanning and image build. Seed jobs generate the Jenkins jobs from these files.
Every application uses three repositories: source code, Helm configuration and deployment state. Services share one Helm library chart, so each service chart is a thin wrapper that switches on what it needs, such as scheduled jobs, autoscaling or disruption budgets. Image tags combine a semantic version with the release branch and build number. The pipeline also writes that version back to source control as a Git tag, so developers can see what each build contains without opening the repository manager.
Promotion and rollback
GitOps promotion and rollback with FluxCD.
One commit promotes a release. Reverting it rolls the release back.
The same image moves from test to production unchanged, and only FluxCD changes the clusters. Every change has an approver, an author and a history in Git, so the release team can see what changed, when, and who approved it.
When CI pushes a new image to Nexus, a webhook starts the CD job, which deploys to the system-test environment automatically. Promotion to UAT and production waits for approval. The application team lead approves UAT, and the release team approves production. The CI pipeline also writes a release manifest listing every service in the release, so promoting a multi-service application takes one job and one commit.
The CD pipeline commits to the deployment-state repository. To roll back, the team reverts one commit or re-promotes the previous manifest; the earlier images are already in Nexus, so nothing needs rebuilding. Every change to every environment has an author and a history in Git.
Hand-over
Hand-over: the organisation's teams run the platform themselves.
Our engineers trained the organisation's administrators and developers hands-on and documented the platform. Their teams now run it and onboard new applications themselves.
- Installation, configuration and operations guides
- Recovery manual
- Developer guide
- Build and release guide
- Deployment configuration guide
- Step-by-step onboarding guide
- Hands-on training for administrators and developers
- Support after go-live
Results
Results
| Area | Before | After | Benefits |
|---|---|---|---|
| CI/CD pipeline | Different in every team, hard to maintain | One standard pipeline, with managed variations | One standard that is much easier to maintain |
| New application | Copy and paste pipeline config | Add one YAML file | Adopts the organisation's best practice from day one |
| Open-source packages | Maintained by hand | One proxy that scans every package | Automated, with security assurance |
| Security review | Late, close to UAT | Runs during coding and every build | Issues surface early, so release dates stay predictable |
| Production deploy | Deployed by hand | GitOps deployment | Automated after approval, with a full audit trail |
| Issue tracing | Search build logs by hand | Read the version from the image and Git tags | Trace a production issue straight back to its code |
Every new application now has a secure, standard path to production.
Securing your software supply chain or delivery path?
Twenty minutes, no preparation needed.
Book a discovery call