Red Robot s.r.o.

My primary contribution to this project is delivering containerisation solutions for existing applications within certain state administration systems, for the subcontractor Red Robot s.r.o.

The core of the solution is built on the WSO2 Micro Integrator, with additional services and integrations implemented as CarbonApps. For implementation simplicity and maintainability, standard Spring Boot applications plus a few frontends are gradually being added alongside them.

The main pain of this project, though, was the process of deploying to environments and releasing new versions, and the testing tied to it. It was done by literally packing source code, Ansible and bash scripts into a zip archive by hand, which the operations administrator then had to install on the target environment laboriously, manually and exactly according to the procedure.

Phase 1

In the first phase of the project, containers for the individual applications were created, together with the CI/CD pipeline for building them, checking them and pulling in their dependencies.

Further tasks were to create, or install and configure, the following for the supplier:

• a version control system for storing source code and running CI/CD (GitLab)
• • migration of the supplier's existing repositories, creating and configuring further ones
  • migration of the subcontractor's repositories
• applications for storing artifacts (GitLab, Nexus)
• a registry for storing docker images (Harbor)
• an application for securely storing sensitive information (Vault)
• deployment management, monitoring of the Kubernetes cluster (ArgoCD)

This work made it possible for the supplier's team to install the Kubernetes clusters afterwards. Everything mentioned above is handled by a CI/CD pipeline that runs terraform scripts for the individual applications, since the number of repositories in GitLab is approaching a hundred (if not past it) and they are split by development area into groups and subgroups with a specific division of access rights - that volume would be beyond human capacity to manage by hand, and on top of that the second phase builds the same environment at the end customer as well.

Phase 2

Once the Kubernetes clusters were installed in the supplier's environment, the standard work of deploying containerised applications could get going,

• creating GitOps repositories with helm charts, the individual resources and ArgoCD applications
• creating and modifying helm charts, adding and modifying externalSecrets, configMaps, services...
• adapting the containerised applications to run in the cluster, initContainers, jobs, ...

The second phase also covered the installation of the Kubernetes cluster and the applications mentioned in phase 1 in the end customer's environment by the subcontractor's team, plus further adjustments and refactoring of HELM charts and templates and a refactor of the secrets in Vault.

Another challenge for me was implementing a CI/CD pipeline that carries the following from the supplier's environment over to the end customer's environment - which for security reasons and out of caution runs in "island" mode

• artifacts from Nexus
• docker images in Harbor
• for selected groups, the versions tagged for release / fix from GitLab
• it also watches the correct order in which the individual repository groups are copied, and the start and completion of the CI/CD builds into the internal GitLab artifactory

The challenge here is also cutting out anything that might want to download itself from the internet and redirecting it to Harbor, where I have to keep my own copy - for example the various docker images used in CI/CD pipelines, Kubernetes init containers and the like.

Result

• the subcontractor and the other parties now work (store source code, use the K8s clusters) in the supplier's environment
• releasing a new version runs automatically
• • tagging the affected libraries and applications
  • • including the GitOps repositories
  • a ping to the end customer
  • the customer runs the "migration" pipeline
  • the customer runs the "configuration" pipeline with Terraform for any changes to the settings of the infrastructure applications
  • adding / filling in the new Vault secrets by hand according to the documentation
• after the preceding steps the customer should be able to head into the ArgoCD/Kubernetes clusters and start spinning the applications up.