Agenda

16:30 Door open, finding seat, enjoying snacks and networking

17:00 Welcome by the Cloud Native Copenhagen

17: 05 Welcome by our host Visma e-conomic

17:15 Talk 1: Securing custom LLMs using HashiCorp Vault

18:00 Break with food and drinks

18:45 Talk 2: The 3 Layers of our Platform and Helmping Ourselves before helping others

19:30 Break with snacks, networking

20:00 Thank you for today

Talk 1: Securing custom LLMs using HashiCorp Vault

Building custom Large Language Models (LLM) using private data dramatically benefits your business by allowing the model to understand specific documents and data unavailable in public datasets. However, there are issues with this approach; one problem is how you secure the data used to train the dataset, and another is how you ensure that this data remains private, specifically when using the public cloud. Secondly, how do you ensure that your compiled models are tamper-proof and have not been manipulated by a hostile actor? This talk will show you how to solve both problems using HashiCorp Vault, and how to do it using common Kubernetes patterns that you are used to. You will learn how to install Vault onto your own server using the Helm, how to install the operator, and all the configuration you need to secure and configure Vault. We will wrap up all this in a healthy layer of fun, and as a takeaway, you can try these techniques on your own using the interactive workbook provided at the end of the session.

Talk 2: The 3 Layers of our Platform and Helmping Ourselves before helping others

At Banking Circle, we've got an Internal Developer Platform which our developer teams use to run their software on, the easy and secure way.

We've designed that Platform with 3 layers in mind:

1. At the core of it, we have our Kubernetes clusters, together with different technologies running on it, such as ArgoCD, Trivy, Kyverno or Cilium
2. On top of the core layer, we build the "foundation", the area where a developer team can work
3. On top of that, we provide mechanisms to enable developers to deliver their software to the Platform

The tooling in the core layer is mostly handled with Helm charts. To help ourselves before helping others, we've also built an internal tool called Helmper. Helmper facilitates importing all the images used in any Helm chart to our own registries. It assists in deploying arbitrary Helm Charts in regulated and/or air-gapped environments.

After using Helmper to import images, we just edit the image references on our values file to point to our own registries. At Banking Circle, we use it in combination with Trivy and Copacetic, to ensure we remove fixable vulnerabilities from images before using them.