Blog

A couple days ago my kubernetes cluster broke after a system upgrade. Resulting in the cluster generating the errors below.

Ever felt like your team isn’t able to deliver the flexibility you would really like? You’ve put in a lot of time to make sure you’re resilient, you barely have down time, the agile process is highly optimized. Still the new feature requests are piling up and it is getting harder and harder to keep all the happy flows straight. It might be time to shift who your customer is and treat your services as a product instead of a bespoke software solution.

After upgrading my cluster to 1.20.5 I had to move to the secure port and start to use authentication because of it. While I found a way to open everything up where it was easier to use (as a homelab) it is very insecure.

This post is to take a tiny step to securing the apiserver.

So it turned out upgrading my master from Fedora 33 to Fedora 34 (and thus from kubernetes 1.18 to 1.20) broke my master. Lets have a look at how to get it running again!

I recently changed how I hosted my blog as my last solution turned out not very stable. As my site isn’t exactly a moneymaker it had to be a low cost solution with some security standards. I’m still a responsible developer after all or so I like to think.

Mental models always help me remember how different things work. While they’re just approximations of a (black box) system, they generally describe some internal workings on an abstract level to explain observed behaviour.

These are 4 principles I try to adhere to when designing and developing software systems. They’ve worked for me at different levels of abstraction and have enabled me to rapidly deploy even complex systems. Lets have a look at what those are and give them some context on why I think they’re important to keep in mind. #1 Agility Agility is not just a way of working. Your codebase, deployments, decision making and leadership needs to be agile too.

The focus is often on resiliency around dependencies or exception handling. In other words when someone else’s code is not working as it should. It is just as important to look at how to deal with unexpected issues in your own code. How you can make sure that you can recover from breaking scenarios that you did not foresee. We’ll focus on some patterns that help you reduce impact of those errors and allow you to focus on a forward fix instead of rollbacks.

We all know the expression “Assumptions are the mother of all f*ck ups”. However that should not mean that you shouldn’t make assumptions. Or maybe we should call them hypothesis even when debugging. Let’s have a closer look.

In this article we’ll continue our journey on getting a multi node kubernetes cluster up and running. I’m doing this on VMs under kvm/qemu using Fedora 33 Server editions, purely out of preference and hardware available. This should however work with any hardware setup you’d like. If you don’t have your master running yet please read K8s The Hard Way