Bernd's Memory Leaks

Getting Started with Spark on Kubernetes

Tue, 25 Dec 2018 13:07:00 +0100

Apache Spark 2.3 brought initial native support for Kubernetes. With the recent release of Spark 2.4 the integration has been improved and client mode is now supported. Time to go through step-by-step and run some primitive Spark jobs.

How to Deploy a 'Jump Pod' on Kubernetes

Sun, 18 Nov 2018 18:09:00 +0100

Having a jump host, bastian host (on AWS, on GCP) or sometimes called edge node is a common way to access computing resources which are not accessible otherwise. In this post I’ll explore how to create a Kubernetes pod acting as such a jump server within the cluster, which is then used to create more pods. I’ll need this later when running Spark on Kubernetes, but you can use it to be able to check the cluster’s internal network, too.

A Comparison of the Basic Building Blocks in AWS, Azure and GCP

Fri, 31 Aug 2018 09:55:00 +0100

This sweet cloud promise, it sounds so simple - log into your cloud account, select a machine type and hit start. But as soon as you are running a dozen machines in all parts of the world for different projects and teams, you will start struggling with pulling apart your bill and granting rights to different people knowing which is what. Then you will find: this becomes unmanageable quite fast. Sure enough, every major cloud provider offers some kind of solution for this organisational mess, and this blog post will try to do a comparision. It is separated into two parts: the simple way, and the enterprise way.

Ein Blick auf die neue Hetzner Cloud

Tue, 27 Feb 2018 10:35:00 +0100

Hetzner Cloud: Die kleinste mögliche Cloud ist Anfang 2018 an den Start gegangen. Man bekommt schon ganz viel, obwohl fast alles noch fehlt.

Kubernetes wins over Apache Hadoop

Wed, 17 Jan 2018 09:55:00 +0100

Recently, Apache Hadoop 3 has been released. Really, who cares? Everyone and his aunt is jumping onto the Kubernetes train. While maybe it seems that Hadoop and Kubernetes are very different beasts, in reality, they are not. It’s kind of the same thing, but in different flavors. Yet, Kubernetes looks like the overall winner.

Things Google practices that your company doesn’t (part 2)

Wed, 29 Nov 2017 22:06:49 +0100

Joel Spolsky’s essays are legend. His collection of posts on his blog “Joel on Software” in the years from 2000 onwards have been very influential in the development community, not only because of their engaging writing style, but because they set clear directions for tech companies. One specific dogma he successfully implanted in my head was: “Never ever do a complete software rewrite” (You can read it here, please return afterwards). This probably is still very true today, because

“when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time”.

Let’s stress this dogma.

Things Google practices that your company doesn’t (part 1)

Mon, 20 Nov 2017 22:02:06 +0100

I’m an IT consultant. Never did I suggest to any of my clients they should interrupt a well-running application in production. Maybe that was not as professional as it might seem. Let’s see why.

What's the Required Score to Become Google Cloud Architect? -- Well, It Depends!

Mon, 20 Feb 2017 15:16:31 +0100

Early February 2017, I got email from Google saying I had passed the Cloud Architect certification exam. Yay! This post reflects what helped me preparing for the test.

Convergence of the Clouds

Thu, 12 Jan 2017 07:48:54 +0100

Recently, I’ve been involved with the Clouds. Not so much everybody’s darling AWS, but more the runner-ups in the race: Google Cloud and Microsoft Azure. Both cool kids, but everyone with his its very different style and philosophy of what it considers “cool”.