Tempered Works

dbt announced “model contracts” in the recent v1.5 release. This looks like a great feature for dbt, but reminded me that I’ve been using contract testing with dbt for a couple of years now, inspired by Pact consumer-driven contracts, but never talked about it. There are some differences, for example: dbt’s new feature is very dbt-centric, the approach I’ve used isn’t - dbt certainly helps, but it isn’t necessary. There’s a GitHub repo to follow along with.

[Read More]

Following high-profile incidents like the 2017 Equifax Breach, checking your dependencies for vulnerabilities is a common practice today. We can use great tools like OWASP Dependency Check, Trivy and Snyk in our builds to raise the alarm when vulnerabilities are found.

The question that I find comes up isn’t whether we should check dependencies - but when?

[Read More]

So this annoying and trivial little problem catches me out every so often. I am always misled by the error message! You’ll see what I mean shortly. For context, it usually happens when I’m working in Docker containers on a build.

[Read More]

Argo is a lightweight, Kubernetes-native workflow solution. Workflows are implemented as Kubernetes manifests, so Helm is a natural choice for packaging them.

Helm also supports templating values which can be really helpful - but that’s where we run into a problem. Helm uses mustache-style string interpolation, and so does Argo.

[Read More]

Lately, I’ve been working in virtual machines to strengthen my security posture for clients. There’s more to come on that, but for now I wanted to share a fix for a confusing problem I had. I was trying to install Fedora 31 as a Packer build. [Read More]

In Part 1, we built a simple Spring Boot webapp and demonstrated a surprising performance problem. A Gatling performance test simulating different numbers of users each making a single request showed our webapp unable to keep up with 40 “users” making one request per second on my fairly powerful computer. We eliminate a couple of potential causes in the first part of the article. If you just want to know what was causing the problem, you can go straight there. [Read More]

Just after the rest of the team had left for their Christmas holidays, my colleague and I discovered a weird performance problem with a Spring Boot application we’d just started writing. This is a the story of discovering the problem and the detective work that led us to the culprit hiding in plain sight. We’re going to recreate the app and the performance tests, but first I’ll tell you how we got here. [Read More]

Dunnhumby is a retail data science company that I’ve been working with lately. I’ve enjoyed writing a couple of articles for their Data Science and Engineering blog. The first is a slightly extended version of an article on here, Scala Types in Scio Pipelines. The more recent article is original and talks about the experiences we’ve had putting together streaming demos of real-time streaming data processing solutions. If you’re interested, you can find that article at Building Live Streaming Demos [Read More]

Every company needs a website, and Tempered Works is no exception! Having bought the domain names when I set the company up, I’ve been putting off getting a website up and running because I’m not really a front-end creative type. When I heard Jason Lengstorf talking to The Changelog about GatsbyJS, I was intrigued… so I tried it out. Why GatsbyJS? I think GatsbyJS is interesting when compared to other static site generators because it’s based on GraphQL and React. [Read More]

Data pipelines in Apache Beam have a distinctly functional flavour, whichever language you use. That’s because they can be distributed over a cluster of machines, so careful management of state and side-effects is important. Spotify’s Scio is an excellent Scala API for Beam. Scala’s functional ideas help to cut out much of the boilerplate present in the native Java API. Scio makes good use of Scala’s tuple types, in particular pairs (x, y). [Read More]