My homelab People have shown an interest in how I run my stuff so I thought I’d open it up here. From a hardware perspective it’s very simple (and cheap) - intentionally, I have better things to spend money on like food. The software stack however is a bit more involved. I’ll go through it all here. The hardware setup Connection to the internet I have a gigabit fibre (FTTH) internet connection with a static IP address and (recently) IpV6 support.

It’s been quite a rocky year after switching jobs but one of the upshots is that I’ve managed to hopefully have finally sorted out this blog. You’ll perhaps notice all the old posts have finally come back since March. Hopefully some of it’s still of some use. Over the next few days and weeks I’ll be making some posts about what I use my new “homelab 😂” for: Personal knowledge management (PKM) How this blog is automatically updated from the PKM (fun) Media server Online book reading Webserver(s) Etc!

Earlier this week, I had an interesting problem someone asked me to solve. A custom analytics tool had been set up which was gathering lots of data at a rate of about 10 rows/second. The analytics table had 5.3 million rows, was innodb, had two fields which were nested JSON (stored as TEXT) and NO INDEXES. The customer wanted to be able to run reports on this table and was unable to as mariadb would just lock up (as expected) when trying to use any kind of clause.

john-hunt.com has been a wordpress site since the early 2000s, mostly a blog of interesting things I’ve discovered during my devops/software engineering career. I’ve recently switched over to using hugo to generate a statically hosted website rather than dealing with security updates that wordpress requires. Hopefully over the next couple of weeks I’ll migrate the old posts over as there were a couple of posts that generated quite a lot of interest (notably the external USB data recovery one.

So for a long time, I’ve been wondering why I can’t use a locally mounted directory within one of my containers - I’d just get an empty directory on the volume mount point inside the container. A lot of googling yielded nothing, but then when I ran ps -aux | grep -i docker I noticed docker was running via snap. It then occurred to me that snap has restrictions on what parts of the filesystem it allows it’s ‘snaps’ to read - this indeed was the problem so a quick uninstall of the snapped version of docker and installed via apt fixed the issue once and for all.

Recently I added a static analysis tool (Bandit - which is an excellent tool and deserves it’s own post) to our Python microservices CI pipeline at work. Upon doing this, I discovered that our requests calls didn’t have timeouts set on them. By default, requests don’t have a timeout set. At first, I thought it was going to be straight forward to just put a timeout on the remote calls, but upon reading the documentation I discovered that the timeout value was not quite what you’d expect - it’s not the wall time to complete the request.

After quite some time I’ve managed to get my server back online. The issue was that running it on a raspberry pi 3 alongside other things pretty much ensured the OOM killer would kick in and start killing things off. I got one of those HP elitedesk PCs for only £40 off eBay to host this and it’s superb! This is now running in a docker container on a low powered 8gb system.

This site is now hosted on my raspberry pi zero w - incredible! Hosting costs are now on a real shoestring budget. I’ll be tinkering with it to get the site to perform a bit better (microssd is not fast..) This was part fun experiment, part cost saving exercise.

This handy snippet is useful for seeing what merged PRs there are between two branches - eg master/release or whatever you’re using. I find it v.handy: git log --merges --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset' --abbrev-commit --date=relative origin/production..origin/master

Stick this in your .bash_profile or .bashrc file: function diffopen() { git diff --name-only "$1" | xargs -d '\n' pstorm; } Then in your project folder, say you want to open all files that differ from your master branch: diffopen master Note, on OSX you’ll need the GNU version of xargs for this to work, as I discovered the hard way..