All posts by admin

Snap and Docker on Ubuntu

Uncategorized

So for a long time, I’ve been wondering why I can’t use a locally mounted directory within one of my containers – I’d just get an empty directory on the volume mount point inside the container. A lot of googling yielded nothing, but then when I ran ps -aux | grep -i docker I noticed docker was running via snap.

It then occurred to me that snap has restrictions on what parts of the filesystem it allows it’s ‘snaps’ to read – this indeed was the problem so a quick uninstall of the snapped version of docker and installed via apt fixed the issue once and for all.

Key search terms for future me/anyone else: docker volume empty docker-compose volume empty.

Python request timeouts & HTTPbin

Uncategorized, , ,

Recently I added a static analysis tool (Bandit – which is an excellent tool and deserves it’s own post) to our Python microservices CI pipeline at work. Upon doing this, I discovered that our requests calls didn’t have timeouts set on them. By default, requests don’t have a timeout set.

At first, I thought it was going to be straight forward to just put a timeout on the remote calls, but upon reading the documentation I discovered that the timeout value was not quite what you’d expect – it’s not the wall time to complete the request.

timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds (more precisely, if no bytes have been received on the underlying socket for timeout seconds). If no timeout is specified explicitly, requests do not time out.

https://requests.readthedocs.io/en/latest/user/quickstart/#timeouts

Ok, so that then left me concerned about whether I’d really understood the documentation properly (or whether it was just plain wrong – which I doubted, but had to make sure so as to not screw everything up)… But how?

Then I discovered something really, really useful – https://httpbin.org/ . This is a bunch of fake API endpoints that behave as you request them to – eg, take 5 seconds to respond with a 200 code, or take 3s between each byte being sent. Great! I used this to prove that the timeouts do indeed behave as documented.

Definitely one to bookmark!

john-hunt.com back online

Uncategorized

After quite some time I’ve managed to get my server back online. The issue was that running it on a raspberry pi 3 alongside other things pretty much ensured the OOM killer would kick in and start killing things off.

I got one of those HP elitedesk PCs for only £40 off eBay to host this and it’s superb! This is now running in a docker container on a low powered 8gb system. You get far more bang for buck compared to a raspberry pi this way.

In terms of comments, it turns out I have about 3500 unmoderated comments, so it’s likely I’ll have to just bin them all due to time constraints. Apologies if you’ve written something useful.

I’ve got some interesting posts coming soon, including how this is all hosted now (major upgrades all around in that sense, but for no (big) extra cost. Some cool things I’ve been playing with and much more.

Docker OS X / Homebrew quick start

Linux

This post is mostly for my own reference. I’m still in the very early stages of understanding and using docker.


# The docker cli client app
brew install docker

# Tool for installing a docker machine (VM and or docker layer for holding containers (the whale))
brew install docker-machine

# Create a local docker machine using virtualbox as the VM, call it 'dev'
# Assuming this is where it gets clever as we can create docker machines for reference locally way off in the cloud
docker-machine create --driver virtualbox dev

# Nothing worked properly until I did this, not sure what it actually does, probably sets the active docker environment
eval "$(docker-machine env dev)"

# Pull down and register the whalesay container
docker pull docker/whalesay

# Run the cowsay command on the whalesay container with the argument 'boo'
docker run docker/whalesay cowsay boo

Of course, this gets more interesting when you’re running redis, nginx etc.. Not sure about dockerfiles yet, probably fairly simple. Really not sure about deployment.

Hashcat quick start!

Uncategorized

I recently had to use hashcat to verify some user credentials. Here are the commands I used:

# Salted md5 passwords (all the same salt in salt+password format):
hashcat -o found-passwords.pot -e salt.txt -m 20 -a 0 input-passwords.txt hashkiller.dic

Input file is just hashes from the database, one hash per line. I think you can put a :salt after each hash and omit the -e salt.txt from the command to use unique salts.

Why you shouldn’t trust WhatsApp’s end to end encryption

Uncategorized

Recently, the instant messaging app “WhatsApp” announced that it is using end to end encryption. While this is generally seen as a good thing, what most people seem to have forgotten is that we have absolutely no reason to trust WhatsApp.

Just because you’re told WhatsApp is using end to end encryption doesn’t really mean anything.. what algorithms are they using, how can we verify this. Most importantly, if the NSA (or someone else, perhaps with lots of money) asks WhatsApp to disable end to end encryption for a specific user – what’s to stop them? How would you know your instant messages are no longer encrypted?

What this means (at least to me) is that your supposed end to end ‘bullet proof’ encryption is probably worse than nothing – it’s lulling you into a false sense of security.

You can’t trust closed source encryption. Ever. Don’t start now.