Docker OS X / Homebrew quick start

This post is mostly for my own reference. I’m still in the very early stages of understanding and using docker.

# The docker cli client app
brew install docker

# Tool for installing a docker machine (VM and or docker layer for holding containers (the whale))
brew install docker-machine

# Create a local docker machine using virtualbox as the VM, call it 'dev'
# Assuming this is where it gets clever as we can create docker machines for reference locally way off in the cloud
docker-machine create --driver virtualbox dev

# Nothing worked properly until I did this, not sure what it actually does, probably sets the active docker environment
eval "$(docker-machine env dev)"

# Pull down and register the whalesay container
docker pull docker/whalesay

# Run the cowsay command on the whalesay container with the argument 'boo'
docker run docker/whalesay cowsay boo

Of course, this gets more interesting when you’re running redis, nginx etc.. Not sure about dockerfiles yet, probably fairly simple. Really not sure about deployment.

Hashcat quick start!

I recently had to use hashcat to verify some user credentials. Here are the commands I used:

# Salted md5 passwords (all the same salt in salt+password format):
hashcat -o found-passwords.pot -e salt.txt -m 20 -a 0 input-passwords.txt hashkiller.dic

Input file is just hashes from the database, one hash per line. I think you can put a :salt after each hash and omit the -e salt.txt from the command to use unique salts.

Why you shouldn’t trust WhatsApp’s end to end encryption

Recently, the instant messaging app “WhatsApp” announced that it is using end to end encryption. While this is generally seen as a good thing, what most people seem to have forgotten is that we have absolutely no reason to trust WhatsApp.

Just because you’re told WhatsApp is using end to end encryption doesn’t really mean anything.. what algorithms are they using, how can we verify this. Most importantly, if the NSA (or someone else, perhaps with lots of money) asks WhatsApp to disable end to end encryption for a specific user – what’s to stop them? How would you know your instant messages are no longer encrypted?

What this means (at least to me) is that your supposed end to end ‘bullet proof’ encryption is probably worse than nothing – it’s lulling you into a false sense of security.

You can’t trust closed source encryption. Ever. Don’t start now.

2015 – The year of virtual reality

For some time now I have been keeping myself abreast of the Oculus Rift project – a brand new virtual reality kit which is due for release this year.

Normally I wouldn’t be interested in things like this but I believe this new VR headset will change the face of not only gaming but many aspects of our lives, and not always for the better.

Back in the 90’s I had a go with a VR headset in London and it really put me off the idea. The terrible low resolution, the neck ache, the lag… It was really crap. From that point on virtual reality got forgotten about for another twenty years or so while the technology caught up with the concept.

Oculus aims, and supposedly delivers an experience which can effectively trick (part of) your brain into feeling the experience is very real. In a nutshell, virtual reality is here and this time it’s not going away.

So many people see this as just another gaming platform. That’s cool, but I believe it will be used for many other applications such as virtual meetings, and even eventually working.. Imagine working on a desk on a beach with palm trees?

It’s not all good news though. In this internet based age people spend less and less time going outside and talking to each other. VR could well be the ultimate end to people leaving their homes.. Maybe not for our generation, but I think the next generation will really struggle with this.. Games like World of Warcraft have already made lots of young people reclusive gaming addicts, but with this level of reality it could do some very serious damage if not used in moderation.

Either way, 2015 marks the beginning of the virtual reality age. Personally I think this is going to be as big, if not bigger than the internet in terms of its impact on society. This won’t happen this year, but in the next 5 or 10 years things will be very different in the western world.

Heartbleed – why you should be really worried

If you’re not already familiar with the heartbleed bug, have a look at this website:¬† – I don’t want to talk about what it is exactly as lots of people have done that.. just why you should be more worried than you are.

If you were (or still are?!) running an affected version of openssl:

  1. You almost certainly have NO way of telling whether someone attacked your server
  2. You may have been attacked through any service using SSL.
  3. All passwords & usernames, root or otherwise may have been logged by a remote attacker. These can be used unless changed
  4. All keys public and private, SSH, SSL or otherwise may have been logged.
  5. If your server was attacked you should consider ALL the contents of the servers hard disk revealed to the attacker.
  6. Even the memory of the server may have been compromised… this could be things like credit card details and other stuff you wouldn’t dream of storing un-encrypted anywhere.
  7. Your server may have been compromised ‘quietly’, you probably have no way of knowing this unless you run an IDS or something similar.

The worst bit is, you almost certainly won’t know if this stuff has happened. I’m not a fear monger, but this really is very, very bad.

If you’re just a person that used a website or service that used/uses an affected version of openssl:

Anything you did with that service is essentially in the hands of an attacker. For example, your usernames and passwords, your credit card information, your emails, your uploaded dropbox photos and work documents.

All in all, if you’ve used the Internet in the last two years, there’s a chance that your data has been stolen and you won’t know until it’s too late.

The only small amount of good news is that hopefully nobody has been exploiting this vulnerability. Hope is never a good security measure.

My biggest concern with the whole thing is not that my server or gmail might get attacked in the future, but that it may have been attacked without my knowing in the past. There’s not much anyone can do about this. Personal data may well have been taken – this is something you can’t fix.

For now, update your passwords, keys, revoke your old SSL certificates. Lock down your servers, secure your firewalls, set up IDS. Nuke your server and start again? Hope nobody really did anything too nasty. This bug may prove much worse than first thought.

PHP: Replacing short open tags with proper ones recursively in a big code base

We recently took on a horrible code base at work, with lots of open tags in the code like this:

<? calculateVat(123..

As far as I know this way of opening PHP code is deprecated and soon won’t be supported at all so I thought I’d just use sed to fix this but it wasn’t quite that simple.

Sed has no way of doing look-aheads with regular expressions meaning we can’t tell it to not turn <?php into <?<?php .. ! So we have to use perl (or something else that has ‘proper’ regexp):

# Convert <? (without a trailing space) to <?php (with a trailing space):

find . -name "*.php" -print0 | 
xargs -0 perl -pi -e 's/<\?(?!php|=|xml|mso| )/<\?php /g'

# Convert <? (with a trailing space) to <?php (retaining the trailing space):

find . -name "*.php" -print0 | xargs -0 perl -pi -e 's/<\? /<\?php /g'

Note, this could probably be improved by not using xargs (xargs has issues with spaces and funny characters in the path) – you’d probably want to use find’s exec command with the curly braces {}…

Anyway, this should fix up your entire codebase, but please CHECK the results afterwards, I only realised it was turning <?xml into <?php xml after checking..

Comments welcome ūüôā

Recovering data from a WD Mybook Live 2TB / 3TB (or similar)

This article was originally written in 2013 and applies to a fairly old model of the WD Mybook Live. The procedure here may well not work for you, please just use it for ideas. Also, check the comments as a lot of other people have tips!

When the WD Mybook Live 3TB NAS was released, I went out and bought one and promptly put all my stuff on it. I have never kept anything *really* important on there as I didn’t have anything to back up all that data on to. Anyway, the NAS was destroyed in a thunderstorm one day but fortunately the hard disk still worked. Unfortunately the way WD formats these NAS hard disks is very strange indeed. Normal means of recovering data from them don’t work. Scouring google for tips on how to get your data back results in nothing useful.

I tried various hard disk enclosures.. these have no chance as they all pretty much only support up to 2TB disks. I tried various ext2/ext3 windows drivers.. no good. I tried linux machines with custom built kernels.. also no good.

There are basically three problems:

  1. The hard disk is big, USB enclosures hate that
  2. The hard disk uses a (new) GTP partitioning scheme, older versions of Linux will struggle.
  3. The hard disk ext4 partition (the one with all your data on) is formatted using 64kb sectors. This is the biggest hurdle as your PC running linux will not be able to mount it!

To recover your data:

A rough understanding of Linux is useful. In short you’re going to need to get the hard disk out of the NAS enclosure, stick it into a PC running a recent(ish) version of Linux and mount the partition using fuseext2. The trick to being able to mount the 64k sector disk is to avoid directly mounting it using the most excellent fuseext2¬†package. You’ll also need somewhere to put the recovered files – maybe another WD NAS? Maybe not ūüôā

Step by step:

  1. I recommend getting an old PC (with sata ports inside) and an old hard disk for installing Xubuntu (no need for ‘heavy’ Ubuntu) on. Don’t plug in your WD hard disk yet, you don’t want to accidentally format it!
  2. Once you’ve installed xubuntu or whatever you’re using, turn off the machine and plug in the WD hard disk. Boot it back up again.
  3. Start a terminal and type:

    sudo apt-get install fuseext2 parted
    sudo parted -l

  4. The parted -l command will show you hard disks and partitions labelled /dev/sd.. something. You will see both the hard disk you installed linux on and the WD hard disk. The WD one will have a label such as: Model: ATA WDC WD30EZRS-11J (scsi), have a look down the list of partitions for the big ext4 one, like this:

    4      4624MB  3001GB  2996GB  ext4         primary

    Make a note of the disk (/dev/sdb) displayed underneath the hard disk model, and the partition number (in my case number 4). The path to the partiton for me is /dev/sdb4 (it may be different for you).

  5. Now you’re ready to mount the disk. To make life easier for you non-terminal types, I’ve provided instructions on mounting it in your home directory:

    sudo mkdir -p ~/WD
    sudo fuseext2 -o ro -o sync_read /dev/sdb4 ~/WD 

You may hit various hurdles along the way. I’m not entirely sure if older PCs can support really big hard disks. If you’re using an earlier mybook world or something I believe they used XFS and software raid partitions which this blog post isn’t really about.

Remember, always back up anything you care about!

Please let me know if you found this useful, and link to it so it helps others stuck in the same situation!!

More info: Mounting filesystems > 4Kb block sizes on Linux

Stopping bots spamming your forms

No doubt many others have thought of this already, but today I had a brainwave..

You have forms that use the dreaded captcha, which just sucks. Better forms use a honey pot trap but it’s still a bit weird. What if we checked to see how long a user had spent filling out a form – a bot will take (not much time) to complete the form where as a human being will take slightly longer, say anything over 5s.

Surely if we just check to make sure the form hasn’t been completed at in-human speeds we know if it’s a bot? If the form is somehow completed very quickly we can fall back to captcha, but for Pete’s sake – let’s stop using captcha by default.