All posts by admin

2015 – The year of virtual reality

Uncategorized

For some time now I have been keeping myself abreast of the Oculus Rift project – a brand new virtual reality kit which is due for release this year.

Normally I wouldn’t be interested in things like this but I believe this new VR headset will change the face of not only gaming but many aspects of our lives, and not always for the better.

Back in the 90’s I had a go with a VR headset in London and it really put me off the idea. The terrible low resolution, the neck ache, the lag… It was really crap. From that point on virtual reality got forgotten about for another twenty years or so while the technology caught up with the concept.

Oculus aims, and supposedly delivers an experience which can effectively trick (part of) your brain into feeling the experience is very real. In a nutshell, virtual reality is here and this time it’s not going away.

So many people see this as just another gaming platform. That’s cool, but I believe it will be used for many other applications such as virtual meetings, and even eventually working.. Imagine working on a desk on a beach with palm trees?

It’s not all good news though. In this internet based age people spend less and less time going outside and talking to each other. VR could well be the ultimate end to people leaving their homes.. Maybe not for our generation, but I think the next generation will really struggle with this.. Games like World of Warcraft have already made lots of young people reclusive gaming addicts, but with this level of reality it could do some very serious damage if not used in moderation.

Either way, 2015 marks the beginning of the virtual reality age. Personally I think this is going to be as big, if not bigger than the internet in terms of its impact on society. This won’t happen this year, but in the next 5 or 10 years things will be very different in the western world.

Heartbleed – why you should be really worried

Linux

If you’re not already familiar with the heartbleed bug, have a look at this website: http://heartbleed.com/ – I don’t want to talk about what it is exactly as lots of people have done that.. just why you should be more worried than you are.

If you were (or still are?!) running an affected version of openssl:

  1. You almost certainly have NO way of telling whether someone attacked your server
  2. You may have been attacked through any service using SSL.
  3. All passwords & usernames, root or otherwise may have been logged by a remote attacker. These can be used unless changed
  4. All keys public and private, SSH, SSL or otherwise may have been logged.
  5. If your server was attacked you should consider ALL the contents of the servers hard disk revealed to the attacker.
  6. Even the memory of the server may have been compromised… this could be things like credit card details and other stuff you wouldn’t dream of storing un-encrypted anywhere.
  7. Your server may have been compromised ‘quietly’, you probably have no way of knowing this unless you run an IDS or something similar.

The worst bit is, you almost certainly won’t know if this stuff has happened. I’m not a fear monger, but this really is very, very bad.

If you’re just a person that used a website or service that used/uses an affected version of openssl:

Anything you did with that service is essentially in the hands of an attacker. For example, your usernames and passwords, your credit card information, your emails, your uploaded dropbox photos and work documents.

All in all, if you’ve used the Internet in the last two years, there’s a chance that your data has been stolen and you won’t know until it’s too late.

The only small amount of good news is that hopefully nobody has been exploiting this vulnerability. Hope is never a good security measure.

My biggest concern with the whole thing is not that my server or gmail might get attacked in the future, but that it may have been attacked without my knowing in the past. There’s not much anyone can do about this. Personal data may well have been taken – this is something you can’t fix.

For now, update your passwords, keys, revoke your old SSL certificates. Lock down your servers, secure your firewalls, set up IDS. Nuke your server and start again? Hope nobody really did anything too nasty. This bug may prove much worse than first thought.

PHP: Replacing short open tags with proper ones recursively in a big code base

Linux, Programming

We recently took on a horrible code base at work, with lots of open tags in the code like this:

<? calculateVat(123..

As far as I know this way of opening PHP code is deprecated and soon won’t be supported at all so I thought I’d just use sed to fix this but it wasn’t quite that simple.

Sed has no way of doing look-aheads with regular expressions meaning we can’t tell it to not turn <?php into <?<?php .. ! So we have to use perl (or something else that has ‘proper’ regexp):

# Convert <? (without a trailing space) to <?php (with a trailing space):

find . -name "*.php" -print0 | 
xargs -0 perl -pi -e 's/<\?(?!php|=|xml|mso| )/<\?php /g'

# Convert <? (with a trailing space) to <?php (retaining the trailing space):

find . -name "*.php" -print0 | xargs -0 perl -pi -e 's/<\? /<\?php /g'

Note, this could probably be improved by not using xargs (xargs has issues with spaces and funny characters in the path) – you’d probably want to use find’s exec command with the curly braces {}…

Anyway, this should fix up your entire codebase, but please CHECK the results afterwards, I only realised it was turning <?xml into <?php xml after checking..

Comments welcome 🙂

Recovering data from a WD Mybook Live 2TB / 3TB (or similar)

Linux

This article was originally written in 2013 and applies to a fairly old model of the WD Mybook Live. The procedure here may well not work for you, please just use it for ideas. Also, check the comments as a lot of other people have tips!

When the WD Mybook Live 3TB NAS was released, I went out and bought one and promptly put all my stuff on it. I have never kept anything *really* important on there as I didn’t have anything to back up all that data on to. Anyway, the NAS was destroyed in a thunderstorm one day but fortunately the hard disk still worked. Unfortunately the way WD formats these NAS hard disks is very strange indeed. Normal means of recovering data from them don’t work. Scouring google for tips on how to get your data back results in nothing useful.

I tried various hard disk enclosures.. these have no chance as they all pretty much only support up to 2TB disks. I tried various ext2/ext3 windows drivers.. no good. I tried linux machines with custom built kernels.. also no good.

There are basically three problems:

  1. The hard disk is big, USB enclosures hate that
  2. The hard disk uses a (new) GTP partitioning scheme, older versions of Linux will struggle.
  3. The hard disk ext4 partition (the one with all your data on) is formatted using 64kb sectors. This is the biggest hurdle as your PC running linux will not be able to mount it!

To recover your data:

A rough understanding of Linux is useful. In short you’re going to need to get the hard disk out of the NAS enclosure, stick it into a PC running a recent(ish) version of Linux and mount the partition using fuseext2. The trick to being able to mount the 64k sector disk is to avoid directly mounting it using the most excellent fuseext2 package. You’ll also need somewhere to put the recovered files – maybe another WD NAS? Maybe not 🙂

Step by step:

  1. I recommend getting an old PC (with sata ports inside) and an old hard disk for installing Xubuntu (no need for ‘heavy’ Ubuntu) on. Don’t plug in your WD hard disk yet, you don’t want to accidentally format it!
  2. Once you’ve installed xubuntu or whatever you’re using, turn off the machine and plug in the WD hard disk. Boot it back up again.
  3. Start a terminal and type:

    sudo apt-get install fuseext2 parted
    sudo parted -l

  4. The parted -l command will show you hard disks and partitions labelled /dev/sd.. something. You will see both the hard disk you installed linux on and the WD hard disk. The WD one will have a label such as: Model: ATA WDC WD30EZRS-11J (scsi), have a look down the list of partitions for the big ext4 one, like this:

    4      4624MB  3001GB  2996GB  ext4         primary

    Make a note of the disk (/dev/sdb) displayed underneath the hard disk model, and the partition number (in my case number 4). The path to the partiton for me is /dev/sdb4 (it may be different for you).

  5. Now you’re ready to mount the disk. To make life easier for you non-terminal types, I’ve provided instructions on mounting it in your home directory:

    sudo mkdir -p ~/WD
    sudo fuseext2 -o ro -o sync_read /dev/sdb4 ~/WD 

You may hit various hurdles along the way. I’m not entirely sure if older PCs can support really big hard disks. If you’re using an earlier mybook world or something I believe they used XFS and software raid partitions which this blog post isn’t really about.

Remember, always back up anything you care about!

Please let me know if you found this useful, and link to it so it helps others stuck in the same situation!!

More info: Mounting filesystems > 4Kb block sizes on Linux

Stopping bots spamming your forms

Programming

No doubt many others have thought of this already, but today I had a brainwave..

You have forms that use the dreaded captcha, which just sucks. Better forms use a honey pot trap but it’s still a bit weird. What if we checked to see how long a user had spent filling out a form – a bot will take (not much time) to complete the form where as a human being will take slightly longer, say anything over 5s.

Surely if we just check to make sure the form hasn’t been completed at in-human speeds we know if it’s a bot? If the form is somehow completed very quickly we can fall back to captcha, but for Pete’s sake – let’s stop using captcha by default.

PHP frameworks… a serious issue

Programming

Several years ago a new framework called CakePHP was released and I was quick to jump on the band wagon. The promise of having base code that does 90% of the work for you was too tempting and we quickly got burned trying to perform more complex SQL queries – hitting the boundaries of what was possible and having to hack our way around it. Ugh.

Still, when you go back to writing an MVC style web app from the ground up you quickly realise that life would be much easier with a framework. A year or so after CakePHP I had a go with CodeIgniter and found it’s less strict Model paradigm easier to work with as you could just pass straight SQL to your database. Sure it lacked some features but it was easy to work with and easy to pick up.

Then I became freelance and did a bunch of work with WordPress, which I really like but it’s not a framework (though I do think it’s under-used and underrated).

So, as of August last year I started a new job where we use Zend Framework. My first impression was ‘wow, this is cool – very professional!’ 6 months in, and though I’ve not used it lots I think I can safely say it’s nothing like Cake or CI – it’s far more complicated, there are no walk through guides – you must invest good time learning it (something which there wasn’t time for me to do..)

I now know enough about ZF1 to get by, but last night I had a look at zend framework 2.. it appears as though a whole new level of complexity has been added. I spent a couple of hours trying to figure out what the hell was going on and (for the time being) gave up!

I concluded a few things:

  1. Zend Framework is a ‘nuts and bolts’ framework.. it feels much more like a bunch of helper classes to help you make your own framework. This is flexible, but complicated and time consuming.
  2. Things that are complicated are more easy to break.
  3. Zend Framework is nothing like the other frameworks out there. It takes a long time to develop good things.
  4. The learning curve is steep. Steep enough to make you wonder if it makes any sense for a business to pursue it..imagine if all your future employees need to spend 2 months learning something before they can begin work?
  5. Following on from 4 – if other companies/startups etc. aren’t using zend framework because it’s too time consuming for creating websites compared to other frameworks then does it make me valuable as someone who knows ZF? I guess you could look at this one in two lights really. In some ways it could make me more valuable!
  6. One worrying thing about working with ZF1 was that different people on the team approached similar work in very different ways because of the flexibility ZF offers.. one person used Zend_Db_Table where another might use something else.. one person may have written a wrapper for something, put something in a library, a service or a base controller – things can quickly get out of hand.

Food for thought, but it’s becoming apparent that investing your time in one framework is a very serious thing to do.. everyone knows PHP, but when the frameworks become almost as complex as the language itself it becomes a serious investment of time for a web programmer.

I’d very much appreciate people’s comments on this 🙂

Please note: I’ll be revisiting this post on and off as I gain more experience with both ZF1 and ZF2 and I’ll be looking into other frameworks such as Yii and Symfony over the coming months.

Update

Uncategorized

Just a quick update.. I assure you this site is still active 🙂 I’m currently extremely busy but I have lots of cool things coming up.. arduino based notification system with REST service, technical project management ideas, gitflow, and mindful programming technique.. nice.

Converting characters

Uncategorized

I’ve often had issues with character sets getting muddled up.. generally from my clients pasting ISO-8859-1 special chars into my sites that are UTF-8. Today I discovered the super-handy iconv() function that’ll convert character sets.. in this case I needed to drop down to ascii for generating pdfs with dompdf:

$output = iconv('UTF-8', 'ASCII//TRANSLIT', $string);

Pretty handy!