Category Archives: Linux

Recovering data from a WD Mybook Live 2TB / 3TB (or similar)

When the WD Mybook Live 3TB NAS was released, I went out and bought one and promptly put all my stuff on it. I have never kept anything *really* important on there as I didn’t have anything to back up all that data on to. Anyway, the NAS was destroyed in a thunderstorm one day but fortunately the hard disk still worked. Unfortunately the way WD formats these NAS hard disks is very strange indeed. Normal means of recovering data from them don’t work. Scouring google for tips on how to get your data back results in nothing useful.

I tried various hard disk enclosures.. these have no chance as they all pretty much only support up to 2TB disks. I tried various ext2/ext3 windows drivers.. no good. I tried linux machines with custom built kernels.. also no good.

There are basically three problems:

  1. The hard disk is big, USB enclosures hate that
  2. The hard disk uses a (new) GTP partitioning scheme, older versions of Linux will struggle.
  3. The hard disk ext4 partition (the one with all your data on) is formatted using 64kb sectors. This is the biggest hurdle as your PC running linux will not be able to mount it!

To recover your data:

A rough understanding of Linux is useful. In short you’re going to need to get the hard disk out of the NAS enclosure, stick it into a PC running a recent(ish) version of Linux and mount the partition using fuseext2. The trick to being able to mount the 64k sector disk is to avoid directly mounting it using the most excellent fuseext2 package. You’ll also need somewhere to put the recovered files – maybe another WD NAS? Maybe not :)

Step by step:

  1. I recommend getting an old PC (with sata ports inside) and an old hard disk for installing Xubuntu (no need for ‘heavy’ Ubuntu) on. Don’t plug in your WD hard disk yet, you don’t want to accidentally format it!
  2. Once you’ve installed xubuntu or whatever you’re using, turn off the machine and plug in the WD hard disk. Boot it back up again.
  3. Start a terminal and type:

    sudo apt-get install fuseext2 parted
    sudo parted -l

  4. The parted -l command will show you hard disks and partitions labelled /dev/sd.. something. You will see both the hard disk you installed linux on and the WD hard disk. The WD one will have a label such as: Model: ATA WDC WD30EZRS-11J (scsi), have a look down the list of partitions for the big ext4 one, like this:

    4      4624MB  3001GB  2996GB  ext4         primary

    Make a note of the disk (/dev/sdb) displayed underneath the hard disk model, and the partition number (in my case number 4). The path to the partiton for me is /dev/sdb4 (it may be different for you).

  5. Now you’re ready to mount the disk. To make life easier for you non-terminal types, I’ve provided instructions on mounting it in your home directory:

    sudo mkdir -p ~/WD
    sudo fuseext2 -o ro -o sync_read /dev/sdb4 ~/WD 

You may hit various hurdles along the way. I’m not entirely sure if older PCs can support really big hard disks. If you’re using an earlier mybook world or something I believe they used XFS and software raid partitions which this blog post isn’t really about.

Remember, always back up anything you care about!

Please let me know if you found this useful, and link to it so it helps others stuck in the same situation!!

More info: Mounting filesystems > 4Kb block sizes on Linux

How to prevent saslauthd sucking up memory

For about a year I noticed that very infrequently my VPS would run out of memory.. at first I thought it was probably just a wordpress plugin, but after a while I discovered it was actually saslauthd. This is a known bug (not known very well though..) with saslauthd on Debian. Anyway, here’s the fix – I’m not totally sure of the implications, so if you run a busy mail server I’d recommend you look into it a bit more before doing it:

I changed this line:

Update the file: /etc/default/saslauthd

OPTIONS=”-c -m /var/run/saslauthd”

to:

OPTIONS=”-c -m /var/spool/postfix/var/run/saslauthd -r”

I think this basically disables threading and enables forking of the process (or something like that) which is what was responsible for the memory leak.

So if you’re running out memory on your server, maybe give this a try :)

[Thanks to Djamu: http://www.howtoforge.com/forums/archive/index.php/t-52750.html ]

BigDump

Recently I’ve started having to work with sites hosted on shared hosting with phpMyAdmin and a measly 2MB upload limit for database imports. To get around this issue, I found a great little tool called bigdump which can just import a mysqldump file (even if gzipped) – just upload via ftp and import! Pretty sweet.

http://www.ozerov.de/bigdump.php

TWiT Live via MediaTomb for your WDTV Live (and probably ps3, whatever..)

I recently upgraded from a WDTV media player to a WDTV Live. The WDTV is just a simple set top box that allows you to play video on your tv, much like an xbox 360 and a whole bunch of other devices. The WDTV live allows you to play stuff over the network which opens up a whole host of cool stuff if you use the mediatomb DLNA server software available for linux (google tversity if you want something similar for windows..)

Anyway, I managed to get my wdtv live to play the live TWiT (http://live.twit.tv) video stream – meaning I now have a real internet tv station right up on my tv!

This is a bit rough and ready, but hopefully I’ll improve it over time. You’ll probably need to have a rough idea of what you’re doing.

Make sure transcoding is set to yes in the config:

<transcoding enabled=”yes”>

In mime type profile mappings in the config:

<transcode mimetype=”video/x-flv” using=”ffmpeg”/>

So anyway, you’ll want to add this profile to your config:

<profile name=”ffmpeg” enabled=”yes” type=”external”>
<mimetype>video/mpeg</mimetype>
<accept-url>yes</accept-url>
<first-resource>yes</first-resource>
<accept-ogg-theora>yes</accept-ogg-theora>
<agent command=”/usr/local/bin/ffmpeg-tr.sh” arguments=”%in %out” />
<!– <buffer size=”14400000″ chunk-size=”512000″ fill-size=”1024″/> –>
<buffer size=”5242880″ chunk-size=”102400″ fill-size=”1048576″/>
<hide-original-resource>yes</hide-original-resource>
</profile>
And create this shell script:
#!/bin/sh
exec ffmpeg -i “$1″ -sameq -f mpeg -me_method zero -aspect 16:9 – > “$2″
The final piece of the puzzle is to create a link to it in the mediatomb database via the web interface:
Type: External Link (URL)
URL: http://bglive-a.bitgravity.com/twit/live/high
Protocol: http-get
Class: object.item.videoItem
Mimetype: video/x-flv
Notes:
  • You might be able to/want to change the mimetype to a made up one, video/x-twitlive or something so it doesn’t conflict with other types.
  • You’ll probably want to turn off the other profiles you won’t be wanting.
  • Play with the buffer sizes, the commented out one seemed to go a bit funny for me.

A special thanks to aTc from #mediatomb on irc.freenode.net – without whom this wouldn’t exist.

Ubuntu Lucid (10.04) on Dell Studio 1555

Ok, so I decided to natively install Ubuntu 10.04 on my Studio 1555.. fairly impressed.. almost everything works out the box which is a bit annoying.

The only issue I’ve had is that the included proprietary ATI driver fails when you try to use suspend, however this is apparently easily circumvented by using the most recent driver from the ATI site (it was a bug with their driver.)

Using the open source driver results in poor power management, so I’d advise against doing that.

Trac quick start on Debian

Trac is pretty easy to set up on Debian, here’s a mini guide of what I did to get it working nice and quick. You’ll probably want to configure users etc afterwards, but this should be enough to get going.

Install:

apt-get install trac libapache2-mod-wsgi

Initiate the trac environment:

trac-admin /var/www/srdev/trac/ initenv

Set permissions:

chown -R www-data /var/www/srdev/trac/

Install the wsgi script and web resources. The first argument before ‘deploy’ should match the install environment (the path above.) The second argument following ‘deploy’ can probably go anywhere, but I just shoved it in the trac dir.


trac-admin /var/www/srdev/trac/ deploy /var/www/srdev/trac/www/

Next, we need to tell apache how to call the wsgi script, we do this using an alias. You can use / if you just want it to be in the same dir as your virtualhost, but in my case I wanted trac to be in a sub directory of the site (www.mysite.com/trac.) Add this to your virtual host (/etc/apache2/sites-enabled/whatever.)

The directory directive specifies some permissions for the script.

WSGIScriptAlias /trac /var/www/srdev/trac/www/cgi-bin/trac.wsgi
<Directory /var/www/srdev/trac/www/cgi-bin/trac.wsgi>
WSGIApplicationGroup %{GLOBAL}
Order deny,allow
Allow from all
&lt/Directory>

Finally, enable the wsgi module, and then restart apache. Now it *should* work :)

a2enmod wsgi
/etc/init.d/apache2 restart

MySQL 5 and old clients (like php4)

Warning: mysql_connect(): Client does not support authentication protocol requested by server; consider upgrading MySQL client in /blah/somepage.php on line 123

The reason for this warning is because PHP4 can’t communicate with a MySQL5 server unless it’s using old passwords (set by using the OLD_PASSWORD() function below.)

There are two simple ways to fix this. The first would be to upgrade your version of php. Unless you *really* need php4 (or below?!) upgrade! I’m not sure php4 is supported any more and should be considered a security risk. If you can’t upgrade, just do this in your database:

update mysql.user set password=OLD_PASSWORD('thepassword') where user = 'theusername';
and:
flush privileges;

This should do the trick. Be warned, if you issue any GRANT statements after this you will have to update mysql.user again.

WDTV – Western Digital TV media player

Picked up a WDTV last night for $200. As you may have guessed from some of my other posts, I’m into my A/V stuff, so this thing seemed like a bargain. Many reviews will rave about how great it is, and I must say it’s not too bad, but I have found there are bugs (even in the latest 1.0.1.2 firmware):

Playback of any AC3 file seems to result in a barely noticable (but very annoying) 100ms or so audio lag. This doesn’t drift which is good, but it makes watching anything with an AC3 soundtrack very annoying. I’ve tried various things to try and fix this and make sure it’s not me or my files, but the unit. Strangely, this only seems to occur when playing back NTSC files, and not PAL (25fps) files.

Other than that, it’s pretty good. I’ll hang on to it until I can test it on someone else’s setup. Otherwise it might have to go back to the shop!

*update*

After more testing with different files, it seems to be off-sync with a bunch of files, not just ones with an AC3 soundtrack. I also tested the device at a friends place before finally taking it back. If Western Digital manages to sort this bug out, I may consider buying it again…probably not though, manufacturers need to start testing stuff rather than just putting things out there and then releasing good firmware a year later.

I also just found out the WDTV violates the GPL, so I don’t want it anyway!

*update2*

Version 1.02 firmware is now released for the WDTV, and I’ve recently bought an AV reciever which has built in a/v delay stuff (and optical/hdmi inputs etc..) so perhaps it is time to re-purchase WDTV after all. Converting MKV/h264 to AVI/xvid for my xbox w/ac3 isn’t so great.

*update3*

I did re-buy the WDTV, and I’ve been very happy with it, but only if using firmware version 1.02.03 (I think it’s that one) which has all the sync issues sorted. Recommended!

Downmixing AC3 5.1 to stereo mp3 for xbox360 avi playback

*Updated!

The xbox360 doesn’t like divx/xvid files that have ac3 sound tracks, so we can use mencoder to convert these files so they use 128kbit stereo mp3 soundtracks instead. This isn’t too hard to do, just use a command similar to:

mencoder input.avi -ovc copy -oac mp3lame -lameopts cbr:br=128 -channels 2 -o output.avi

Easy when you know how.

Edimax EW-7728IN IEEE802.11n on Ubuntu

I recently purchased the Edimax EW-7728IN IEEE802.11n Draft 2.0 wireless card. Although I’ve not tested the performance yet, I have managed to get it to successfully connect to my Billion 7300N 802.11n wireless router at a rate of between 270 and 300Mb/sec.

Before I began I read on the Ubuntu forums that many people had problems trying to get the thing to work. I found it was fairly straight forward which is probably a result of the new(ish) drivers on the edimax website for Linux.

You can download the latest source from http://www.ralinktech.com/ralink/Home/Support/Linux.html – I didn’t see any restrictions or licenses on there for the source, so I’ve decided to compile and upload the modules as I need them. You can check back here to pick up the latest if you’re not confident compiling it yourself.

Please don’t try running the modules on kernels that they are not for. This will not work! You will need to compile it against headers for your currently running kernel if you don’t find it listed here.

Date Kernel Download
1st Oct 2008 2.6.24-19-generic rt2860sta.ko.bz2

Then you’ll want to extract the kernel module:

bunzip2 rt2860sta.ko.bz2

Now test that it works before you continue:

sudo insmod rt2860sta.ko

Now run iwconfig to see if the module loaded OK. If so, you should have the ra0 device listed.

If all is well, copy the module to the kernel module directory:

sudo cp rt2860sta.ko /lib/modules/2.6.24-19-generic/ubuntu/net/rt2860/rt2860sta.ko

Now we’d like to tell it to auto-load upon system startup. We can do this by adding the line: rt82860sta to the /etc/modules file. Easy!

Update!!

I’ve recently found a link to a dynamically building kernel module .deb file, which is great because it means you can just install this .deb on your system and it should work. With a bit of luck the Ubuntu team will include the driver in Intrepid, but I wouldn’t hold your breath:

http://www.array.org/ubuntu/dists/intrepid/eeepc/binary-i386/rt2860-dkms_1.7.1.1_all.deb