Sunday, October 23, 2005

Narrowly Avoided Trouble...

It seem as though we have narrowly avoided trouble here at THEmdg.org. You see, our budget is small. As a result, our infrastructure is not what it should be. Rarely are backups done on data, and there are NO full backups done...ever. So, if a drive goes down, we curl and cry.

Well, it happened today.

Went to pull a few mp3's out for a slideshow and found 0 files. "Oh yeah, maybe the drive just needs to be remounted," We thought. No dice. Upon further investigation, we found that the drive that holds about 80GB's of MP3's was unaccessable. Scary I/O Error stuff. The actual error had something to do with a bad superblock. A superblock a bit like the card catalogue of a partition. Without it, the data is still there, but ou can't find any of it.

After mourning the loss of an 8 year collection of music and after scheming how to replace them (a daunting task!), We had the idea to put a little more faith in linux to see if the data was recoverable.

Check out this magic:

Step 1 - Find out where the backup superblocks are stored.
-------------------
avast:/tmp# mke2fs -n /dev/hdb1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
14663680 inodes, 29304560 blocks
1465228 blocks (5.00%) reserved for the super user
First data block=0
895 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
----------------

Step 2 - Rebuild the jounral with a new (not corrupted) superblock
-------------------
fsck.ext3 -b 98304 /dev/hdb1
-------------------

After some bad block fixes etc etc
-------------------
/dev/hdb1: ***** FILE SYSTEM WAS MODIFIED *****
/dev/hdb1: 13333/14663680 files (21.3% non-contiguous), 15266103/29304560 blocks
------------------

What? That did it? We quickly remounted the drive:

avast:/mnt# mount butters

Woa...no errors...what now? It worked? Checking the mounted file system we were shocked to see all files intact! The MP3's are alive and well. Three cheers for linux magic.

The moral of the story? If you want to keep stuff you put on themdg.org...back it up. Chances are, we don't.

1 comment:

Anonymous said...

Holy Crap. That reminds me of when Jefferson totally erased our hard drive and I lost all pictures of Ethan's 2nd birthday. We now have an external hard drive which I make Jake update at gunpoint every few months.

Cami