Using a mobile phone to test an optical fiber for continuity

The best way to reliably test an optical fiber connection is by using speciality equipment. However, having them can be expensive so I am laying out a cheap, quick, dirty and somewhat unreliable way of testing them.

All that is needed is two people, one at each end, each equipped with a camera and a flashlight, both of which can be found in any modern cell phone.

The basic procedure is using the flashlight as light transmitter on one end and the camera as a light receiver on the other end. Using naked eyes to search for light through an optical fiber can be dangerous and should be avoided. That’s what the camera is for.

Multi-mode fibers are the easiest to check, since they scatter the light to multiple angles, so it’s easy to detect it from the other end.

However, for single mode fibers things can get tricky, since both the flash light and camera should be somewhat aligned. After a little effort though, one should be able to check it.

If the distance between the ends is too great for the flashlight power, one could connect the optical equipment on one end and detect its light using the camera from the other. (But if you have optical equipment readily available, why not just connect it directly and see if it works?)

I should mention again that is test is just for continuity. Nothing is better than speciality test equipment.

Permanently blacklisting specific domains from search results

Are you annoyed as much as me by some top Google results, like the ones from experts-exchange.com or freedownloadscenter.com and the likes?

Fear no more, the solution is here. Meet surfclarity, a simple firefox extension that hides search results from domains specified by you.

I swear, after using it for a while, besides boosting my search efficiency, I became a calmer man.

Tips for a new android phone

I bought an Android phone some time ago, the HTC Hero. After much fiddling with it, these are my suggestions for anyone interested in doing so too:

- Find out what you really want. A nice starting point is PdaDB. This site lets you choose features, like an Android OS and GPS, and lists every PDA having them.

- HTC Sense, a UI for Android phones exclusively from HTC, is really cool and actually makes Android useful. Consider buying a phone with HTC Sense included. Be careful since not all HTC models bundle it.

- Search E-bay before you buy. Most android phones are expensive and the HTC Hero I bought is no exception, as it costs about 550,00 Euros retail price here in Greece. Luckily, I bought it from e-bay for 411,00 Euros, including postage.

- If you buy from E-bay or other similar auction site, make sure the phone is described as “Unlocked”. Many phones are locked to a specific mobile carrier so you should probably avoid them. There are means to unlock a locked phone but most people will probably want to avoid the hassle.

- Some phones are marketed by a different name. For example, my “HTC Hero” is branded as “T-Mobile G2 Touch”. Other than the name, any re-branded phone is pretty much the same as the original. Besides making sure you buy it unlocked, you should keep in mind that re-branded phones require special firmware provided by the carrier. Other than a small delay for upgrades, that doesn’t seem to be a problem so far, as T-Mobile doesn’t keep back on android or HTC updates. That could change in the future though.

- If you buy from another country, keep in mind that the phone may be kept in customs until import taxes are paid seperately. This could likely double the cost of the phone, as they are not included in postage you paid obtaining the phone. Personally, I only buy from EU countries anything that costs more than 50,00 Euros.

- If you truly want to buy from Greece, then I suggest searching with Skroutz. It should find the best price for your phone in any Greek e-shop.

- Because of a bug in Android, if you don’t set up your Google credentials right from the start, you may need to factory reset your phone, erasing any customizations you may have made. Let this be one from the very first things you do.

- After you buy the phone, there should probably be an Android firmware update for it. Since the ROM upgrade does a factory reset on your phone, this should probably be too on of the first things to do.

- Market, the place to install android applications from, is strangely only fully accessible via the phone, even for simple tasks such as browsing or search. If you want to search and find apps for your android phone, and you probably will, I recommend going to Androlib.com or Cyrket.com.

- At first, this phone will consume most of your free time from just by playing with it. You will probably neglect your spouse and/or job the first days of use. Take appropriate actions to remedy the situation, like buying flowers or taking some days off work. Consider yourselves warned!

- You may want to buy some accessories for you phone. My advice is, again, use E-bay. I bought PDA holders for my bicycle and car at one third of the price I found in a cheap retail store here.

- You may want to protect your phone from scratches using a plastic wrap. One of the best is supposed to be InvisiShield from Zagg which claims to be virtually indestructible. Keep in mind that such protection covers the entire phone and may ruin the comfortable feel of it. Thankfully, one can choose to install only parts of it, so I used only the screen cover. However, if one wants just a screen protective cover, can as well choose one of the many other, and cheaper, available from other manufacturers. I bought 6 no-name covers for just 1,00 Euro from E-Bay, postage included! In any case, expect the feel of the phone to change to the worst.

- Since I chose not to wrap my entire phone in plastic wrap, I bought silicone skins from E-bay. Although a little more bulky than Zagg’s plastic wrap, the feel of it is extremely nice and protects it from damage in case I drop it. I definitely need it!

- A dock is a cradle, something like a base station for the phone. It is used both for charging and USB connection. I bought some for my phone, but I regretted. For me a dock is something I could just “place” the phone on, without plugging or using any force whatsoever, and instantly leave it charged and connected to the USB port. The docks I found needed significant two hand effort for unplugging, so they were useless to me. What is more, after a few uses, they didn’t support my phone correctly upwards and caused interrupted USB communication. I may try other ones in the future(…but I probably won’t…).

That’s all, I hope these will prove useful and bring painlessly more people to the wonderful Android platform.

Update: Added information about docks

Update 2: Added information about re-branded phones

SMART (un)reliability and SATA device order

A RAID5 software array under linux began complaining about unreadable sectors.

Immediately I initiated a SMART test, via the smartctl command but the hard disk checked out just fine. I have tens of disks under various software RAIDs and most of them throw an error now and then but in most cases there is no reason for an alarm. So I let it slip.

However, after a weekend, my logs became full of errors, even as the disk was reported as OK by SMART, although I re-ran the test.

I decided to replace it live, not bringing the server down, even though the server is an OEM computer with standard SATA HDDs screwed on the tower. It’s something I’ve done before so if one is careful with the wires, everything ends up OK.

Only it didn’t. Vendor hot-plugged  HDD’s have nice warning LEDs on them, to indicate which drive is the faulty one. At this point, I just had to guess.

I knew that my second HDD had the problem, and the M/B had 4 SATA’s, SATA0, SATA1, SATA2, SATA3. So, I disconnected SATA1, thinking that SATA numeric order is the same as the linux device order.

Unfortunately, that was not the case this time, although in my experience, it usually is. The result was that I disconnected a functional hard disk which was immediately marked as faulty by the MD subsystem.

When I realized it, I reconnected it, re-enable it, and wait for the array to rebuild. After that, I would shut down the computer to replace the really damaged disk.

The rebuild started OK but after a few minutes the damaged disk decided to die. Since the functional disk was in the middle of the rebuild, only one disk of the RAID5 array was truly online, so the array crashed.

So, I was left with a functioning disk with current raid5 data, a functioning disk  with semi-replicated raid5 data(which md didn’t recognize) and a really damaged disk with current raid5 data.

Sigh.

Fortunately, I had experience saving damaged md arrays in the past. The system partition was completely corrupted but my data partition was preserved, since the replication process didn’t have time to touch it. Although I did have a backup, obviously, in case I needed it.

Lessons learned:

- Don’t trust SMART alone when it reports a disk as OK. Do trust it, however, if it tells that it’s not. Better safe than sorry.

- Leave the online disk replacement for real hot plug systems, no matter how confident I am.

- Don’t guess which drive to replace, always check the Serial number. If that is not possible online, do it offline. Period.

- Prefer keeping the data partition on the end of the disk, so that replication will touch it last.

The bottom line is that I screwed up a basic array disk replacement out of laziness. If I did the replacement offline or if I checked the serials instead of guessing the correct disk, none of this would have happened. I should have known better…

Honey, I shrunk the HDD…

A funny thing happened on one of my hard disks. It shrunk.

I have a couple of ST315003 41AS in software RAID1 over USB for my backup needs, when suddenly one of them insists of being 500GB in size, instead of the nominal 1.5TB.

This is the kern.log output of the problematic disk:

scsi 2:0:0:0: Direct-Access ST315003 41AS PQ: 0 ANSI: 2 CCS
sd 2:0:0:0: [sdb] 976817134 512-byte hardware sectors: (500 GB/465 GiB)
sd 2:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdb] Mode Sense: 34 00 00 00
sd 2:0:0:0: [sdb] Assuming drive cache: write through
sd 2:0:0:0: [sdb] 976817134 512-byte hardware sectors: (500 GB/465 GiB)
sd 2:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdb] Mode Sense: 34 00 00 00
sd 2:0:0:0: [sdb] Assuming drive cache: write through
sdb: sdb2
sdb: p2 size 2909291175 limited to end of disk
sd 2:0:0:0: [sdb] Attached SCSI disk
sd 2:0:0:0: Attached scsi generic sg1 type 0

And this is the of the other disk:

scsi 13:0:0:0: Direct-Access     ST315003 41AS                  PQ: 0 ANSI: 2 CCS
sd 13:0:0:0: [sde] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)
sd 13:0:0:0: [sde] Write Protect is off
sd 13:0:0:0: [sde] Mode Sense: 00 38 00 00
sd 13:0:0:0: [sde] Assuming drive cache: write through
sd 13:0:0:0: [sde] 2930277168 512-byte hardware sectors: (1.50 TB/1.36 TiB)
sd 13:0:0:0: [sde] Write Protect is off
sd 13:0:0:0: [sde] Mode Sense: 00 38 00 00
sd 13:0:0:0: [sde] Assuming drive cache: write through
sde: sde2
sd 13:0:0:0: [sde] Attached SCSI disk
sd 13:0:0:0: Attached scsi generic sg5 type 0

The “new” size is reported on BIOS and Windows too, even on different machines.

Even Seagate’s SeaTools couldn’t find anything wrong with it, reporting it as a healthy 500GB disk.

Weird.

mysqldump doesn’t dump stored procedures

I use a mysqldump based backup script for my mysql databases.

Recently, bad memory modules caused a server to freeze randomly every other day. After the third time, some mysql databases were corrupted beyond repair, so I had to drop them and restore them. So far so good.

It seems, however, that mysqldump does not dump stored procedures by default, so I needed to restore them from somewhere else. Now I know better, I need to include the parameter “–routines” to mysqldump.

MySQL stores the stored procedures in mysql/proc table which I backup too. However, MySQL didn’t allow me to manually insert any records to that table. What is more, the stored procedures are saved in a BLOB field which PHPMyAdmin doesn’t display, so I couldn’t not see them and recreate them by hand.

(I know I could just SELECT them in the command line or fetch them directly from the mysql dump, but the truth is I was bored digging in that much text)

Hack/Solution: I restored the mysql/proc table under a different name, stopped mysql, remove mysql/proc table files and renamed the restored ones in their place.

Ugly, but quick and it worked.

http://www.passwordmeter.com/

Interesting: http://www.passwordmeter.com/

Measures the strength of a password via Javascript. It is also provided as a GPL download for (semi-)offline use.

Using the Egreat M33H Media Player with Linux

I set out to buy a media player and finally chose EGreat M33H.

It’s a great piece of hardware. It has HDMI output, Greek menus and Greek subtitle support, Ethernet, an internal bay for a 3.5″ SATA disk, it can play Matroska files, it works as a DVB-T PVR and last but not least, it doesn’t cost an arm and a leg. In other words, it is perfect for me.

I ordered one on Ebay, unwrapped and plugged it in. Unfortunately, it didn’t just work out of the box, or else I wouldn’t be making this blog post.

First of all, it didn’t recognize the hard disk I connected. It turned out that was because it cannot repartition/reformat a hard disk by itself, it needs one ready beforehand. My old PVR could do that so I guess I am a little spoiled.

That disk was my old system disk, so it had multiple partitions on it which probably confused the media player. So, I connected the hard disk to my PC and created just one big NTFS partition with mkfs.ntfs. After that, the hard disk was recognized perfectly.

Next problem was that it couldn’t receive any DVB-T channels. Trying to troubleshoot it, I searched the included CD, only to find that it had a manual in Chinese only.

Fortunately, E-Great site has everything one may need. Please note that Google’s first result for the E-Great many not be that one, but instead another E-Great site that has very little content to download.

However, I managed to be confused even with the manuals. Since no troubleshooting instructions where included, I just upgraded to the latest firmware. Fortunately, that solved the DVB-T reception problem.

Next, I needed to establish the network connection to play some content. Again, my assumptions where wrong.

I expected the media player to act as a NAS but this model doesn’t support such functions. So, how can I copy content to it via the network?

This device, it seems, scans the network for any server that support SMB, NFS or UPnP. Then, it allows you to play or copy content locally via the menus. The instructions were for windows where they even detailed installing Service for Unix on Windows for sharing via NFS!

Interestingly enough, it can only use content that is shared via a passwordless folder named “share”, either by SMB or NFS.

E-Great also provides a proprietary UPnP server for Windows that can be used to advertise content to the media player. My Ubuntu has MediaTomb which is a great UPnP server, but in the end I preferred to share via SMB.

The bottom line is that it works just fine with Linux.

Update: It seems that the device can only read NTFS, not write to it. Also, although it obviously runs linux, judging by the device names it uses when detecting the USB and hard disks, it doesn’t like ext2 too.

I settled to FAT32, created by mkfs.vfat. I’ll see how that turns out….

Converting a remote physical host to KVM virtual host

I administer a host located somewhere in Denmark at EasySpeedy.

Lately, the hardware could not cut it any more so I set for moving to a better server with more RAM.

Since I would bring the server offline for some time anyways, I figured I should convert the server to a KVM guest for better management.

My problem was that my old physical host had 250GB HDD and so did the new one, so there was no room for the conversion. I knew that I wanted to resize it to 200GB but would be the best plan of action? I could not simply plug a temporary harddisk.

At first I tried qemu-nbd. qemu-nbd is a daemon that mounts a qcow2 disk images on Netword Block Daemon. The NBD listens to a TCP port, which NBD clients connect to and mount the block device via the network. After I created a new qcow2 disk, I mounted under qemu-nbd and tried a network dump-restore. Three times I tried it and three times qemu-nbd crashed after moving some gigabytes of data.

Fortunately, NBD allows use of a socket too, instead of only TCP port for client-server connection. That worked OK.

The final step of every move is reinstalling GRUB, after chrooting to the target environment. Sadly, GRUB refused to be installed to my NBD mounted disk, since it complained about not being a BIOS device.

In the end, I booted my KVM via a rescue CD and did a network dump/restore directly to my virtual disk and tried to install GRUB. A new problem arose:

sysresccd:/#grub-install --no-floppy --recheck /dev/sda
Probing devices to guess BIOS drives. This may take a long time.
Searching for GRUB installation directory ... found: /boot/grub
The file /boot/grub/stage1 not read correctly.
sysresccd:/#

After Googling, I found that the Inode Size > 128 of an ext3 filesystem prevented GRUB from installing. Since Iwas doing a dump/restore move, the new filesystem was using the default option, which set the Inode Size to 256.

After a re-format and a new dump/restore, I was able to install GRUB and move on to booting my new guest.

Only it didn’t boot, it froze loading the kernel. My physical host had the 32bit k7 debian kernel which didn’t seem to like my i686 kvm emulation. I installed the i686 kernel and everything finally worked.

Things I will consider the next time. I hope they will help someone out there:

• qemu-nbd doesn’t seem to be able to move large amounts of data. Use a socket when possible.
• Since grub doesn’t seem to support installing to an NBD, use a rescue CD instead.
• The current version of GRUB doesn’t seem to like Inode Size greater that 128. Format the boot partition with “-I 128″
• Use a seperate boot partition, just for situations like this.
• Consider the possible CPU architecture change, and pre-install conservative kernels prior to the move.

Usefull: Mounting KVM qcow2 qemu disk images

Mounting KVM qcow2 qemu disk images

http://blog.loftninjas.org/2008/10/27/mounting-kvm-qcow2-qemu-disk-images/