2011-08-23

Playo Android

This could be an interesting piece of hardware: an android phone, minus the phone, with HDMI output and USB ports.
If, as they say, it will be sold for less than 200€, and it won't have a locked firmware, it could become the ultimate living room geek toy.
It has a full keyboard, while others don't, and by installing a full linux distribution on it (like some android phones) you could have your own miniserver.
I assume that, given its phone roots, power consumption is still as little as a phone.

2011-08-22

Small & Quick VM to test network connectivity

It's quite some time I've been searching for a VM that allows to troubleshoot networking problems on ESX machines.
I've used DSL and his child Tinycore. They are good, they even offer a GUI that may be wasteful for simple network troubleshooting, but both lack tcpdump (at least in the downloadable ISO).
Now I've found something just right for the job.
PLD RescueCD: it's reasonably small (89MB ISO), runs entirely from RAM (as low as 40MB), it has tcpdump on board, nc, nmap and a load of other tools.
As a plus, it has no GUI :-)

I've used it to make sure the correct VLAN was mapped on a ESX machine:
Create a custom VM with 1 cpu, 64MB ram, no disk, 1 ethernet adapter bound to the port group you're investigating, and map the virtual CD to the PLD ISO file.
Boot up the VM and you'll be dropped to the root prompt.

#tcpdump

Even if ESX does not allow promiscuous mode on VM adapters, you'll still be able to see ARP requests from other machines that are on the same VLAN as your own VM.
That should be enough to understand if you've pinched the right VLAN.
Someone may say that ESX itself shows a network hint next to each adapter, but I've seen that such hint is not always reliable or at least is not real time, so a newly attached VLAN may not be immediately visible.

2011-08-21

Copying files with on-the-fly compression

having to copy a really big file in the shortest time, this might be useful:

gzip -1 </path/to/sourcefile | ssh -c blowfish-cbc target.host gunzip ">" /path/to/destfile

the light compression (-1) seems to be a good tradeoff: higher compression ratios lead to higher overall transfer time.
the blowfish encryption puts lower strain on the cpu.
using an old 10 Mbit connection, i've copied a vm disk file with a throughput of 13.980.000 bytes/sec

gzip vs bzip2

i've lost track of how many vm images i've moved back and forth.
trying to find the fastest way, i've done some comparisons of compression options available.
benchmark was run against a 10GB vmdk containing only a clean windows install.

Packer      Minutes    Compressed Size   Percentage
bzip2 -1    35         5882676084        56.1%
bzip2 -9    41         5805348305        55.3%
gzip -1     13         5993116500        57.1%
gzip -9     20         5834635837        55.6%

while bzip2 is the clear winner over size, its running time limits its usage where time is not an issue.
most of the times gzip -1 is the best choice.

2011-08-20

The case of the missing gateway

I've never quite understood the purpose of the NETWORK entry in the ifcfg file of the redhat-based linux distributions.
It can be derived from ip and mask, and if you leave it out altogether, the network works fine as well.
But if you decide to use it, be sure to write it correctly.
The other day I found an ESX machine (that's using redhat as the service console) that loses its default gateway after each reboot.
The gateway was correctly specified in the /etc/sysconfig/network file. After some fiddling I've noticed that the NETWORK clause in /etc/sysconfig/network-scripts/ifcfg-vswif0 was totally wrong. It's not even possible to set it with the vsphere client so I think someone mistyped it while manually configuring the ESX.
Once corrected, a

#service network restart

brought back the ESX online with the vcenter server.

2011-08-19

X bandwidth usage

while evaluating X access to linux boxes i've made some stats using different access protocols:
  1. nx client from www.nomachine.com (encapsulate and compress X protocol)
  2. vnc
  3. ssh -X
i've recorded a sample session with wireshark and here are the raw numbers:


Client                       Bytes on the wire
nx (lan quality)             4924556
ssh -X                       4383203
vnc (16 bit high quality)    4256691
nx (adsl quality)             586752
nx (modem quality)            480296
vnc (8 bit low quality)       433575


aside from the naked data, there are some other factors that might weight in:
  • at low data rates, nx feels snappier and looks better than vnc.
  • at high data rates, plain X with ssh tunnelling seems a better choice, but only if you are using a native linux client: the nx viewer can run on windows with less effort than an Xming+putty setup, and it can publish the full Gnome/KDE desktop of the remote machine.

2011-08-18

Comparing Mhz from 2005 to 2011

i know i may asking for the holy grail here, but i'd like to share some thoughts on that.
a couple of months ago i was planning an hw refresh for a farm built way back in 2005, based on Xeon MP 3.0 Ghz processors, with newer blades with some X5660 processors.
looking at the vcenter statistics i can see the actual farm is using something like 50Ghz as an average, and has a physical to virtual cpu ratio of about 1.5.
by a rule of thumb, i may say that on newer processors, doubling the p2v cpu ratio won't cause any problem, even if i don't have any evidence to prove it.
speaking of Mhz instead, i'm trying to find out how a 2011 core compares to a 2005 one.
even if X5660 cores are running at 2.8Ghz, i don't think they're "slower" than Xeon MP ones.
i've fiddled a litte with the published spec.org CINT2006 benchmark results
here is a sample:


 xeon 3.8Ghz        11
e5345 2.33Ghz       16
e7330 2.4Ghz        17
e7458 2.4Ghz        20
x5460 3.16Ghz       27
x5560 2.8Ghz        35


so, it seems that the X5560 is at least 3 times faster than the old xeon.


after some weeks, i've stumbled upon this blog post that seems to come to the same conclusion
http://it20.info/2011/06/the-cloud-and-the-sunset-of-the-ghz-based-cpu-metric/
MIPS have served well for the mainframe world, why shouldn't they work for x86 also?

2011-08-17

Linux V2P

Virtualization may be the best thing since the microprocessor itself, but sometimes you may have to backtrack your own footsteps.
Here are the steps to convert a Red Hat 5.5 linux vm running on an ESX to a physical box:

Using a disk imaging tool, dump the vmdk of the vm to the internal disk of the target machine.Boot the physical box with the Red Hat CD in rescue mode
Do not mount any partitions, but select "continue" to jump into the shell.

Mount the restored partition:

#chroot /mnt/sysimage

edit the modules list and put every driver the physical machine needs.
you can see which drivers are needed by looking at modprobe.conf from another machine that has the same hw as the one you are V2P-ing.
if you don't have another machine, you can use the same target machine: just before imaging it with the source vmdk, install it from scratch with the Red Hat CD and take note of the resulting modprobe.conf

#vi /etc/modprobe.conf

here is the one i used as a reference (an HP machine)

options bnx2x disable_tpa=1
alias eth0 bnx2
alias eth1 bnx2
alias eth2 tg3
alias eth3 tg3
alias scsi_hostadapter cciss
alias scsi_hostadapter1 lpfc
alias scsi_hostadapter2 usb-storage

then you will have to rebuild the initrd image to include the needed drivers.
note that kernel version number must match the one used on your vm.

/sbin/mkinitrd -v -f /boot/initrd-2.6.18-164.el5.img 2.6.18-164.el5

and reboot.
at boot time you will probably get some errors on X initialization because the graphic card has changed: just answer yes at the reconfigure option and select 800x600 as default resolution.

Update: if you are looking for P2V instead, look at this post.


2011-08-16

CSV Parser

It might take a fool to write a CSV parser in zsh.
Here's one.
It should work in bash too.

#
# read a csv formatted file
# filed1,field2,etc
# fields can be quoted if they contain a comma
#
readcsv()
{
    ifs=$IFS
    IFS=$'\n'

    csvfile=$CSV
    # read full file stripping EOLs
    table=( $(cat $csvfile | tr -d "\r") )

    # line scan
    for line in $table
    do
        campi=( )
        campo=''
        ncampo=1
        quoted=0

        # we scan all the line
        for (( i=1 ; i <= $#line ; i++ ))
        do
            carattere=''
            if [ $line[$i] = \" ]
            then
                # we found some quotes to strip out
                if [ $quoted = 1 ]
                then
                    # closing quotes
                    quoted=0
                else
                    # opening quotes
                    quoted=1
                fi
            else
                if [ $line[$i] = , ]
                then
                    # comma found
                    if [ $quoted = 0 ]
                    then
                        # if it's out of quotes then it's the end of a field
                        campi[ncampo]=$campo

                        let ncampo=$ncampo+1
                        campo=''
                    else
                        carattere=','
                    fi
                else
                    carattere=$line[$i]
                fi
            fi
            # append char to current field
            campo=$campo$carattere
        done
        # at the end of the line all goes into the last field
        campi[ncampo]=$campo

        # now you can access all the field with an array
        #$campi[1]
        #$campi[2]
        #$campi[n]

    done
    IFS=$ifs
}

2011-08-15

Sending commands to your PC from your Android phone.

You'll need: ConnectBot (free on the android market) on the phone, linux on the pc (I've used Ubuntu, but any will do).
First, make sure ssh is enabled PC side. If you PC has a public internet address you'll be able to send commands to it from the other side of the globe, otherwise, as my own setup, you can use the home wi-fi to control it from other rooms.
On the phone, using ConnectBot generate an RSA key pair.
Be sure to check "load at startup".
Then export the public key to your PC (by means of copy/paste via email for example).
On the PC, in your home dir of create a file named .ssh/authorized_keys as follows:

command="sudo shutdown -h now" ssh-rsa AAAAxxxxxc2EAAAADAQABAAAAgQDbTVtZxxxxRz+8EINWYbYvsD/dla30xxxx8LJiQrzgoLmFdTmM7FW+X//dmijMli8xxxx4yWVq/UTeGSDpc/drT4H3Csj7SSLnDksjxxxZqAZwTIncxxxxwu9VL5O6fqnxOWBh2+efnZV3+e2tbiGhqQ== username

You can put any command as you wish as the first token, and you'll need to paste your public key after ssh-rsa.
Create a new ConnectBot connection on the phone. Be sure to select public key authentication using the key you've generated before.
You can even create a shortcut on the phone desktop referring to the connection. This way with a single click on the phone you can issue commands on the pc.