system census and ssh-keyscan

A fairly common problem for admins is to check a list of systems to see if they are alive in some script.

The most common solution is to ping each system and check for a response.
For example (the domain and exact IP address are anonymized)
# ping -c 3 frey
PING frey.domain (IP) 56(84) bytes of data. --- frey.domain ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms

# ping -c 3 freya
PING freya.domain (IP) 56(84) bytes of data.
64 bytes from freya.domain (IP): icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from freya.domain (IP): icmp_seq=2 ttl=64 time=0.045 ms
64 bytes from freya.domain (IP): icmp_seq=3 ttl=64 time=0.045 ms
--- freya.domain ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.044/0.044/0.045/0.007 ms

#

I usually just use ping -c 3 host | tail -2 | head -1 | awk '{print $4}' for look at the next to last line and the number of packets received.

Some problems with ping are:

• Some OS's (depending on version and security settings) require root access to use ping.
• Some hardware (esp server class NIC's) will reply to the ICMP ping requests even if the system itself is unresponsive.
• Many firewalls and some routers block ICMP traffic.
• Some packets are always going to be lost, so you can't just ping each system once.
• It is slow; you have to wait for n pings if it's ok, or n timeouts for down systems.

I've recently been querying the system for its ssh public key which requires the system be up and responsive to get a reply. It is also fairly quick and never gets dropped like ICMP and UDP traffic.
# ssh-keyscan -t rsa freya
# freya SSH-2.0-OpenSSH_5.0
freya ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAr4+j538gsyn9DwGbh4q0V2ACyamef7SPRGtFwlgnO7
qmQbLLo/rt4bOpCJxDE7bsen5uyLlYjU5tRPS16QbryI7j4bi0setMNsbwa4V/Ode4WJhHQt5addPPG/5oYD
qs4B4qMdnGUt7VGgSFuI90tOwHp/FRXEvYa8SW6SbHZc9N2vDZQWHkKqyUV1WNnn1ZfztAjYo6qJtG2hMhvX
BGEsQ3jhHv7XOPM4Ls60wExT+oNTz6ykNQXBA2C5matoDE7jWWo0uc+IPPdALN1zPx9TIRw/PbTQhOM/pEEm
SOgDkhoa2kNNO38fAf6tCOUJtx37FmGlXSWIbPkYt/MDs8nw==

To just get a yes or no, you can use grep -c for the hostname. I usually use something like this to give me a response including the ssh version:
# ssh-keyscan -t rsa frey 2>&1 | grep "^#" | cut -c 2-

# ssh-keyscan -t rsa freya 2>&1 | grep "^#" | cut -c 2-
freya SSH-2.0-OpenSSH_5.0

Of course, the downside of using ssh is that you need a ssh daemon running on your target systems, but the reduction of false positives is nice, and you can use the output to do things like populate your ssh keylist or to make sure that the systems's hostname agrees with the DNS entry for that IP.