Algorithm for finding hosts which provide a UDP-based service on a large network

Gene Michael Stover

created Friday, 2006 January 13
updated Tuesday, 2006 January 17

Copyright © 2006 Gene Michael Stover. All rights reserved. Permission to copy, store, & view this document unmodified & in its entirety is granted.


Contents

1 What is this?

As of Monday, 2006 January 16, this is almost finished. I'd like to do the SNMP & CIFS examples before I declare it complete.

I describe an algorithm which finds hosts which provide a UDP-based service. The algorithm is fast; it can scan the 10.x.y.z network in about 2.5 hours if you have enough memory & there are few providers of the service. (The algorithm's run time increases as the number of providers increases.)

The algorithm allows you to specify the UDP-based service. It does not know anything of the service.

The algorithm produces short bursts of UDP traffice, but there are significant pauses between the bursts.

The algorithm uses just one thread.

The algorithm does not contain any stealth features.

I have implemented the algorithm in a command line program which is licensed under the terms of the Gnu General Public License. The source code is available for download.

2 Inputs

The inputs to the algorithm are:

3 Output

The algorithm produces a list of the IP addresses of the hosts which responded to the message.

I assume the list of IP addresses is available on standard input (stdin).

4 The Algorithm

The algorithm uses the unix select (2) system call. In the pseudocode here, I'll express that as ``wait for an event on the socket''.

  1. Create a UDP socket s.

  2. Until there are no more addresses to scan

    1. Load IP addresses into a list in memory. Load as many as will fit, but it's okay if they don't all fit.

    2. Until the list is empty

      1. Wait for an event on socket s with timeout T. If we have not sent to all of the list, we are interested in input & output. If we have sent to all of the list, we are only ready for input.

      2. If input is waiting on s, read it with recvfrom. Extract the sender's IP address & print it. Remove that IP address from the list.

      3. If s is ready for output & we have not sent all of the list, send the message to the next IP address on the list.

      4. If there was a timeout (i.e., we were only interested in input, & there was none), clear the list.

  3. Exit

5 How it works

Imagine asking the algorithm to scan a bunch of IP addresses, & it just happens that none of the hosts provide the service that interests you. As the algorithm sends the message to the hosts, none of them reply. So, T seconds after sending the final address, the algorithm times out, clears the list, & announces that it's done & none of the hosts are service providers.

Run time for that instance is T seconds plus the number of seconds to send all the UDP messages.

Now imagine that one of the IP addresses is for a host that does provide the service. Then the algorithm will send many of the IP addresses before receiving a reply from that host. It will print that host's IP address, remove it from the list, & begin the process again. But this time, the process is the case in which none of the hosts are providers. So it's the run time from the ``no providers'' case (a little over T seconds), plus the number of seconds for the one host to reply, which will be one or two seconds in practice, & just under T seconds in the worst case. So the run time will be roughtly $T + 2$ seconds in practice or $2 T$ seconds in the words case.

As you imagine cases in which more & more of the hosts are service providers, it just increases the number of iterations through the list. Each iteration removes one IP address from the list, so that it eventually degeneratese to the case in which none of the hosts are providers.

6 Example: echo

The echo service, which isn't used much (probably for security reasons), is on UDP port 7. You send it a line, & it sends you that same line. The message to make echo reply can be just about anything. Here's the one I used:

$ cat src/echo.sca
hello, world
$

I have echo running on one of my computers, at 10.1.0.5. The generator I used generates all the 10.1.[01].y addresses, preceeded by 128,000 others (which might be routed over the Internet - it's a test). It finds 10.1.0.5.

7 Example: daytime

The daytime is an old one, like echo. It's on port 13. I used the same generator as for the echo test. I used a file of one empty line for the message. (Just about any file would have worked. As with echo, daytime does not examine the payload it receives.)

I'm running daytime on 10.1.0.2.

$ bin/gen |bin/scan 13 src/daytime.sca >001.txt

MAX_OCTETS_IN_LIST is 1048576.
S_lst_len is 262144.
src/scan.c:159: Message contains 1 octets
src/scan.c:239: S_LoadLst: S_count is 131584. Last is 10.1.1.255.
sendto: Permission denied
src/scan.c:344: sendto (10.1.0.0) failed
sendto: Permission denied
src/scan.c:344: sendto (10.1.0.0) failed
src/scan.c:239: S_LoadLst: S_count is 0. Last is 0.0.0.0.$ 
$ cat 001.txt
10.1.0.2
$

8 Example: ONC RPC portmapper

The portmapper service for ONC RPC is on port 111 (both TCP & UDP, but we're only scanning the UDP ports).

The only trick to scanning for the portmapper is that the message is not obvious. I looked through the <rpc/*.h> header files to figure out how to construct a remote call to the portmapper. Then I wrote src/msg-portmapper.c to construct the message & send it to stdout. The message does not have to be perfect; it just needs to motivate the portmapper to reply.

Here are the results of the run. I used the same generator as for the previous two examples.

$ bin/msg-portmapper >src/portmapper.sca
$ time bin/gen |bin/scan 111 src/portmapper.sca >001.txt

MAX_OCTETS_IN_LIST is 1048576.
S_lst_len is 262144.
src/scan.c:159: Message contains 56 octets
src/scan.c:239: S_LoadLst: S_count is 131584. Last is 10.1.1.255.
sendto: Permission denied
src/scan.c:344: sendto (10.1.0.0) failed
sendto: Permission denied
src/scan.c:344: sendto (10.1.0.0) failed
src/scan.c:239: S_LoadLst: S_count is 0. Last is 0.0.0.0.
real	1m4.705s
user	0m1.010s
sys	0m3.390s
$ cat 001.txt
10.1.0.2
10.1.0.5
$

You can see that I'm running the RPC portmapper service at 10.1.0.2 & 10.1.0.5.1

9 Example: SNMP

To construct a message for SNMP, I used a remote printer-controlling application to query a printer while I had a network sniffer running. The sniffer showed the forty or so octets that the application sent to the printer in its first transmission. I typed those numbers into a Lisp expression which dumped them as octets into the snmp.sca file.

SNMP is on port 161.

After that, I ran the scanner on a LAN which contains a lot of SNMP devices, & the scanner located most of them. It worked better when the timeout was 60 seconds.

I won't show the results here because I'm writing this essay at home, & that network with all the SNMP devices is at a customer's site.

10 Example: CIFS

A. Source Code

The program which scans is scan.c.

gen.c generates IP addresses from some hard-coded data. It's for testing or demo purposes.

counter.c is a simple UDP service that can be helpful for tests or demos.

msg-portmapper.c generates the message for scanning for the ONC RPC portmapper service.

B. What about nmap?

Yes, nmap can scan like this.

I didn't use nmap because my client needed code which was unencumbered by the license agreement on nmap.

``Unencumbered by the GPL? But you, yourself, slapped the GPL on your own code?'' Yes, I did, & since I'm the license-holder, I can create a custom, non-GPL license for my client.

``If the user installed nmap separately, wouldn't that allow your client's software to work with nmap without itself being GPLed?'' I believe it would.

``So could you do that? Could you require the user to install nmap himself?'' No.

C. Other File Formats

Bibliography

Gene Michael Stover 2008-04-20