Emulating a z/OS Mainframe with Hercules

Note: I started writing this article back in 2015 and hit a few roadblocks that I’ve been able to finally reconcile in the last few months. There are a lot of similar guides out there (which I will reference in my sources), but I found them to be too ambiguous to be completely helpful. While I’ve learned a lot from writing this and troubleshooting the issues from existing¬†guides, I am still far from a mainframe expert. There may be errors here, or things I could have accomplished in a better, more “proper” way. That said, I ultimately have a usable z/OS system up and running, and I hope I can help you have the same ūüôā

Introduction

I recently became aware of the fact that mainframes are still alive and well in the corporate world. But why? Why not just use supercomputers? Mainframes aim to perform a high number of instructions¬†per second, usually measured in the millions. If you hear someone talking about millions of instructions per second (MIPS), they’re probably measuring mainframe throughput. Supercomuters on the other hand aim to have a high number of floating-point operations per second (FLOPS). The difference is that mainframes usually deal with information processing in a short window while supercomuters usually deal with simulations requiring a lot of floating-point arithmetic. A supercomputer might be more suited to weather calculations on Jupiter, but a mainframe is still a better candidate for processing a lot of transactions like you might find in banking or airline booking systems.

Okay, but why not use some sort of content distribution network or cloud computing? For years, mainframes have been touted as the go-to for mission critical processing, with minimal downtime. While cloud computing is catching up in this regard, it can be argued that mainframes are still unrivaled when considering their efficiency and maintainability. One mainframe may be able to process a chunk of data more efficiently than thousands of linked machines in remote locations. Now, consider maintenance. Would you rather update one machine or thousands? And scalability? Many cloud providers supply controls to ramp up power when needed (such as during the holidays) or dial it back during more sleepy periods. Mainframes offer the same sort of control, and can easily scale up or down as needed without someone (or piece of software) needing to roll out or switch off a few hundred more servers.

Mainframes are an interesting piece of technology that still have a purpose, but they rarely discussed these days with the influx of new technologies in processing. It’s easy to try these services out, even for an amateur, but getting your hands on a mainframe is incredibly difficult in comparison. Even if you happened to be employed at a company still utilizing one, you would need training and shadowing sessions before even having the chance to touch a keyboard on a production machine.

Of course, there are ways to explore these systems without needing a physical unit, and that is what I’m going to get into momentarily. It is now possible to get your own taste of Big Iron right from your personal computer.

Requirements

Before we get into installing Hercules, an IBM mainframe emulator, you are going to need to find an image of z/OS. z/OS is the operating system of choice for modern IBM mainframes, but it is a little hard to get your hands on unless you actually have a full-scale system set up somewhere already. There are images of z/OS floating around the Internet that can be found, specifically version 1.10. I will not be sharing where these files can be found, and if you do find them, make sure you adhere to the software license while running z/OS.

Now, we also need a host system to support the Hercules emulator. While Hercules will run in Linux, Windows, and OSX, this guide will use a machine running Linux, specifically Debina 9 (Stretch). I will assume that you already have a system running Debian (or similar) and a non-root, sudo user with access to the z/OS files.

After all of this is set up, we can begin installation!

Configuring Hercules and c3270

First, we need to install some basic utilities and applications. But, one of them (c3270) is not available right away as it is classified as “non-free” software under Debian. You can still install packages like this, you just need to configure your system to do so. We need to edit the sources.list file to allow non-free packages.

Simply add non-free to the end of the stretch and stretch-updates sources by editing /etc/apt/sources.list with your favorite text editor:

$ sudo nano /etc/apt/sources.list

After editing, it should look like this:

$ cat /etc/apt/sources.list

deb http://ftp.us.debian.org/debian/ stretch main non-free
deb-src http://ftp.us.debian.org/debian/ stretch main non-free

deb http://security.debian.org/debian-security stretch/updates main
deb-src http://security.debian.org/debian-security stretch/updates main

# stretch-updates, previously known as 'volatile'
deb http://ftp.us.debian.org/debian/ stretch-updates main non-free
deb-src http://ftp.us.debian.org/debian/ stretch-updates main non-free

Now we are ready to install the packages we need. All of them can be installed by running the following command:

$ sudo apt-get install -y c3270 hercules

As this starts executing, go and put on a pot of coffee. As soon as you turn the machine on and walk back to your computer, this command will probably be through.

The above has installed hercules, our IBM system emulator as well as c3270, a IBM 3270-compatible terminal emulator that we will use to interface with our system.

Now, I’m going to assume you have the z/OS files somewhere on your Linux machine, possibly in a directory path like IBM\ ZOS\ 1.10/Z110SA/images/Z110\ -\ Copy. I will assume that the root IBM folder is in your home directory. We will reorder things by creating a directory MAINFRAME within the home directory to house the z/OS installation:

$ cd ~
$ mv IBM\ ZOS\ 1.10/Z110SA/images/Z110\ -\ Copy ~/MAINFRAME
$ cd ~/MAINFRAME
$ mkdir PRTR

We will now have the following heirarchy:

$ ls ~/MAINFRAME
CONF DASD PRTR

At this point, we need to edit the config file that Hercules reads to boot our mainframe. You can open up the config file in your favorite text editor and follow along with the lines we will modify:

$ nano ~/MAINFRAME/CONF/ADCD_LINUX.CONF

First, we need to edit lines 38/39/40 of the config to map to your PRTR, CONF, and DASD directories in your ~/MAINFRAME directory. We will be using full directory paths, so use your username in place of mine, famicoman.

#********************************************************************
# SYMBOLS DEFINITION *
#********************************************************************

DEFSYM DASD "/home/famicoman/MAINFRAME/DASD"
DEFSYM PROD "/home/famicoman/MAINFRAME/PROD"
DEFSYM PRTR "/home/famicoman/MAINFRAME/PRTR"

Now, we edit networking information on line 115. We will need two unused IP addresses on our local network. We can get our machine’s current IP address using the ip command.

$ ip address show eno1
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether fc:3f:db:09:60:59 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.248/24 brd 192.168.1.255 scope global dynamic eno1
valid_lft 81093sec preferred_lft 81093sec
inet6 fe80::fe3f:dbff:fe09:6059/64 scope link
valid_lft forever preferred_lft forever

Our Debian machine is located at 192.168.1.248. We can pick two additional addresses in the 192.168.1.1 Р192.168.1.254 range. 192.168.1.20 and 192.168.1.21 are currently unused so these will be chosen. 192.168.1.20 will be something of a virtual gateway for the mainframe (think of this sort of like an address for Hercules itself, which we will use as our entry point) while 192.168.1.21 will be an address for the z/OS machine. Keep in mind that 192.168.1.20 will be exposed to your network independently of your host machine, creating a logically separate machine. This means you can access it with its own address, and create separate firewall rules, port forwarding, etc. as though it was physical machine on your LAN.

We will replace the content at line 115 in the config with the following to create a virtual adapter to handle networking with our chosen addresses:

#********************************************************************
# CTCI COMMUNICATION DEVICES *
#********************************************************************
0E20.2 3088 CTCI /dev/net/tun 1500 192.168.1.20 192.168.1.21 255.255.255.255

Lastly, we edit line 31. This line changes the default port for Hercules console connections (made by c3270) from 23 to something of your choosing. I will be using port 2323 as I may be using port 23 otherwise, and it is not a privileged port.

CNSLPORT 2323

Now we can launch Hercules! (Do you smell your coffee percolating yet?)

I prefer to use screen sessions to keep thing organized (If you don’t have screen, install it with sudo apt-get install screen or just use tmux). This is also handy with using a virtual or remote host machine as you can keep the sessions going when not connected to the host. The below will place you in a new screen session where we will launch Hercules:

$ screen -S hercules

And now for the launch, specifying the config we edited earlier:

$ sudo hercules -f ~/MAINFRAME/CONF/ADCD_LINUX.CONF

Hercules will begin to load (and give you a lot of logs). Then you will be presented with the Hercules console.

The Hercules console after launching. Note our tun0 device opening and our custom console port specified.

Now, we want to create a 3270 terminal session with Hercules. So, hold <CTRL> + A + D to detach your screen session, returning you to your original console window on the Debian host. Next, create a new screen session for our 3270 connection:

$ screen -S c3270

Now in our new screen session, we will launch c3270 to connect into Hercules, emulating a 3270 connection to actual hardware:

c3270 localhost 2323

You should be presented with a Hercules splash screen:

The Hercules splash screen.

Detach from your c3270 screen session and reattach to the hercules session. It might be a good idea to open a new terminal window on the host machine to keep multiple screen sessions open at once. I suggest two terminal windows, one with hercules and one with c3270. To reattach your hercules screen session, use the below command after detaching:

$ screen -r hercules

Now that you are presented with the Hercules console again, you should see your connection from the 3270 session in the logs.

HHCTE009I Client 127.0.0.1 connected to 3270 device 0:0700

Booting z/OS

Now we can boot z/OS for the first time! In the Hercules console, type the following and hit <RETURN>:

ipl a80

z/OS will now boot. Your coffee should be done by now, so go grab a cup. I’ll wait.

Depending on the specs of the host machine, this could take a long, long time. The first boot took around 90 minutes for me, and could take even longer. At this point, you will get a lot of logging info in both the c3270 session and the hercules session. A lot of this looks like it could be reporting that something has gone horribly wrong, but don’t worry, it is likely okay. This is probably a good time to go for a walk outside with your coffee. Maybe take a good book and settle under a tree for a bit.

A Potential Boot Issue

I did run into the following message on my c3270 session at some point while attempting boot:

IXC208I THE RESPONSE TO MESSAGE IXC420D IS INCORRECT: IS NOT A VALID
ACTION
IXC420D REPLY I TO INITIALIZE SYSPLEX ADCDPL, OR R TO REINITIALIZE
XCF.
REPLYING I WILL IMPACT OTHER ACTIVE SYSTEMS.

If this happens to you, you can safely type the following in the c3270 session and hit <RETURN>:

R 00,I

This will allow z/OS to continue booting.

This message in the 3270 console halted boot-up. Entering the provided command can resume system startup.

If you are unsure whether or not z/OS is fully booted (It can be hard to tell), the easiest thing to do is open another c3270 connection to localhost (maybe create a new screen¬†session via screen -S terminal). If you get the Hercules splash screen again you can safely close the session (<CTRL> + ], then type “exit”), wait a little longer, and try connecting again. Eventually, your second terminal session should connect and get to the log-on screen for your z/OS installation.

Welcome to the DUZA system!

To log in, we enter”TSO” at the prompt. When prompted for a username, enter “IBMUSER”.

Login starts by asking for a USERID.

Then, enter “SYS1” as the password.

The password gets blanked out as you type it.

From here, press <RETURN>, then the ISPF menu will launch.

You will get some brief messages after logging in. Press the <RETURN> key to go to the ISPF menu.

 

The ISPF menu serves as a gateway to a lot of system functionality.

Now in the ISPF menu, type “3.4” to load the Data Set List Utility.

Replace “IBMUSER” in the “Dsname Level” field with “DUZA” and press <RETURN>.

We will use the Data Set List Utility to locate our network settings.

Scroll down using the <F8> key in the Data Sets list and locate the one called DUZA.TCPPARAMS. With your cursor, click on the ‘D’ in “DUZA.TCPARAMS” and use the left-arrow key to navigate three spaces to the left. Type the letter ‘E’ and hit <RETURN> to see items in this data set.

We need to edit the TCPPARAMS for the DUZA system.

On the next screen, use your cursor to click on the first position on the line to the left of the word “PROFILE”. Type the letter ‘E’ and hit <RETURN> to edit this item.

Finally, we can edit the Profile.

Use <F8> to page down to line 90:

000090 DEVICE LCS1 LCS E20
000091 LINK ETH1 ETHERNET 0 LCS1
000092
000093 HOME
000094 10.0.1.20 ETH1
000095
000096 GATEWAY
000097 10.0.1.100 = ETH1 1500 HOST
000098
000099 DEFAULTNET 10.0.1.100 ETH1 1500 0
...
000109 START LCS1

Modify the lines so they look like the following with out IP addresses outlined earlier (and don’t forget line 109!):

000090 DEVICE CTCA1 CTC e20
000091 LINK CTC1 CTC 1 CTCA1
000092
000093 HOME
000094 192.168.0.210 CTC1
000095
000096 GATEWAY
000097 192.168.0.1 = CTC1 1492 HOST
000098
000099 DEFAULTNET 192.168.0.5 CTC1 1492
...
000109 START CTCA1

To save the updated config, place your cursor to the first underline character to the right of “Command ===>” and type “SAVE” followed by the <RETURN> key. Next, type “END” at the same location, again pressing the <RETURN> key.

Here is what the updated settings look like via the 3270 terminal:

Our updated networking is ready to save. Note the IP addresses we specified earlier when configuring Hercules.

Next, we need to recycle the TCPIP service on the system. Go back to your first c3270 console session (detaching your terminal¬†session) and type¬†“STOP TCPIP” followed by the <RETURN> key in the console.

STOP TCPIP.

Wait a minute or two and then type “START TCPIP” followed by the <RETURN> key. After both commands, you should see a lot of console output regarding the TCPIP service. After starting the service back up, wait a few minutes before proceeding to make sure everything has come back up.

After running START TCPIP.

After restarting the TCP service, we need to detach the session and do a few more things on our host machine.

Back on the Debian host machine we need to enable IPv4 forwarding and proxy arp with the following two commands to get networking sorted out:

$ sudo sh -c "echo '1' > /proc/sys/net/ipv4/conf/all/proxy_arp"
$ sudo sh -c "echo '1' > /proc/sys/net/ipv4/conf/all/forwarding"

Testing Networking

We can now test whether we can remote into our z/OS machine, and if we can get out from the inside. From the console on the host Debian machine, telnet to our mainframe using port 1023:

$ telnet 192.168.1.20 1023

Login with the credentials we used earlier (IBMUSER/SYS1) and try out a traceroute command:

Trying 192.168.1.20...
Connected to 192.168.1.20.
Escape character is '^]'.
EZYTE27I login: IBMUSER
EZYTE28I IBMUSER Password:
IBM
Licensed Material - Property of IBM
5694-A01 Copyright IBM Corp. 1993, 2008
(C) Copyright Mortice Kern Systems, Inc., 1985, 1996.
(C) Copyright Software Development Group, University of Waterloo, 1989.

All Rights Reserved.

U.S. Government Users Restricted Rights -
Use,duplication or disclosure restricted by
GSA ADP Schedule Contract with IBM Corp.

IBM is a registered trademark of the IBM Corp.

IBMUSER:/u/ibmuser: >traceroute 8.8.8.8
CS V1R10: Traceroute to 8.8.8.8 (8.8.8.8)
Enter ESC character plus C or c to interrupt
1 192.168.1.21 (192.168.1.21)  1 ms  1 ms  1 ms
2 192.168.1.1 (192.168.1.1)  70 ms  4 ms  3 ms
3 71.185.57.1 (71.185.57.1)  5 ms  6 ms  4 ms
4 100.41.14.204 (100.41.14.204)  10 ms 100.41.14.206 (100.41.14.206)  8 ms 100.41.14.204 (100.41.14.204)  10 ms
5 * * *
6 * * *
7 140.222.0.187 (140.222.0.187)  10 ms 140.222.2.201 (140.222.2.201)  9 ms 140.222.0.187 (140.222.0.187)  6 ms
8 204.148.79.46 (204.148.79.46)  16 ms  11 ms  11 ms
9 108.170.246.33 (108.170.246.33)  12 ms 108.170.246.1 (108.170.246.1)  12 ms 108.170.240.97 (108.170.240.97)  10 ms
10 108.170.226.95 (108.170.226.95)  10 ms 209.85.254.75 (209.85.254.75)  15 ms 216.239.41.203 (216.239.41.203)  12 ms
11 8.8.8.8 (8.8.8.8)  11 ms  19 ms  10 ms

You can additionally try out some more Unix commands:

IBMUSER:/u/ibmuser: >uptime
07:55PM  up 6 day(s), 03:54,  1 users,  load average: 0.00, 0.00, 0.00
IBMUSER:/u/ibmuser: >uname -a
OS/390 DUZA 20.00 03 7060
IBMUSER:/u/ibmuser: >whoami
OMVSKERN
IBMUSER:/u/ibmuser: >ls
CEEDUMP.20050812.162501.65568  ptest.c                        setup1
SimpleCopy.class               ptest.o                        setup2
SimpleCopy.java                ptestc                         setup3
hfsin                          ptestc.trc.16842781            zfs
hfsout                         setup

Back in your second 3270 connection (which like me you may have named terminal), you can keep entering”EXIT” in the “Command ===>” field until you return back to the ISPF menu we saw earlier.

There are many options from the ISPF menu. Take some time to explore them when you get a chance!

From here, you can enter “6” in the “Option ===>” field to get to the Command menu. From here, you can try out other various commands like ping¬†or netstat by¬†entering them into the “===>” field.

Here is the output of netstat. Notice how previously used commands are cached for you.

Shutting it Down

You always want to make sure to shut down your mainframe in the proper way. Otherwise, you may end up with corrupted data or an unbootable system!

From your first c3270 session, enter in¬†“S SHUTSYS”.

S SHUTSYS

Then after a little while enter in “Z EOD”.

Z EOD

Starting the shutdown process.

After a few minutes the machine will halt. Then, switch over to your Hercules console and enter in “exit” to close out Hercules.

exit

Rebooting the mainframe follows the same start-up process from initial boot, so you can easily come back to things.

Conclusion

That’s it, you now have a functioning mainframe! Albeit, it will be much slower than a real mainframe on real hardware (emulation on my machine usually only clocks between 5-12 MIPS).

Toggle back and forth between the console and graphical view in Hercules with the <ESC> key.

Feel free to explore the system, and start learning how to use z/OS and customize your installation!

Sources

 

Generate A Vanity .onion Address For Your Tor Hidden Service Using Scallion

Ever wonder how Tor sites get those custom vanity .onion addresses such as silkroada7bc3kld.onion? These addresses can be generated by hidden service operators for production use, and are just as secure as the automatically generated (and often more cryptic) addresses.

Hidden service .onion addresses are really just the public part of a key pair. Utilizing asymmetric encryption, a hidden service uses the public key (a 16 character string that functions as the actual address prefix) and a private key (a much longer string that is known only to the hidden service) to verify the identity of the service. Anyone connecting to the public key can only do so if the hidden service has access to the private key. Under normal circumstances, only the service operator has access to that private key, so you could trust that the address has not been hijacked.

Keep in mind, while it takes a long time, it is possible for someone to generate the same keypair as another hidden service. While computationally expensive, entities able to throw enough resources at generating an identical address would be able to do so much more quickly than someone acting alone on a sole machine.

 

Generation with Scallion

Scallion is one tool that can be used for generation. Unlike previous tools for generation addresses, Scallion focuses on GPUs, meaning it works much faster than CPU-targeting utilities in most cases. In my experience, Scallion does not work on ARM devices (Use Shallot or Eschalot instead), but if you have an x86_64 processor and some sort of video graphics (integrated or otherwise), you should be good to go.

Let’s get started generating custom .onion addresses. I will assume that you have access to a Linux machine and are familiar with the terminal. I will be using Debian, but this guide should be easy to modify for most distributions.

First, install some dependencies and then clone Scallion onto your machine:

$ sudo apt install clinfo mono-complete mono-devel nvidia-opencl-common nvidia-opencl-dev nvidia-opencl-icd libssl1.0-dev beignet beignet-dev ocl-icd-opencl-dev ocl-icd-libopencl1
$ git clone https://github.com/lachesis/scallion.git

Now, we will move to the scallion directory, and build the scallion executable:

$ cd scallion
$ xbuild scallion.sln /p:TargetFrameworkVersion="v4.5"

Next, we will get a list of all of the devices that can be used for generating addresses:

$ mono scallion/bin/Debug/scallion.exe -l
Id:0 Name:Intel(R) HD Graphics Skylake Desktop GT2
    PreferredGroupSizeMultiple:16 ComputeUnits:24 ClockFrequency:1000
    MaxConstantBufferSize:134217728 MaxConstantArgs:8 MaxMemAllocSize:3221225472

We can see I have one device with an identifier 0 that I can target. You may have more than one device.

Now we can use Scallion to find an address that starts with a word or phrase of our choice. Let’s start Scallion with 8 threads, and have it use device 0. We will look for addresses that start with “apple”.¬†After a little waiting, you should get some similar output with the .onion address (public key) and the private key:

$ mono scallion/bin/Debug/scallion.exe -t8 -d 0 apple
Cooking up some delicious scallions...
Using kernel optimized from file kernel.cl (Optimized4)
Using work group size 16
Compiling kernel... done.
Testing SHA1 hash...
CPU SHA-1: d3486ae9136e7856bc42212385ea797094475802
GPU SHA-1: d3486ae9136e7856bc42212385ea797094475802
Looks good!
LoopIteration:1  HashCount:16.78MH  Speed:98.7MH/s  Runtime:00:00:00  Predicted:00:00:00  Found new key! Found 1 unique keys.

  2018-01-03T00:24:24.645322Z
  applencoaipu4tqj.onion
  -----BEGIN RSA PRIVATE KEY-----
MIICXgIBAAKBgQCcDTg3+pON2oUclpStVlFVhtcpleNwtmdO3ZVuN2cPe9tyATjH
fye++edUSTwVm6EZZABrK3iSBdGAITXxRpc5dBM+SHPals6DEECRffa+d2QazJq2
xjhU7sfocXMzly+lALtE3T/I8yhFwcDlv/LGsWn6P9Hh2A2otDz3SCeGCQIEXGYD
DwKBgBkW9kDgDFafPvLhA0YIaDei5tBR5gJXt2vqabJfbi8P7RKF3GJ6vlHXu7xS
XikDmN5lJ+dAeFH5mH4mx0TAyfpjHvwrvCcVFPuXnt8ufDHYnRc5B8hYg/bpe0eS
9iZpSFKvq1Io49Wlt04KKAW86Nk0EJRPlkU6ewfOvs5AHI9vAkEAz+N847csHbcx
79RlBhvoT+GUYoSdKvtB+0pyv4mRYEV3SHFATVwXlTksOcPkI1dFVftkoXaEEld2
RTmsVlaLNQJBAMAqnKx+s4LAj2NxzBTpbcpeVY+DauWBoNMUo5Qdqm3SSV4hPsbd
Bf99XvCWS+7tD+jhks4mffOcKQZNK4JHVgUCQQDH6n4Uf2QWhZHvnY0niHE0ydiu
f2KIBc2spzWzcCWiyBqmtAbjhT3/HajJHB3zYdzHPrI0uVFHVqrjBnhRKSVjAkEA
vWUSQ9u4jPBH/z3ahdD6kbvQA90Jxo/JQgrwaHAUrli/SvmOC3xx/kSWLVPSlSk4
p96zeIgMolOS4Tfiff+newJAUaCQumLZori7RCT+2XOXFCoV03TLlujS8L+2sNH1
LPR8Brc3CBv+ZlleYnJCR4J88py8dFGYSYM95qmpCek1SA==
-----END RSA PRIVATE KEY-----

After generating a private key and address, you will want to use them with your Tor hidden service. The private key and address usually sit in files within the /var/lib/tor/hidden_service/ directory and are named hostname and private_key respectively.

 

For a full list of options and flags, we can run the scallion executable with the --help flag:

$ mono scallion/bin/Debug/scallion.exe --help
Usage: scallion [OPTIONS]+ regex [regex]+
Searches for a tor hidden service address that matches one of the provided regexes.

Options:
  -k, --keysize=VALUE        Specifies keysize for the RSA key
  -n, --nonoptimized         Runs non-optimized kernel
  -l, --listdevices          Lists the devices that can be used.
  -h, -?, --help             Displays command line usage help.
      --gpg                  GPG vanitygen mode.
  -d, --device=VALUE         Specifies the opencl device that should be used.
  -g, --groupsize=VALUE      Specifies the number of threads in a workgroup.
  -w, --worksize=VALUE       Specifies the number of hashes preformed at one
                               time.
  -t, --cputhreads=VALUE     Specifies the number of CPU threads to use when
                               creating work. (EXPERIMENTAL - OpenSSL not
                               thread-safe)
  -p, --save-kernel=VALUE    Saves the generated kernel to this path.
  -o, --output=VALUE         Saves the generated key(s) and address(es) to this
                               path.
      --skip-sha-test        Skip the SHA-1 test at startup.
      --quit-after=VALUE     Quit after this many keys have been found.
      --timestamp=VALUE      Use this value as a timestamp for the RSA key.
  -c, --continue             Continue to search for keys rather than exiting
                               when a key is found.
      --command=VALUE        When a match is found specified external program
                               is called with key passed to stdin.
                               Example: "--command 'tee example.txt'" would
                               save the key to example.txt
                               If the command returns with a non-zero exit code,
                                the program will return the same code.
 

Generate A Vanity .onion Address For Your Tor Hidden Service Using Eschalot

Ever wonder how Tor sites get those custom vanity .onion addresses such as silkroada7bc3kld.onion? These addresses can be generated by hidden service operators for production use, and are just as secure as the automatically generated (and often more cryptic) addresses.

Hidden service .onion addresses are really just the public part of a key pair. Utilizing asymmetric encryption, a hidden service uses the public key (a 16 character string that functions as the actual address prefix) and a private key (a much longer string that is known only to the hidden service) to verify the identity of the service. Anyone connecting to the public key can only do so if the hidden service has access to the private key. Under normal circumstances, only the service operator has access to that private key, so you could trust that the address has not been hijacked.

Keep in mind, while it takes a long time, it is possible for someone to generate the same keypair as another hidden service. While computationally expensive, entities able to throw enough resources at generating an identical address would be able to do so much more quickly than someone acting alone on a sole machine.

 

Generation with Eschalot

Eschalot is one tool that can be used for generation. Eschalot is based off of another tool I previously covered called Shallot. While Shallot only allowed for some basic matching with regular expressions, Eschalot gives the user a bit more control and even supports word lists. Eschalot will not be as fast as a tool like Scallion, but it is (in my opinion) more portable as Scallion seems to have issues running on ARM-based SOCs.

Let’s get started generating custom .onion addresses. I will assume that you have access to a Linux machine and are familiar with the terminal.¬†I will be using Debian, but this guide should be easy to modify for most distributions.

First, install OpenSSL if we don’t have it, then clone Eschalot onto your machine:

$ sudo apt-get install openssl
$ git clone https://github.com/ReclaimYourPrivacy/eschalot.git

Now, we will move to the eschalot directory, and build the eschalot executable:

$ cd eschalot
$ make

We can now make sure everything is working using the builtin testing option:

$ make test
./worgen 8-16 top150adjectives.txt 3-16 top400nouns.txt 3-16 top1000.txt 3-16 > wordlist.txt
Will be producing 8-16 character long word combinations.
Reading 3-16 characters words from top150adjectives.txt.
Reading 3-16 characters words from top400nouns.txt.
Reading 3-16 characters words from top1000.txt.
Loading words from top150adjectives.txt.
Loaded 150 words from top150adjectives.txt.
Loading words from top400nouns.txt.
Loaded 400 words from top400nouns.txt.
Loading words from top1000.txt.
Loaded 974 words from top1000.txt.
Working. 100% complete, 31122412 words (approximately 377Mb) produced.
Final count: 31366539 word combinations.
./eschalot -vct4 -f wordlist.txt >> results.txt
Verbose, continuous, no digits, 4 threads, prefixes 8-16 characters long.
Reading words from wordlist.txt, please wait...
Loaded 31366539 words.
Sorting the word hashes and removing duplicates.
Final word count: 31363570.
Thread #1 started.
Thread #2 started.
Thread #3 started.
Thread #4 started.
Running, collecting performance data...
Found a key for kindland (8) - kindlandudsw7nga.onion
Found a key for loudhour (8) - loudhourvype7cyn.onion
Found a key for cutwaxwin (9) - cutwaxwinstsf6mk.onion
Total hashes: 177519717, running time: 10 seconds, hashes per second: 17751971

When done, simply clean up the test results:

$ make cleantest

 

Now is a good time to use Eschalot to find an address that starts with a word or phrase of our choice. Let’s start Eschalot in verbose mode, with 4 threads, and have it continue to look for addresses even after it has found one. We will look for addresses that start with “apple”.¬†After a little waiting, you should get some similar output with the .onion address (public key) and the private key:

$ ./eschalot -t4 -v -c -p apple
Verbose, continuous, no digits, 4 threads, prefixes 5-5 characters long.
Thread #1 started.
Thread #2 started.
Thread #3 started.
Thread #4 started.
Running, collecting performance data...
Found a key for apple (5) - appleiujtls4awea.onion
----------------------------------------------------------------
appleiujtls3awea.onion
-----BEGIN RSA PRIVATE KEY-----
MIICXQIBAAKBgQDQo/ArZ/Em1/1wvTlpVSjV5eTnYelelzJ670kBsO8/Yjvq5GGu
M7LdAUVUBdrF9qO2I24eKNul0sam+jmvov8WvFAk2eSk2p5VefVabxvUvSiOVNwA
cw3XmHlbWPpriOmyFiTOQtzYXHqG4YNPna9VQMR5S+86hy4AEiJoJNEdpQIEAT1u
AwKBgF8uYVx00lZPzAUxAFmrw85H78001d767m+7OLWmj5dDSgC0nbNWHbICwP8y
fczIoQ/DLHGOHK1WtPiUXNw7E/EX3f4DQ3E0zssIRZOBhmy6D3umLUHBILI/6n/F
hzD8U/Zy3W6rgZ35Ib34Mv35Kq/d6DFUD7u6qWv8CrD0gt7zAkEA6ASM6cI/EaT+
iTh+OSt+Pdh6N3Kofe2QGbqb8Y99tsX07nHOzl8EH0kkIxO26RDuWdT5J3wRUCkA
UGyqVrefNwJBAOY0y7qkSTtLE1S/TwhMWc8mEWqewj4FVT+H/sY0hlfZ5VW7xMjf
usKD99eTDLgJ0joqpIOlmmUHAFHKCkdHwAMCQQDVBCQWZJ/qzFqmFvs20erbTzrk
4VNgV6QmOJDidMAxqyjyocUnA1/vASHJcJnunEsdXvpADVKsV3hwO4jSIxqPAkAs
yCr9mXF5c+wWErdlNdYmOTthqIeCxZssr0fRYVadbtfQ9rEHV0UFEdUp50JWAX3s
oy7t+b7kPoTpTteKshyTAkAPzLpDXQPEWKIJL5uqTR1G/iR7FlQ/jaFd44OCNtmz
azXGB3asSbLcmc3z9KnOb5xzeApPW7hIPy/yxYCgEdrE
-----END RSA PRIVATE KEY-----

Additionally, you can use the included worgen utility to generate word lists that can be fed into Eschalot. Below is an example series of commands that will generate 10-character strings by mixing nouns that are 3-10 characters long each, and then run the list through Eschalot. Eschalot comes with several different word lists included what can be used by the worgen utility.

$ ./worgen 10-10 nouns.txt 3-10 nouns.txt 3-10 > wordlist.txt
$ ./eschalot -vct4 -l 10-10 -f wordlist.txt > results.txt

After generating a private key and address, you will want to use them with your Tor hidden service. The private key and address usually sit in files within the /var/lib/tor/hidden_service/ directory and are named hostname and private_key respectively.

 

For a full list of options and flags, we can run the eshalot executable with no arguments:

$ ./eschalot
Version: 1.2.0

usage:
eschalot [-c] [-v] [-t count] ([-n] [-l min-max] -f filename) | (-r regex) | (-p prefix)
-v : verbose mode - print extra information to STDERR
-c : continue searching after the hash is found
-t count : number of threads to spawn default is one)
-l min-max : look for prefixes that are from 'min' to 'max' characters long
-n : Allow digits to be part of the prefix (affects wordlist mode only)
-f filename: name of the text file with a list of prefixes
-p prefix : single prefix to look for (1-16 characters long)
-r regex : search for a POSIX-style regular expression

Examples:
eschalot -cvt4 -l8-12 -f wordlist.txt >> results.txt
eschalot -v -r '^test|^exam'
eschalot -ct5 -p test

base32 alphabet allows letters [a-z] and digits [2-7]
Regex pattern examples:
xxx must contain 'xxx'
^foo must begin with 'foo'
bar$ must end with 'bar'
b[aoeiu]r must have a vowel between 'b' and 'r'
'^ab|^cd' must begin with 'ab' or 'cd'
[a-z]{16} must contain letters only, no digits
^dusk.*dawn$ must begin with 'dusk' and end with 'dawn'
[a-z2-7]{16} any name - will succeed after one iteration

You can also run the worgenexecutable with no arguments for a complete list of options:

$ ./worgen
Version: 1.2.0

usage: worgen min-max filename1 min1-max1 [filename2 min2-max2 [filename3 min3-max3]]
  min-max   : length limits for the output strings
  filename1 : name of the first word list file (required)
  min1-max1 : length limits for the words from the first file
  filename2 : name of the second word list file (optional)
  min2-max2 : length limits for the words from the first file
  filename3 : name of the third word list file (optional)
  min3-max3 : length limits for the words from the first file

  Example: worgen 8-12 wordlist1.txt 5-10 wordlist2.txt 3-5 > results.txt

              Generates word combinations from 8 to 12 characters long
              using 5-10 character long words from 'wordlist1.txt'
              followed by 3-5 character long words from 'wordlist2.txt'.
              Saves the results to 'results.txt'.

 

Generate A Vanity .onion Address For Your Tor Hidden Service Using Shallot

Ever wonder how Tor sites get those custom vanity .onion addresses such as silkroada7bc3kld.onion? These addresses can be generated by hidden service operators for production use, and are just as secure as the automatically generated (and often more cryptic) addresses.

Hidden service .onion addresses are really just the public part of a key pair. Utilizing asymmetric encryption, a hidden service uses the public key (a 16 character string that functions as the actual address prefix) and a private key (a much longer string that is known only to the hidden service) to verify the identity of the service. Anyone connecting to the public key can only do so if the hidden service has access to the private key. Under normal circumstances, only the service operator has access to that private key, so you could trust that the address has not been hijacked.

Keep in mind, while it takes a long time, it is possible for someone to generate the same keypair as another hidden service. While computationally expensive, entities able to throw enough resources at generating an identical address would be able to do so much more quickly than someone acting alone on a sole machine.

 

Generation with Shallot

Shallot is one tool that can be used for generation. Under the name onionhash, Shallot was first created and maintained by an anonymous developer named Bebop. After Bebop disappeared, development continued with the help of a programmer named `Orum who renamed it Shallot before disappearing himself. Eventually, katmagic moved the code to github where it lives today, but without active development. Over the years, other developers have made fixes, but none of them have been moved into the master branch of the application. Shallot will not be as fast as a tool like Scallion, but it is (in my opinion) more portable as Scallion seems to have issues running on ARM-based SOCs.

Let’s get started generating custom .onion addresses. I will assume that you have access to a Linux machine and are familiar with the terminal.

First, clone Shallot onto your machine:

$ git clone https://github.com/katmagic/Shallot.git

Now, we will move to the Shallot directory, and download and apply some patches:

$ cd Shallot
$ wget https://patch-diff.githubusercontent.com/raw/katmagic/Shallot/pull/9.patch
$ git apply 9.patch
$ wget https://patch-diff.githubusercontent.com/raw/katmagic/Shallot/pull/16.patch
$ git apply 16.patch
$ wget https://patch-diff.githubusercontent.com/raw/katmagic/Shallot/pull/25.patch
$ git apply 25.patch

Wait, what are these for?

  • Patch #9 fixes an off-by-one error that caused generation of incorrect keys.
  • Patch #16 adds an optimization for computing powers of 2 using a bitshift.
  • Patch #25 adds use of memcmp to speed up regular expression use.

Next, we will configure and make to build the shallot executable:

$ ./configure && make

We can now test it by generating an address that starts with “apple” utilizing regular expressions:

$ ./shallot ^apple

After a little waiting, you should get some similar output with the .onion address (public key) and the private key:

$ ./shallot ^apple
-----------------------------------------------------------------
Found matching domain after 9231616 tries: applelmehzgcx37v.onion
-----------------------------------------------------------------
-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQC1szzknIej9Cn32XEarL1TFJXGOWpllK8NDSLKsJfwBIW3kyFb
F996LeIX7wKomRuudZ8TfyCQsdL4XT27MvZTX/HTjYc1TErpw6s+0n1WXm/+sbgD
/8X1vpt/m4OLZS+JOgDSFNM9zi1Qy2GOAMGlyBA5nXKxo5h60vOA87RVowIEARq5
/wKBgBGN/v/RFriNmAd572mI8SiMK5NBO6yu33wz1kf15Xqh/K6QE9Tsr/htYNjr
/RBb7JgIDCXFl0Bsjfnqtsp/WPe5eWUJkzePpzrIyrgAgFneHYYPeKbhUuEwB20i
mFBKXgZmX2yK6BtJDMAjorq/E/hoe9ecKzGaTrv04LTquiIvAkEA43+W3lXBgUIE
EO9ckIHrQ94DvhCtwQGg1vQFLovbvgBL1rqDgmmrWrUrqjqAJXZj8Iou+k+Z9xkK
fn6O4WfMsQJBAMx2ymXmNI8mldwGcl18LvnYGeTCEy0pD6j2yM5LsBNj9G0ZKcFV
n9gAkG5VumCW3yNvGVao7s9B0cjw63n5jJMCQF1ETnd7YOZb20e6GPWxJ1jlXAG7
CapYtn42LBPD9JgNNw8RqKz+zOPu61kWFXMOQnlruLh127218FVsfbvilt8CQFi7
dTZ0DwIPQRwB5QaGe2ymXFSj1yMbDMh9Z/7TXcmdSnigfDfGykQN27qYPwB8CcTM
MfZszZukdmgYFYx+H8cCQDrGHNbtSePX53ATAf2nP6Wqzi438d4Aegev5qOaTLk6
ol+H4euHzOO7R/YmWcXRWSZAwFfmboNb5xMfR54SK+k=
-----END RSA PRIVATE KEY-----

The private key and address usually sit in files within the /var/lib/tor/hidden_service/ directory and are named hostname and private_key respectively.

 

For a full list of options and flags, we can run the shallot executable with no arguments:

$ ./shallot
Usage: shallot [-dmopv] [-f ] [-t count] [-x time] [-e limit] pattern
  -d        : Daemonize (requires -f)
  -m        : Monitor mode (incompatible with -f)
  -o        : Optimize RSA key size to improve SHA-1 hashing speed
  -p        : Print 'pattern' help and exit
  -f  : Write output to 
  -t count  : Forces exactly count threads to be spawned
  -x secs   : Sets a limit on the maximum execution time. Has no effect without -m
  -e limit  : Manually define the limit for e

The repository on Github also has a handy chart to estimate how long it will take to generate an address matching a certain number of characters on a 1.5GHz processor:

characters | time to generate (approx.)
-------------------------------------------------------------------    
1          | less than 1 second    
2          | less than 1 second    
3          | less than 1 second   
4          | 2 seconds    
5          | 1 minute    
6          | 30 minutes    
7          | 1 day    
8          | 25 days  
9          | 2.5 years  
10         | 40 years  
11         | 640 years  
12         | 10 millenia  
13         | 160 millenia  
14         | 2.6 million years
 

Configuring a Tor Hidden Service

Tor hidden services allow various types of services (web server, telnet server, chat server, etc) to be operated within the Tor network. This allows both users and service operators to conceal their identities and locations. Just about anything that can be run on the clearnet can be run within the Tor darknet.

Setting up a hidden service on Tor is a simple process and depending on the level of detail, an operator can keep their service completely anonymous. Depending on your use-case, you may or may not choose to anonymize your service at all. For anonymous operation, it is recommended to bind services being offered to localhost and make sure that they do not leak information such as an IP address or hostname in any situation (such as with error messages).

For this guide, we assume a Debian Stretch (or similar) Linux system with a non-root, sudo user. It is also assumed that the target machine has been set up with some standard security practices such as disallowing root logins over SSH, and basic firewall rules. This Tor hidden service will be masked on the darknet, but if the hosting server is deanonymized, a malicious party could uncover the machine’s actual clearnet IP address and attempt to penetrate it or otherwise disrupt service. Depending on the software running the services you are hiding, you may wish to install into a virtual machine to limit damage to the system by code vulnerabilities.

Installing Tor

Before configuring a relay, the Tor package must be set up on the system. While Debian does have a Tor package in the standard repositories, we will want to add the official Tor repositories and install from there to get the latest software and be able to verify its authenticity.

First, we will edit the sources list so that Debian will know about the official Tor repositories.

$ sudo nano /etc/apt/sources.list

At the bottom of the file, paste the following two lines and save/exit.

deb http://deb.torproject.org/torproject.org stretch main
deb-src http://deb.torproject.org/torproject.org stretch main

Now back in the console, we will add the Tor Project’s GPG key used to sign the Tor packages. This will allow verification that the software we are installing has not been tampered with.

$ gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89
$ gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -

Lastly, run an update and install Tor from the repositories we just added.

$ apt-get update
$ apt-get install tor deb.torproject.org-keyring

 

Configuring the Hidden Service

We will be editing the torrc file, so let’s bring it up in our text editor:

$ sudo nano /etc/tor/torrc

Going line by line in this file is tedious, so to minimize confusion, we will ultimately rewrite the whole file. We will implement logging into a file located at /var/log/tor/notices.log and assume the local machine has a web server running on port 80. Paste the following over the existing contents in your torrc file:

Log notice file /var/log/tor/notices.log

############### This section is just for location-hidden services ###

## Once you have configured a hidden service, you can look at the
## contents of the file ".../hidden_service/hostname" for the address
## to tell people.
##
## HiddenServicePort x y:z says to redirect requests on port x to the
## address y:z.

HiddenServiceDir /var/lib/tor/hs_name_of_my_service/
HiddenServicePort 80 127.0.0.1:80

#HiddenServiceDir /var/lib/tor/other_hidden_service/
#HiddenServicePort 80 127.0.0.1:80
#HiddenServicePort 22 127.0.0.1:22

After saving the file, make and permission a log file, then we are ready to restart Tor:

$ sudo touch /var/log/tor/notices.log
$ chown debian-tor:debian-tor /var/log/tor/notices.log
$ sudo service tor restart

If the restart was successful, the Tor hidden service is active. If not, be sure to check the log file for hints as to the failure:

$ sudo nano /var/log/tor/notices.log

Now that the hidden service is working, Tor has created the hidden service directory we defined in the torrc, /var/lib/tor/hs_name_of_my_service/. There are two files of importance within this directory.

There is a hostname file at /var/lib/tor/hs_name_of_my_service/hostname that contains the hidden service’s public key. This public key acts as a .onion address which users on the Tor network can use to access your service. Make a note of this address after reading it from the file with cat:

$ sudo cat /var/lib/tor/hs_name_of_my_service/hostname
nb2tidpl4j4jnoxr.onion

There is also a private_key file that contains the hidden service’s private key. This private key pairs with the service’s public key. It should not be known or read by anyone or anything except Tor, otherwise someone else will be able to impersonate the hidden service. If you need to move your Tor hidden service for any reason, make sure to backup the hostname and private_key files before restoring them on a new machine.

After restarting the hidden service, it may not be available right away. It can take a few minutes before the .onion address resolves on a client machine.

 

Example – Configure A Web Server with Nginx

Let’s use this hidden service to host a website with Nginx.

First, we will install Nginx and create a directory for our HTML files

$ sudo apt-get install nginx
$ sudo mkdir -p /var/www/hidden_service/

Now, we will create an HTML file to serve, so we need to bring one up in our editor:

$ sudo nano /var/www/hidden_service/index.html

Paste the following basic HTML and save it:

<html><head><title>Hidden Service</title></head><body><h1>It works!</h1></body></html>

Next, we will set the owner of the files we created to www-data for the web server and change the permissions on the /var/www directory.

$ sudo chown -R www-data:www-data /var/www/hidden_service/
$ sudo chmod -R 755 /var/www

We want to make some configuration changes for anonymity. First, let’s edit the default server block:

$ sudo nano /etc/nginx/sites-available/default

Find the block that starts with server { and you should see a line below that reads #listen 80;. Replace this line with to explicitly listen on localhost:

listen localhost:80 default_server;

Now find the line in the block for server_name  set the server name explicitly:

server_name _;

Next we need to edit the Nginx configuration file:

$ sudo nano /etc/nginx/nginx.conf

Find the block that starts with http { and set the following options:

server_name_in_redirect off;
server_tokens off;
port_in_redirect off;

The first option will make sure the server name isn’t used in any redirects. The second option removes server information in error pages and headers. The third option will make sure the port number Nginx listens on will not be included when generating a redirect.

Now we need to create a server block so Nginx knows which directory to serve content from when our hidden service is accessed. Using our text editor, we will create a new server block file:

$ sudo nano /etc/nginx/sites-available/hidden_service

In the empty file, paste the following configuration block. Make sure that the server_name field contains your onion address which you read from the hostname file earlier and not my address, nb2ticpl4j4hnoxq.onion.

server {
listen   127.0.0.1:80;
server_name nb2tidpl4j4jnoxr.onion;

error_log   /var/log/nginx/hidden_service.error.log;
access_log  off;

location / {
        root /var/www/hidden_service/;
        index index.html;
    }
}

After saving the file, we need to symlink it to the sites-enabled directory and then restart Nginx:

$ sudo ln -s /etc/nginx/sites-available/hidden_service /etc/nginx/sites-enabled/hidden_service
$ sudo service nginx restart

To test the hidden service, download and install the Tor Browser on any machine and load up your .onion address.

 

Example – Configure A Web Server with Apache

Let’s use this hidden service to host a website with Apache. Note: Many criticize Apache for leaking server information by default. Apache takes more effort to secure.

First, we will install Apache and create a directory for our HTML files

$ sudo apt-get install apache2
$ sudo mkdir -p /var/www/hidden_service/

Now, we will create an HTML file to serve, so we need to bring one up in our editor:

$ sudo nano /var/www/hidden_service/index.html

Paste the following basic HTML and save it:

<html><head><title>Hidden Service</title></head><body><h1>It works!</h1></body></html>

Next, we will set the owner of the files we created to www-data for the web server and change the permissions on the /var/www directory.

$ sudo chown -R www-data:www-data /var/www/hidden_service/
$ sudo chmod -R 755 /var/www

Now, we need to make a few changes to the Apache configuration. Let’s start by setting Apache up to only listen to port 80 on 127.0.1.1:

$ sudo nano /etc/apache2/ports.conf

Change the line Listen 80 to Listen 127.0.0.1:80 and save the file.

Now we will access the security configuration file:

$ sudo nano /etc/apache2/conf-enabled/security.conf

Change the line for ServerSignature to ServerSignature Off and the line for ServerTokens to ServerTokens Prod to restrict information the httpd reports about the server.

Then, we will make an edit to the main Apache configuration file to override the server name Apache uses:

$ sudo nano /etc/apache2/apache2.conf

At the very bottom of the file, paste the following. Make sure that the ServerName field contains your onion address which you read from the hostname file earlier and not my address, nb2tidpl4j4jnoxr.onion.

ServerName nb2tidpl4j4jnoxr.onion

Next, we will disable Apache’s mod_status module to turn off status information:

$ sudo a2dismod status

Now we need to create a virtual host so Nginx knows which directory to serve content from when our hidden service is accessed. Using our text editor, we will create a new server block file:

$ sudo nano /etc/apache2/sites-available/hidden_service

In the empty file, paste the following configuration block. Make sure that the server_name field contains your onion address which you read from the hostname file earlier and not my address, nb2tidpl4j4jnoxr.onion.

<VirtualHost *:80>

 ServerName  nb2ticpl4j4hnoxq.onion

 DirectoryIndex index.html
 DocumentRoot /var/www/hidden_service/

  CustomLog /dev/null common

</VirtualHost>

After saving the file, we need to symlink it to the sites-enabled directory and then restart Nginx:

$ sudo ln -s /etc/apache2/sites-available/hidden_service /etc/apache2/sites-enabled/hidden_service
$ sudo service apache2 restart

To test the hidden service, download and install the Tor Browser on any machine and load up your .onion address.

 

Conclusion

Your hidden service should be up and running, ready to server Tor users. Now that your relay is functioning, it may be a good idea to back up your hostname and private_key files mentioned earlier in the /var/lib/tor/hs_name_of_my_service/ directory.

I would strongly recommend taking a look at riseup.net’s Tor Hidden Services Best Practices guide to learn more about proper setup of your hidden service.

Additionally, subscribe to the tor-onions mailing list for operator news and support!

Sources

 

Configuring and Monitoring a Tor Middle Relay

The Tor network relies upon individuals and organizations to donate relays for user traffic. The more relays within network, the stronger and faster the network is. Below, we will create a middle relay which receives traffic and sends it off to another relay. Middle relays will never serve as exit points for traffic back out to the clear Internet (a job for an exit relay). Because of this, many see running a middle relay as a safer way of contributing to the Tor network as opposed to running an exit relay which could find an operator at fault if illegal activity or content exits his node.

For this guide, we assume a Debian Stretch (or similar) Linux system with a non-root, sudo user. It is also assumed that the target machine has been set up with some standard security practices such as disallowing root logins over SSH, and basic firewall rules. This Tor relay will be public, and should be secured like any public-facing server.

 

Installing Tor

Before configuring a relay, the Tor package must be set up on the system. While Debian does have a Tor package in the standard repositories, we will want to add the official Tor repositories and install from there to get the latest software and be able to verify its authenticity.

First, we will edit the sources list so that Debian will know about the official Tor repositories.

$ sudo nano /etc/apt/sources.list

At the bottom of the file, paste the following two lines and save/exit.

deb http://deb.torproject.org/torproject.org stretch main
deb-src http://deb.torproject.org/torproject.org stretch main

Now back in the console, we will add the Tor Project’s GPG key used to sign the Tor packages. This will allow verification that the software we are installing has not been tampered with.

$ gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89
$ gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -

Lastly, run an update and install Tor from the repositories we just added.

$ apt-get update
$ apt-get install tor deb.torproject.org-keyring

 

Keeping Time

It is important that a Tor relay keeps accurate time, so we will change the timezone and set up the ntp client.

First, list timezones to and find which one corresponds to the location of the machine:

$ timedatectl list-timezones

Next, we set the timezone to the one for the machine’s location. Amsterdam is used below as an example.

$ sudo timedatectl set-timezone Europe/Amsterdam

Finally, install ntp:

$sudo apt-get install ntp

You can check your changes using the timedatectl command with no options:

$ timedatectl
      Local time: Sat 2017-12-30 21:49:25 CET
  Universal time: Sat 2017-12-30 20:49:25 UTC
        RTC time: Sat 2017-12-30 20:49:25
       Time zone: Europe/Amsterdam (CET, +0100)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: no
 Last DST change: DST ended at
                  Sun 2017-10-29 02:59:59 CEST
                  Sun 2017-10-29 02:00:00 CET
 Next DST change: DST begins (the clock jumps one hour forward) at
                  Sun 2018-03-25 01:59:59 CET
                  Sun 2018-03-25 03:00:00 CEST

 

Configuring the Relay

By default, all new relays are set up to be exit nodes. Since we want to create a middle relay, there is some configuration that needs to be done.

We will be editing the torrc file, so let’s bring it up in our text editor:

$ sudo nano /etc/tor/torrc

Going line by line in this file is tedious, and to minimize confusion, I will outline some configuration outlines and paste a sample torrc file below that you can use with minimal changes.

  • We don’t need a SOCKS proxy, so uncomment the line SOCKSPolicy reject *
  • We want to keep a separate log file, so uncomment the line Log notice file /var/log/tor/notices.log
  • We will be running as a daemon, so uncomment the line RunAsDaemon 1
  • We will be running monitoring via ARM, so uncomment the line ControlPort 9051
  • Relays need an ORPort for incoming connections, so uncomment the line ORPort 9001
  • It is recommended that a relay has an FQDN or at least a subdomain of one. If not, the machine’s IP address can be used. We will unncomment the line Address noname.example.com and use our address in place of noname.example.com
  • The relay should also have a nickname, so uncomment the line Nickname ididnteditheconfig and use our own nickname in place of ididnteditheconfig
  • Contact information should also be provided, so uncomment the line #ContactInfo Random Person and use our own info in place of Random Person
  • We will be running a directory port, so uncomment the line DirPort 9030
  • The most important option, we don’t want to allow any exits, so uncomment the line ExitPolicy reject *:*
  • Optionally, we may want to limit the bandwidth that Tor uses. To do so, uncomment the lines RelayBandwidthRate 100 KBytes and RelayBandwidthBurst 200 KBytes. These values are defined for one way transport, so note that the actual bandwidth rate above could be 200KB/s total (100KB/s for each input and output). Burst defines a maximum rate, so a burst of 200 KBytes means bandwidth could reach 400KB/s total (combined input and output). Many will likely want their relay to be considered Fast by the network, meaning that the relay’s bandwidth is in the top 7/8ths of all relays. At the time of writing, a rate of¬†500 KBytes/s seems to be on the low end of achieving this according to a Tor team member.
  • Optionally, we may want to limit the total traffic Tor uses over a period. To do so, uncomment the lines AccountingMax 40 GBytes and AccountingStart month 3 15:00. The AccountingMax value is defined for one way transport, so note that setting 40 GBytes could use 80 GBytes total (40 GBytes for each input and output). If the machine is on a provider that limits monthly bandwidth, it is a good idea to adjust this value to align with the provider’s data cap and adjust the AccountingStart values to reset to reset when the data cap does. NOTE: If you do set accounting, the relay will not advertise a directory port and you will not get directory connections. Relays with directories are expected to have a lot of bandwidth and limiting it will result in other nodes not making directory connections.

Now, here is a full sample torrc file for the tor middle relay:

## Configuration file for a typical Tor user

## Tor opens a SOCKS proxy on port 9050 by default -- even if you don't
## configure one below. Set "SOCKSPort 0" if you plan to run Tor only
## as a relay, and not make any local application connections yourself.
#SOCKSPort 9050 # Default: Bind to localhost:9050 for local connections.
#SOCKSPort 192.168.0.1:9100 # Bind to this address:port too.

## Entry policies to allow/deny SOCKS requests based on IP address.
## First entry that matches wins. If no SOCKSPolicy is set, we accept
## all (and only) requests that reach a SOCKSPort. Untrusted users who
## can access your SOCKSPort may be able to learn about the connections
## you make.
#SOCKSPolicy accept 192.168.0.0/16
#SOCKSPolicy accept6 FC00::/7
SOCKSPolicy reject *

## Logs go to stdout at level "notice" unless redirected by something
## else, like one of the below lines. You can have as many Log lines as
## you want.
##
## We advise using "notice" in most cases, since anything more verbose
## may provide sensitive information to an attacker who obtains the logs.
##
## Send all messages of level 'notice' or higher to /var/log/tor/notices.log
Log notice file /var/log/tor/notices.log
## Send every possible message to /var/log/tor/debug.log
#Log debug file /var/log/tor/debug.log
## Use the system log instead of Tor's logfiles
#Log notice syslog
## To send all messages to stderr:
#Log debug stderr

## Uncomment this to start the process in the background... or use
## --runasdaemon 1 on the command line. This is ignored on Windows;
## see the FAQ entry if you want Tor to run as an NT service.
RunAsDaemon 1

## The directory for keeping all the keys/etc. By default, we store
## things in $HOME/.tor on Unix, and in Application Data\tor on Windows.
#DataDirectory /var/lib/tor

## The port on which Tor will listen for local connections from Tor
## controller applications, as documented in control-spec.txt.
ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
#HashedControlPassword 16:872860B76453A77D60CA2BB8C1A7042072093276A3D701AD684053EC4C
#CookieAuthentication 1

############### This section is just for location-hidden services ###

## Once you have configured a hidden service, you can look at the
## contents of the file ".../hidden_service/hostname" for the address
## to tell people.
##
## HiddenServicePort x y:z says to redirect requests on port x to the
## address y:z.

#HiddenServiceDir /var/lib/tor/hidden_service/
#HiddenServicePort 80 127.0.0.1:80

#HiddenServiceDir /var/lib/tor/other_hidden_service/
#HiddenServicePort 80 127.0.0.1:80
#HiddenServicePort 22 127.0.0.1:22

################ This section is just for relays #####################
#
## See https://www.torproject.org/docs/tor-doc-relay for details.

## Required: what port to advertise for incoming Tor connections.
ORPort 9001
## If you want to listen on a port other than the one advertised in
## ORPort (e.g. to advertise 443 but bind to 9090), you can do it as
## follows.  You'll need to do ipchains or other port forwarding
## yourself to make this work.
#ORPort 443 NoListen
#ORPort 127.0.0.1:9090 NoAdvertise

## The IP address or full DNS name for incoming connections to your
## relay. Leave commented out and Tor will guess.
Address tor.peer3.famicoman.com

## If you have multiple network interfaces, you can specify one for
## outgoing traffic to use.
# OutboundBindAddress 10.0.0.5

## A handle for your relay, so people don't have to refer to it by key.
Nickname peer3famicoman

## Define these to limit how much relayed traffic you will allow. Your
## own traffic is still unthrottled. Note that RelayBandwidthRate must
## be at least 20 kilobytes per second.
## Note that units for these config options are bytes (per second), not
## bits (per second), and that prefixes are binary prefixes, i.e. 2^10,
## 2^20, etc.
RelayBandwidthRate 2048 KBytes  # Throttle traffic to 2048KB/s (16384Kbps)
RelayBandwidthBurst 3072 KBytes # But allow bursts up to 3072KB/s (24576Kbps)

## Use these to restrict the maximum traffic per day, week, or month.
## Note that this threshold applies separately to sent and received bytes,
## not to their sum: setting "40 GB" may allow up to 80 GB total before
## hibernating.
##
## Set a maximum of 40 gigabytes each way per period.
#AccountingMax 400 GBytes
## Each period starts daily at midnight (AccountingMax is per day)
#AccountingStart day 00:00
## Each period starts on the 3rd of the month at 15:00 (AccountingMax
## is per month)
AccountingStart month 24 15:00

## Administrative contact information for this relay or bridge. This line
## can be used to contact you if your relay or bridge is misconfigured or
## something else goes wrong. Note that we archive and publish all
## descriptors containing these lines and that Google indexes them, so
## spammers might also collect them. You may want to obscure the fact that
## it's an email address and/or generate a new address for this purpose.
#ContactInfo Random Person 
## You might also include your PGP or GPG fingerprint if you have one:
#ContactInfo 0xFFFFFFFF Random Person 
ContactInfo famicoman[at]gmail[dot]com - 1DVLNHpcoAso6rvisCnVQbCFN8dRir1GVQ

## Uncomment this to mirror directory information for others. Please do
## if you have enough bandwidth.
DirPort 9030 # what port to advertise for directory connections
## If you want to listen on a port other than the one advertised in
## DirPort (e.g. to advertise 80 but bind to 9091), you can do it as
## follows.  below too. You'll need to do ipchains or other port
## forwarding yourself to make this work.
#DirPort 80 NoListen
#DirPort 127.0.0.1:9091 NoAdvertise
## Uncomment to return an arbitrary blob of html on your DirPort. Now you
## can explain what Tor is if anybody wonders why your IP address is
## contacting them. See contrib/tor-exit-notice.html in Tor's source
## distribution for a sample.
#DirPortFrontPage /etc/tor/tor-exit-notice.html

## Uncomment this if you run more than one Tor relay, and add the identity
## key fingerprint of each Tor relay you control, even if they're on
## different networks. You declare it here so Tor clients can avoid
## using more than one of your relays in a single circuit. See
## https://www.torproject.org/docs/faq#MultipleRelays
## However, you should never include a bridge's fingerprint here, as it would
## break its concealability and potentially reveal its IP/TCP address.
#MyFamily $keyid,$keyid,...

## A comma-separated list of exit policies. They're considered first
## to last, and the first match wins.
##
## If you want to allow the same ports on IPv4 and IPv6, write your rules
## using accept/reject *. If you want to allow different ports on IPv4 and
## IPv6, write your IPv6 rules using accept6/reject6 *6, and your IPv4 rules
## using accept/reject *4.
##
## If you want to _replace_ the default exit policy, end this with either a
## reject *:* or an accept *:*. Otherwise, you're _augmenting_ (prepending to)
## the default exit policy. Leave commented to just use the default, which is
## described in the man page or at
## https://www.torproject.org/documentation.html
##
## Look at https://www.torproject.org/faq-abuse.html#TypicalAbuses
## for issues you might encounter if you use the default exit policy.
##
## If certain IPs and ports are blocked externally, e.g. by your firewall,
## you should update your exit policy to reflect this -- otherwise Tor
## users will be told that those destinations are down.
##
## For security, by default Tor rejects connections to private (local)
## networks, including to the configured primary public IPv4 and IPv6 addresses,
## and any public IPv4 and IPv6 addresses on any interface on the relay.
## See the man page entry for ExitPolicyRejectPrivate if you want to allow
## "exit enclaving".
##
#ExitPolicy accept *:6660-6667,reject *:* # allow irc ports on IPv4 and IPv6 but no more
#ExitPolicy accept *:119 # accept nntp ports on IPv4 and IPv6 as well as default exit policy
#ExitPolicy accept *4:119 # accept nntp ports on IPv4 only as well as default exit policy
#ExitPolicy accept6 *6:119 # accept nntp ports on IPv6 only as well as default exit policy
ExitPolicy reject *:* # no exits allowed

## Bridge relays (or "bridges") are Tor relays that aren't listed in the
## main directory. Since there is no complete public list of them, even an
## ISP that filters connections to all the known Tor relays probably
## won't be able to block all the bridges. Also, websites won't treat you
## differently because they won't know you're running Tor. If you can
## be a real relay, please do; but if not, be a bridge!
#BridgeRelay 1
## By default, Tor will advertise your bridge to users through various
## mechanisms like https://bridges.torproject.org/. If you want to run
## a private bridge, for example because you'll give out your bridge
## address manually to your friends, uncomment this line:
#PublishServerDescriptor 0

After saving the file, we are ready to restart Tor:

$ sudo service tor restart

Now, we need to make sure everything worked properly and that the relay is functioning as expected. To do so, we will check the logs:

$ sudo nano /var/log/tor/log

If everything worked as expected, the following lines should appear near the bottom of the log file:

[notice] Self-testing indicates your ORPort is reachable from the outside. Excellent.
[notice] Tor has successfully opened a circuit. Looks like client functionality is working.
[notice] Self-testing indicates your DirPort is reachable from the outside. Excellent. Publishing server descriptor.

If you see anything different, make sure your torrc file is configured properly and that your firewall is set to allow connections to the ports you set for the ORPort and DirPort (by default, 9001 and 9030 respectively).

 

Monitoring Your Relay with Nyx

To monitor Tor relays, many people use a popular tool called Nyx which provides graphical information about activity and status of the node. Nyx utilizes the ControlPort we set earlier to connect into our relay. This port shot not need to be accepted by a firewall if Nyx will be running on the same machine and should be password-protected otherwise.

First, Nyx needs to be installed:

$ sudo apt-get install python-setuptools
$ sudo easy_install pip
$ sudo pip install nyx

Then,Nyx can be run:

$ nyx

The result is a nice representation of the relay’s traffic, utilization, flags, and general information.

 

Monitoring Your Relay with ARM (Deprecated)

While ARM is no longer maintained, it does still function.

To monitor Tor relays, many people use a popular tool called ARM which provides graphical information about activity and status of the node. ARM utilizes the ControlPort we set earlier to connect into our relay. This port shot not need to be accepted by a firewall if ARM will be running on the same machine and should be password-protected otherwise.

First, ARM needs to be installed:

sudo apt-get install tor-arm

Then, ARM can be run:

arm

The result is a nice representation of the relay’s traffic, utilization, flags, and general information.

 

Conclusion

While my relay picked up traffic quickly, it took a long time to be able to fully utilize the bandwidth rates that I gave it. A new Tor middle relay goes through many stages before it can be deemed stable and reliable by the network. I highly recommend reading The lifecycle of a new relay to understand the whole process and know why you may not see traffic right away.

You may notice in my screenshots of Nyx and ARM above, my relay has procured several flags. If you are trying to obtain certain flags for your relay (which sort of act like markers of the relay’s capabilities), I recommend reading this StackExchange post on the subject.

If you want to see some statistics for your relay or share them with others, consider checking out the Atlas and Globe projects. This provides information on a relay by fingerprint of that relay, though you can perform searches with the relay’s nickname. Check out my relay on Atlas and my relay on Globe for examples.

Now that your relay is functioning, you may wish to backup your torrc, backup your relay’s private key (/var/lib/tor/keys/secret_id_key), read and implement operational security practices, and join the tor-relays mailing list.

 

Sources

 

Bypass Your ISP’s DNS & Run A Private OpenNIC Server (2600 Article)

Now that the article has been printed in¬†2600 magazine, Volume 34, Issue 3 (2017-10-02), I’m able to republish it on the web. The article below is my submission to 2600 with some slight formatting changes and minor edits.

Bypass Your ISP’s DNS & Run A Private OpenNIC Server
By Mike Dank
Famicoman@gmail.com

Introduction

With recent U.S. legislation regarding Internet privacy, we see another example of control moving away from consumers and towards service providers. Following the news of this change, many have taken a renewed interest in methods that can take back some of the control and privacy that ISPs and other organizations have slowly been chipping away.

One such service that consumers can liberate (and run) for themselves is DNS. The Domain Name System is responsible for retrieving IP addresses (like 123.45.67.89) from domain names (like 2600.com). For a simplified explanation, when you go to visit a website your machine hasn’t seen before, your machine will query a caching server that is usually owned by your ISP or a company like Google or OpenDNS. This server will return the proper IP address, if they have it cached, or query its way along a chain of DNS servers to the authoritative one controlling that domain. Once found, the IP address for the domain entered will trickle back to you and complete the initial request, allowing your machine to resolve it.

Companies that control these services have a direct look into the sites you are trying to visit. You can bet that more than just a few of them are logging queries and using them for marketing purposes or creating profiles based on who is sitting behind the keyboard at the address of origin. However, there are alternative DNS providers out there who can offer more privacy than others are willing to supply.

One such project, OpenNIC, has been operating a network of DNS servers for many years. Unlike traditional DNS providers, OpenNIC provides an alternate root to the ICANN system (which resolves traditional TLDs, top level domains like .com, .net, etc.) while maintaining backwards compatibility with them. Using OpenNIC, you can still resolve all of the same sites, but also get access to those run by OpenNIC operators, with TLDs such as .geek, .pirate, and .bbs. OpenNIC is made up of hobbyists, engineers, and tinkerers who not only want to explore the ins and outs of DNS, but also offer enhanced privacy and free domain registration for TLDs within their root! You may see OpenNIC as just-another-organization to query, but many operators are privacy-oriented, running their own servers devoid of logging and/or in countries that don’t poke around in your network traffic.

Aside from using an official OpenNIC DNS server to query your home traffic against directly, you can also set one up yourself. Using a modest VPS (512MB of RAM, 4GB of disk) hosted somewhere outside of the US (or the 14-eyes jurisdiction, if you prefer), you can subvert organizations who may be nefariously gathering information from your queries. While your server will still ultimately connect upstream to an OpenNIC server, any clients at home or on the go never will ‚ÄĒ they will only directly query your new DNS server directly.

Installation & Configuration

Setting up a DNS server is relatively easy to do with just a basic understanding of the shell. I’m running a Debian system, so some of the configuration may be different depending on the distribution you are running. Additionally, the steps below are for configuring a BIND server. There are many different DNS server packages out there to choose from, though BIND is arguably the most widespread on GNU/Linux hosts.

After logging into our server we will first want to switch to the root account to configure BIND.

$ su -

Next, we will install bind9 and DNS utilities using the package manager. This will automatically configure a (non-publicly accessible) DNS server for us to work with and various DNS tools that will aid in setting up the server (specifically, dig).

$ apt-get install bind9 dnsutils -y

Now, we will pull down the OpenNIC root hints file for BIND to use. The root hints file simply contains information about OpenNIC’s root DNS servers that control the alternative TLDs OpenNIC has to offer (as well as provide backwards compatibility to ICANN domains). On Debian, we save this information to ‘/etc/bind/db.root’ for BIND to access.

$ dig . NS @75.127.96.89 > /etc/bind/db.root

While the root hints information does not change often, new TLDs can be added to OpenNIC periodically. We will set up a cron job that updates this file once a month (you can specify this to be more frequent is you wish) at 12:00AM on the first of the month. Let’s edit the crontab to add this recurring job.

$ crontab -e

At the bottom of the file, paste the following and save, activating our job.

0 0 1 * * /usr/bin/dig . NS @75.127.96.89 > /etc/bind/db.root

Next, we will want to make some changes to the BIND configuration files. Specifically, we will allow recursive queries (so our BIND installation can query the OpenNIC root servers), enable DNSSEC validation (to verify integrity of DNS data on query to OpenNIC servers), and whitelist our client’s IP address. Edit ‘/etc/bind/named.conf.options’ and replace the contents with the following options block, making any edits as needed to specify a client‚Äôs IP address.

options {        
    directory "/var/cache/bind";

    //Allow localhost and a client IP of 1.2.3.4        
    allow-query { localhost; 1.2.3.4; };        
    recursion yes;

    dnssec-enable yes;        
    dnssec-validation yes;        
    dnssec-lookaside auto;

    auth-nxdomain no;    # conform to RFC1035        
    listen-on-v6 { any; };  //Only use if your server has an ipv6 iface! 
};

Now, we will also change the logging configuration so that no logs are kept for any queries to our server. This is beneficial in that we know our own queries will never be logged on our server (as well as queries from anyone else we might authorize to use our server at a later date) for any reason. To make this change, edit ‘/etc/bind/named.conf’ and add the following logging block to the bottom of the file.

logging {
    category default { null; };
};

Finally, restart BIND so it can use our new configuration.

$ /etc/init.d/bind9 restart

Now, make sure that our server is using itself for DNS by checking the ‘/etc/resolv.conf’ file. If it doesn’t exist already, place the following line above any other lines starting with “nameserver”.

nameserver 127.0.0.1

Testing resolution of both OpenNIC and ICANN TLDs can be done with a few simple ping commands.

$ ping -c4 2600.com 
$ ping -c4 opennic.glue

Conclusion & Next Steps

Now that the server is in place, you are free to configure your client machine(s), home router, etc. to make use of the new DNS server. Provided you have port 53 open for both UDP and TCP on the server’s firewall, you should be able to add a similar ‘nameserver’ line to the ‘/etc/resolv.conf’ file (as seen in the previous section) on any authorized client machine, using the server’s external IP address instead of the loopback ‘127.0.0.1’ address.
Instructions for DNS configuration on many different operating systems and devices are readily available from a myriad of sources online if you aren’t using a Linux-based client machine. Upon successful configuration, your client should be able to execute the two ping commands in the previous section, verifying a proper setup!

As always, be sure to take precautions and secure your server if you have not done so already. With a functioning DNS server now configured, this project could be expanded upon (as a follow-up exercise/article) by implementing a tool such as DNSCrypt to authenticate and secure your DNS traffic.

Sources

 

Running & Using A Finger Daemon

The finger application was written in the 1970s to allow users on a network to retrieve information about other users. Back before Twitter and other micro-blogging platforms, someone could use the finger command to retrieve public contact information, project notes, GPG keys, status reporting, etc. from a user on a local or remote machine.

Finger has mostly faded into obscurity due to many organizations viewing the availability of public contact information as a potential security hole. With great ease, attackers could learn a target’s full name, phone number, department, title, etc. Still, many embraced the reach that finger could provide. Notably, John Carmack of id Software maintained detailed notes outlining his work in game development.

These days, finger is usually found only on legacy systems or for novelty purposes due to much of its functionality being replaced with the more-usable HTTP.
 
Installing finger & fingerd

This guide assumes we are running a Debian-based operating system with a non-root, sudo user. To allow finger requests from other machines, make sure the server has port 79 open and available.

The first thing we will need to do is install the finger client, finger daemon, and inet daemon:

The inet daemon is necessary to provide network access to the finger daemon. inetd will listen for requests from clients on port 79 (designated for finger) and spawn a process to run the finger daemon as needed. The finger daemon itself cannot listen for these connections and must instead rely on inetd to act as the translator between the sockets and standard input/output.

To ensure that we have IPv6 compatibility (as well as maintain IPv4 compatibility), we will edit the inetd.conf configuration file:

sudo nano /etc/inetd.conf

Find the section that is labeled INFO, and comment out the line under it defining the finger service:

#finger    stream    tcp    nowait        nobody    /usr/sbin/tcpd    /usr/sbin/in.fingerd

Now below it we will add two lines that define the service for IPv4 and IPv6 explicitly:

finger    stream    tcp4    nowait        nobody    /usr/sbin/tcpd    /usr/sbin/in.fingerd
finger    stream    tcp6    nowait        nobody    /usr/sbin/tcpd    /usr/sbin/in.fingerd

Then we will restart inetd to run the changes:

sudo /etc/init.d/inetutils-inetd restart

Now we can use the finger command against our machine:

finger @locahost

 
User Configuration

Each user will have some user information displayed such as real name, login, home directory, shell, home phone, office phone, and office room. Many of these fields are probably not set for the current user account, but many of these can easily be updated with new information.

The chfn utility is built specifically to change information that is retrieved by the finger commands. We can run it interactively by invoking it:

chfn

If we run through this once, we may not be able to edit our full name or wipe out the contents of certain fields. Thankfully, chfn takes several flags to modify these fields individually (and with empty strings accepted!):

$ chfn -f "full name"
$ chfn -o "office room number"
$ chfn -p "office phone number"
$ chfn -h "home phone number"

Now that our information is set, we can start creating files that will be served by finger.

The first file will be the .plan file. This is typically used to store updates on projects, but can be used for pretty much anything such as schedules, favorite quotes, or additional contact information.

nano ~/.plan

Next, we can create a .project file. This file is traditionally used to describe a current project, but can house any content provided it displays on a single line.

nano ~/.project

Next, if we have a GPG key, it can also be included via the .gnupg file.

gpg --armor --output ~/.gnupg --export "my name"

Depending on our machine’s configuration, we can also set up mail forwarding which will be shown when our user account is queried via a .forward file.

echo my@other.email.com > ~/.forward

Now that all the files are created, we need to change the permissions on them to allow them to properly be read by finger. This command will allow others to read and execute our new files:

chmod o+rx ~/.plan ~/.project ~/.gnupg ~/.forward

Afterwards, anyone with finger should be able to query the account provided the host is reachable and the port is exposed:

$ finger famicoman@peer0
Login: famicoman                        Name: mike dank
Directory: /home/famicoman              Shell: /bin/bash
Office: #phillymesh, famicoman@gmail    Home Phone: @famicoman
On since Wed Mar  1 18:28 (UTC) on pts/0 from ijk.xyz
   5 seconds idle
No mail.
PGP key:
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1

mQINBFhQteQBEADOQdY/JyNsRBcqNlHJ3L7XeMWWqG+vlGYjF5sOsKkWDrgRrAhE
gJthGdPSYLIn5/kRst5PkDGjZHFq1k4VAaUlMslCbLgj0xMzUWmhGRKtFjrnQzFi
UVaW/GcW682b5wKkEbSpwRrLHJ19cwYiQHRA6dahiCkWkdh7MluHKTwU1kaUrs3E
3satrSAlOJHH2bg5mDuQPTov/q6hot2pfq8jseQuwVflssqOt4tx3o0tcwbKJwQs
8qU3cVkfg+gzzogM5iMmKAjhFt9Ta7E2kp+iR8gOkH7CK2tu+WIpdpSYuIOe2nEa
AdYSGJRmIZxqwGPwZu893OiYTiLF2dGEWj4cSmZFJ9BYxw9b7dMePDs3l9T1DyJN
FF6JyqtiTpSOfw4+9oNL/+8kmRFMtQBGDH5Dqn4Bg+EYUUGWh3BQb1UGRGNRbls1
SYw4unPsMAaGd0tqafyEOE7kHcQwGMWyI4eFi0bkBRTCvjyqTqTImdr4+xRKn4rW
vS+O+gSSWnrKW67TF0vFuPKV4w4sdPhldcYZjiPNI1m6nmnih4756LW+W5fCKMyN
RN4yPmaF6awEM3fPf3BWhxDmsBvYLqLlE9b6DQ6DmIsC6x5S/jBFr/W0cgZKuMBR
wIHGIpCTgkVsvjTxTeDnZQzEuaZFOpHrYBYG0+JccE5ZZ2/aEupB1tLWzwARAQAB
tCtNaWtlIERhbmsgKEZhbWljb21hbikgPGZhbWljb21hbkBnbWFpbC5jb20+iQI+
BBMBAgAoBQJYULXkAhsDBQkJZgGABgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAK
CRAWGa5NfPKo9xBPEADGK7ol4nU1cDpYc6XPYb4w4s8G/9Ht3UGZvy4ClB7TvntR
HuixWISQyElK4pDntrpXfLmDgqNQqUtjev6w0uEEc7MQW0WBYPlJ9rmVuDJtPjOP
hr9wUnwrd6KypHGw1Y6qukf/w8gCDZyZ8BVOm3r6XT6VlpuVVBP16ax6Mh7sTf4i
/D8D1JH89zApfplqiiA+OAvunbm19jrczby5ILhevbwfpP+ob7FqevCQi/ppGncg
s2LZceIYPh09OzakBiZkIwzMsuWHYxrenUmg8kuVaVyaGXdCc+KZr7c4oGioLxOL
8p8PunjL+i/uGtvZ3tQEvHjB9r1Ghu1t5MUQC4xTvgLnvLOQA0gmknwRQ6nSWrQf
yDJPnaJcBKlfnCR7eKtotsTobUCWDMC9sEvLjLYMgxtEbu46/J57oDQ1LniejI9X
rTUN8cnRLpOQUy0eTBTtUWYqdqO9fAjvMIsnWR4IcMlTJazghygQ+zENvGlEfey2
NL4nC2Yus3EAeCEJC52vccSsf3b7HT/4GemNOVjh72kZ1FM6HL+UNaU7JLppNQk6
mFKC3/wIKxKBjm/vR21Efl+f2iwUAiLjrugaY6g4BXPX3p6iCHftEW7gA6s4UC0A
1HYScq9Duxv+AQpe/mfVA2SBrD3OLTknW2Z5EZSTHqyzRQtHviXTRkdtQscF7rkC
DQRYULXkARAAugwI5dpLpIYI4KZcHsEwyYqUL35ByGCuqklYGOSMkkX0/WFH2ugv
Vs8fqgrn/koXjKBPpxdElfTAGbD2MSTvAUzwMgvMaLZUxY2Qh1RipkNXvSAO+W6W
nJKyBvasboK8l61yta/RrPvUr+equxtBawD3ja9DPzfTYuVCewR6ztcdvqAho5D1
Ds5HxO6aLPsMW8Fj8kf+Ae5eypuNBAN0ivpkDfQyaungh2EjrVpJzWJ8ZqYah/cK
1R55rPhq/JtjcO8g2nnsi+L5EuMma9+50lVPHlHjO9Y0xaQMq2/rLJcDu8UbymX7
LSoxzixiLYtig3GAB+1XkIqMwaEkFF1zTlAZ7drAfhH1AJ0L7SQBur5J1EM3iRYO
XmelyxJPwKgL4K6U4NnnrVpf71GyBktXyuEiOWDFjINs2OzwL4z/l1oWuJuFW/vO
C/yM5Ed7yzLXm0exAnW8Y0u1hwbhRy31zIYbLeB+PuiAqrnvSr1xhAWalBM3dqF+
VIsuKlcoFOmX74/OCEFQ/cqrrkhQ/PrkZqQZCyjyUhkuIqasMZGhLakAIO226E1r
IsUX0jMliJ/A89ZyDmoZyyvRjyydCkOS5HXahFLCufPC8FgcAL6VcpVF2sHBmWcj
vOlfr7OXHzKvKVNy8qymzyfRCPHoYQvxW9UAhru/iIqKOTIDo3OwJ9EAEQEAAYkC
JQQYAQIADwUCWFC15AIbDAUJCWYBgAAKCRAWGa5NfPKo91+XD/0a+gcsXpKo2gy+
oQC/osyJhesx2CGzZqmSB3fpyq9D+jnsCzt/Bh/EtR+2sUWxokIVc5dLzyr0icNl
c0iJBO6It662Q9FnNemiGgv2PLYbjjDC/CP82QWoWrSzPpDKu0DgrF6+MPQgRleT
Z8g0+nLHYMgCTAPfwaaYHLvLLaOt0Ju7L2kt9TSKn2aU2NJReVC5mm3Jxg6Bz3Ae
iQ6iigKI7R/huVDVzBuJQNiToRQswbb/PidYdwyiI3GJC6m+q8HEWAl+cGahqUVg
IIDlmj6SeIsP0r+SGygX0PRWyW/NmQPWamG5e4TDL0pZpu1CzGqfN/A2KLXVO6ss
X2l2HWadBQgd0goFNX7PK210I5B8SfiJM5+cLgChcT9g3mKil8XUkKTSE+c0Q9e5
1AkUrUS69KZeqqtJOB350YP9ZGY7TbOXjXp0Y0/fo3/X+0sZzmP8jIyozLAecM0e
fPrIeA2mego6nWaSRp5FH8KlvHpUvFcKNV+SVSbrbzmSVSpgKmQRp2kd+tyDOPrt
2tXIoWMYdVtlluSYqR2lPv7WFyxCPX8DmxK6fYeVoqBf+7g4GZPdcSviDBlJEJfc
HAZZKsSZzgMugwqLidQ+W53eIDyIOVw0tvcHDJ1S5mpWqvROf7gfNXXFvLjECACN
wnqdjeFGPLlP5Q6tVPvp8j7prVlvZQ==
=xm3N
-----END PGP PUBLIC KEY BLOCK-----
Project:
Philly Mesh - http://mesh.philly2600.net - #phillymesh:tomesh.net
Plan:
%=============================================%
==2017-01-26===================================
%=============================================%
+ Installed fingerd

* Configuring SILC network
* Documentation for fingerd and silcd

By default, finger can display login and contact information for all accounts on a machine. Luckily, accounts can be individually configured so that finger will ignore their existence if there is a .nofinger file in their home directories:

sudo touch /home/someotheraccount/.nofinger && chmod o+rx /home/someotheraccount/.nofinger

 
Conclusion

You should now have finger and fingerd installed and configured on your server for each user to make use of. Keep in mind that the information you enter here will be public (provided the server is) and people around the world may be able to gleam you contact information or even last login time via the finger command.
 
Sources

 

Building DIY Community Mesh Networks (2600 Article)

Now that the article has been printed in¬†2600 magazine, Volume 33, Issue 3 (2016-10-10), I’m able to republish it on the web. The article below is my submission to 2600 with some slight formatting changes for hyperlinks.

Building DIY Community Mesh Networks
By Mike Dank
Famicoman@gmail.com

Today, we are faced with issues regarding our access to the Internet, as well as our freedoms on it. As governmental bodies fight to gain more control and influence over the flow of our information, some choose to look for alternatives to the traditional Internet and build their own networks as they see fit. These community networks can pop up in dense urban areas, remote locations with limited Internet access, and everywhere in between.

Whether you are politically fueled by issues of net neutrality, privacy, and censorship, fed up with an oligarchy of Internet service providers, or just like tinkering with hardware, a wireless mesh network (or “meshnet”) can be an invaluable project to work on. Numerous groups and organizations have popped up all over the world, creating robust mesh networks and refining the technologies that make them possible. While the overall task of building a wireless mesh network for your community may seem daunting, it is easy to get started and scale up as needed.

What Are Mesh Networks?

Think about your existing home network. Most people have a centralized router with several devices hooked up to it. Each device communicates directly with the central router and relies on it to relay traffic to and from other devices. This is called a hub/spoke topology, and you’ll notice that it has a single point of failure. With a mesh topology, many different routers (referred to as nodes) relay traffic to one another on the path to the target machine. Nodes in this network can be set up ad-hoc; if one node goes down, traffic can easily be rerouted to another node. If new nodes come online, they can be seamlessly integrated into the network. In the wireless space, distant users can be connected together with the help of directional antennas and share network access. As more nodes join a network, service only improves as various gaps are filled in and connections are made more redundant. Ultimately, a network is created that is both decentralized and distributed. There is no single point of failure, making it difficult to shut down.

When creating mesh networks, we are mostly concerned with how devices are routing to and linking with one another. This means that most services you are used to running like HTTP or IRC daemons should be able to operate without a hitch. Additionally, you are presented with the choice of whether or not to create a darknet (completely separated from the Internet) or host exit nodes to allow your traffic out of the mesh.

Existing Community Mesh Networking Projects

One of the most well-known grassroots community mesh networks is Freifunk, based out of Germany, encompassing over 150 local communities with over 25,000 access points. Guifi.net based in Spain, boasts over 27,000 nodes spanning over 36,000 km. In North America we see projects like Hyperboria which connect smaller mesh networking communities together such as Seattle Meshnet, NYC Mesh, and Toronto Mesh. We also see standalone projects like PittMesh in Pittsburgh, WasabiNet in St. Louis, and People’s Open Network in Oakland, California.

While each of these mesh networks may run different software and have a different base of users, they all serve an important purpose within their communities. Additionally, many of these networks consistently give back to the greater mesh networking community and choose to share information about their hardware configurations, software stacks, and infrastructure. This only benefits those who want to start their own networks or improve existing ones.

Picking Your Hardware & OS

When I was first starting out with Philly Mesh, I was faced with the issue of acquiring hardware on a shoestring budget. Many will tell you that the best hardware is low-power computers with dedicated wireless cards. This however can incur a cost of several hundred dollars per node. Alternatively, many groups make use of SOHO routers purchased off-the-shelf, flashed with custom firmware. The most popular firmware used here is OpenWRT, an open source alternative that supports a large majority of consumer routers. If you have a relatively modern router in your house, there is a good chance it is already supported (if you are buying specifically for meshing, consider consulting OpenWRT’s wiki for compatibility. Based on Linux, OpenWRT really shines with its packaging system, allowing you to easily install and configure packages of networking software across several routers regardless of most hardware differences between nodes. With only a few commands, you can have mesh packages installed and ready for production.

Other groups are turning towards credit-card-sized computers like the BeagleBone Black and Raspberry Pi, using multiple USB WiFi dongles to perform over-the-air communication. Here, we have many more options for an operating system as many prefer to use a flavor of Linux or BSD, though most of these platforms also have OpenWRT support.

There are no specific wrong answers here when choosing your hardware. Some platforms may be better suited to different scenarios. For the sake of getting started, spec’ing out some inexpensive routers (aim for something with at least two radios, 8MB of flash) or repurposing some Raspberry Pis is perfectly adequate and will help you learn the fundamental concepts of mesh networking as well develop a working prototype that can be upgraded or expanded as needed (hooray for portable configurations). Make sure you consider options like indoor vs outdoor use, 2.4 GHz vs. 5 GHz band, etc.

Meshing Software

You have OpenWRT or another operating system installed, but how can you mesh your router with others wirelessly? Now, you have to pick out some software that will allow you to facilitate a mesh network. The first packages that you need to look at are for what is called the data link layer of the OSI model of computer networking (or OSI layer 2). Software here establishes the protocol that controls how your packets get transferred from node A to node B. Common software in this space is batman-adv (not to be confused with the layer 3 B.A.T.M.A.N. daemon), and open80211s, which are available for most operating systems. Each of these pieces of software have their own strengths and weaknesses; it might be best to install each package on a pair of routers and see which one works best for you. There is currently a lot of praise for batman-adv as it has been integrated into the mainline Linux tree and was developed by Freifunk to use within their own mesh network.

Revisiting the OSI model again, you will also need some software to work at the network layer (OSI layer 3). This will control your IP routing, allowing for each node to compute where to send traffic next on its forwarding path to the final destination on the network. There are many software packages here such as OLSR (Optimized Link State Routing), B.A.T.M.A.N (Better Approach To Mobile Adhoc Networking), Babel, BMX6, and CJDNS (Caleb James Delisle’s Networking Suite). Each of these addresses the task in its own way, making use of a proactive, reactive, or hybrid approach to determine routing. B.A.T.M.A.N. and OLSR are popular here, both developed by Freifunk. Though B.A.T.M.A.N. was designed as a replacement for OLSR, each is actively used and OLSR is highly utilized in the Commotion mesh networking firmware (a router firmware based off of OpenWRT).

For my needs, I settled on CJDNS which boasts IPv6 addressing, secure communications, and some flexibility in auto-peering with local nodes. Additionally, CJDNS is agnostic to how its host connects to peers. It will work whether you want to connect to another access point over batman-adv, or even tunnel over the existing Internet (similar to Tor or a VPN)! This is useful for mesh networks starting out that may have nodes too distant to connect wirelessly until more nodes are set up in-between. This gives you a chance to lay infrastructure sooner rather than later, and simply swap-out for wireless linking when possible. You also get the interesting ability to link multiple meshnets together that may not be geographically close.

Putting It Together

At this point, you should have at least one node (though you will probably want two for testing) running the software stack that you have settled on. With wireless communications, you can generally say that the higher you place the antenna, the better. Many community mesh groups try to establish nodes on top of buildings with roof access, making use of both directional antennas (to connect to distant nodes within the line of sight) as well as omnidirectional antennas to connect to nearby nodes and/or peers. By arranging several distant nodes to connect to one another via line of sight, you can establish a networking backbone for your meshnet that other nodes in the city can easily connect to and branch off of.

Gathering Interest

Mesh networks can only grow so much when you are working by yourself. At some point, you are going to need help finding homes for more nodes and expanding the network. You can easily start with friends and family Рsee if they are willing to host a node (they probably wouldn’t even notice it after a while). Otherwise, you will want to meet with like-minded people who can help configure hardware and software, or plan out the infrastructure. You can start small online by setting up a website with a mission statement and making a post or two on Reddit (/r/darknetplan in particular) or Twitter. Do you have hackerspaces in your area? Linux or amateur radio groups? A 2600 meeting you frequent? All of these are great resources to meet people face-to-face and grow your network one node at a time.

Conclusion

Starting a mesh network is easier than many think, and is an incredible way to learn about networking, Linux, micro platforms, embedded systems, and wireless communication. With only a few off-the-shelf devices, one can get their own working network set up and scale it to accommodate more users. Community-run mesh networks not only aid in helping those fed up with or persecuted by traditional network providers, but also those who want to construct, experiment, and tinker. With mesh networks, we can build our own future of communication and free the network for everyone.

 

I’m in 2600 Magazine

As of the Autumn 2016 issue, I now have an article appearing in 2600: The Hacker Quarterly! My article is titled “Building DIY Community Mesh Networks,” and covers topics in building and organizing local mesh networks.

33-3_cover_large

The issue can be purchased in Barnes & Noble stores, as well as physically or digitally through the 2600 site and Amazon.com. I will shortly be making the article available online as well.