Shiva, the spam honeypot. Tips and tricks for getting it up and running.

Following my last honeypot blog post, I’m going to be discussing the email spam honeypot called Shiva.

I’m going to walk you through installing and configuring Shiva, including some tips and tricks I’ve learnt along the way; which should help you fine tune this honeypot and get it working just the way you want it.

Shiva is a high interaction email spam honeypot written in Python 2.7 and built on the Lamson Python SMTP framework. Shiva contains two parts, the Receiver and the Analyzer [sic].

Shiva has some great features built in which are easily configurable:

• Controlled Relay You want the spammer to think they have found a genuine open SMTP relay so you need to allow some of their “test” email to be relayed, otherwise they will probably move on to the next open relay. This section allows you to configure how many individual spam messages do get relayed and in what time frame. Personally, I’ve kept this figure quite low, roughly between 25 and 50.
• Backend Database Shiva uses a mail parser to pull out all the important information and stores it in a MySQL database for you. This is useful for querying at a later date, and generating statistics.
• Fuzzy Hashing Shiva uses a fuzzy hashing technique to identify new and already seen spam. This means a low powered box can cope with millions of spam, without clogging up the database but still retaining the information that’s useful.

There are other useful features, but I suggest you read the user manual once you get more accustomed to Shiva and how it works.

Installation

So let’s start on the installation. For the purposes of this blog, I’m using xUbuntu but any Debian based distro is fine. Obviously if you’re using a VPS, Ubuntu Server is a good choice.

The first thing to do is make sure we’re up to date:

Let’s then install all the required pre-requisites:

I also had to install libffi and libfuzzy:

And, if you want to store you spam data in a local database, you need to install the following MySQL packages:

The MySQL installation should prompt you to choose a root password, make it complex and remember it as we’ll need it later.

Your machine is now ready to get the latest Shiva. The easiest way to get Shiva is via GitHub:

Assuming that worked correctly, you should have a folder called “shiva-installer”. If all is present and correct, let’s start the installation:

The installation script will check all the required dependencies are installed. If they aren’t, it will kindly tell you. If the dependency checks pass, the install script will ask you if you wish to set up local databases for storing spam. Press Y if you do, N if you don’t.

Configuration

Once the install script has finished, we need to perform a couple more steps before we can start our spam honeypot. Some of these are things I learnt the hard way, others are essential. An important file that we will dip in and out of several times is the shiva configuration file, shiva.conf, which the install script will have created in the sub folder “shiva”. All the other installed files will also have been created within this sub folder, so remember its path as we’ll be using it lots.

The shiva.conf file is worth looking through. It’s broken down into sections with each section having a heading label between square brackets. For example [global] and [receiver].

If you want to store your spam information in a local database, we need to change some settings in the shiva.conf file. Edit the file with your favourite test editor and look for the section labelled [database]. Change the password to the MySQL root password that you remembered earlier.

Save the conf file, make sure you are in the “shiva” install directory, and run:

If all went well, you should see some output that says both the main and tempdb were successfully created.

We need to configure exim to work for our spam email to be relayed. One tip that I learnt the hard way, the default exim config file comes with IPv6 enabled, which the Shiva exim install script doesn’t like. So, before we run the Shiva exim script, we need to edit:

And change the line:

to read:

This disables IPv6, and stops the nightmare of the Shiva exim install script from failing. Believe me, it’s a headache. Once that’s saved, again making sure you are in the Shiva directory, run this:

You shouldn’t see much, just a couple of lines that say the MTA is stopping, then another saying it is restarting.

From an install point of view, that’s pretty much it. We just need to configure some settings in the config file and we’re ready to fire Shiva up for the first time. So, open the shiva.conf up and change the IP address under [receiver] to be your NAT’d or public IP address and change the listenport to 25. This helps your spammers to find you. Bear in mind that most popular ISPs will not accept port 25 / SMTP traffic so don’t expect this to work behind your home router.

We’ll change the Analyzer section next. The only settings I’d suggest changing are the “individualcounter” and the “globalcounter”. For me, the default numbers are a little too high, so I’ve lowered both of mine down to 15 and 30. They can be increased again at a later date. In theory, you will have already changed the [database] settings from earlier, but it’s worth checking again. Make sure the localdb is set to true.

There are lots of other configurable settings in the Shiva.conf file but for further reading, I’d suggest going through the user manual as they are clearly explained in that.

Get It Started

Remember earlier I said that Shiva was broken down into the Receiver and the Analyzer? It’s best if both are running at the same time, otherwise you’ll get a big build-up of spam to analyse which can cause problems. My personal favourite way to keep both Shiva processes and any other processes running whilst not connected to my remote server is the great tool called “screen”.

Screen is a way of opening a shell (or screen) within a shell. To start a new screen session, type “screen” at the command prompt. To see if you already have any screens running, type screen –ls and to resume an existing screen session, type screen –r.

with it. A full break down of how screen works, and its numerous keyboard shortcuts is way out of scope for this post. There is good introductory resource here good introductory resource here though.

Most screen keyboard shortcuts are activated via the CTRL (Control) and “a” key followed by another key. For example, CTRL + A, followed by C will create a new screen window.

Here are a few useful ones:

• CTRL + A, K – kills the active screen window
• CTRL + A, W – shows you which screen number you have active (this is also a good way of finding out if you’re in screen still or not)
• CTRL + A, D – disconnects from the current screen session. It’s always good to get into the habit of disconnecting from screen so you know where you are when you reconnect to SSH
• CTRL + A, 1 – jumps to screen 1. Press other numbers to open any other screen windows you have open

Also, screen numbering starts from zero. Think of screen as having multiple tabs open but you have to select each tab with a keyboard shortcut. I’d also suggest getting into the habit of disconnecting from your screen session and resuming each time you reconnect to your remote host. That way, you should always know where you are.

Start a screen session, and we’ll fire up the receiver section of Shiva. From the command line, type these commands:

I’ve left the shell prompts in so you can spot the correct changes. If you want to check if the receiver has fired up correctly, there is a logs sub folder, take a look at the lamson.log. If you see something like “SMTP Receiver successfully started” then all is fine.

Onto the analyser part. Start another new screen session (CTRL + A, C). From the command line, type these commands:

Again, I’ve left the shell prompts in so you can spot the correct changes. If you want to check if the analyser has fired up correctly, check the log in the sub folder lamson.log. If all went well, you should see something like “Queue receiver started”.

A quick explanation of the files in the each of the logs folders:

• lamson.err – All the errors and relay logs are stored in this file
• lamson.log – All the logs generated by SMTP receiver
• lamson.out – Anything supposed to be printed on console by program is written in this file
• clearlogs.sh – A Bash script that will delete the log files and create fresh empty ones for you

Lessons Learned and Tips

Keep an eye on high volume spammers and block them with iptables if needs be. I had to deny some entire blocks from Taiwan. They were sending so much spam through my honeypot that the analyser couldn’t keep up. This had two effects, one caused a massive back log and build-up of files, the other caused a disk space error. The disk space error wasn’t actually space, but inodes. The queue to be analysed had grown so large it hit the maximum number of index nodes on the file system. Type the command:

Look for 100% usage which appears something like this:

It’s more than likely that the problem. It’s not easy to cure, but here’s how I go about it. Normally it’s the folder /install path/shiva/queue/new/ that is causing the problem. If you try to “rm” the directory/files you’ll get an error stating “Argument list too long”. Make sure you are in the “/queue/new/” directory and try this command:

It’s slow, but works a treat. Bingo!

Back down to a normal level. Keep an eye out for spammers passing large volumes through your honeypot, again iptables is your friend here.

That’s about all for this post. Any questions, catch me on Twitter @ghostie_