Virtual Instinct- Reloaded

I did a previous post “Virtual Instinct” on virtual machine (VM) basics. In this post, i am going to implement an IPv6 test network using virtual machines. The motivation was that i am writing (or looking into writing, whichever you prefer ) detection scripts for some IPv6 attack tools. I needed to design and create an IPv6 network that would help test both local and global scope attacks, while ensuring that the attack doesn’t get outside my machine and i don’t end up serving in Berkeley ghetto. As can be seen in the figure below, this is the design i came up with. Simple enough. But the fun starts when we start implementing it . Lets go through the implementation step by step.

Creating VM1
In the de facto state, my laptop already had connection to the Internet via wlan0. That’s the right part of this figure and usually connection to a wireless channel takes place automatically. Lets turn our attention to the left part. I installed VirtualBox, and created VM1 with Ubuntu 11.10. By default, it comes with the NAT configuration. I don’t want NAT because i don’t want to go anywhere close to wlan0. That’s prohibited territory for my VMs. I want my host to connect to VM1 over a separate interface. So i open the VB manager, click on VM1, click Settings in the top menu and select Network. Now you can see that there are 4 network adapters available to you. The first one is open by default. Check ‘Enable Network Adapter’, attach to ‘Host-only adapter’ (http://www.vmware.com/support/ws55/doc/ws_net_configurations_hostonly.html), it will be automatically assigned the name vboxnet0, click OK and we are good. Whenever i mention network settings again, you should follow the same steps i just described. Next i installed guest additions, enabled clipboard sharing and created a shared folder with my host OS (all explained in my previous VM post). This part is not mandatory, but it will make your life very easy.

Creating VM2
The next task is to create VM2. Now VM1 was up and running in about 15 minutes. Do i really want to wait that long and create another machine from scratch when i know that all i need is to replicate VM1, same OS and all? So what i do is that i right click VM1 in VB manager, select clone and further specify that i want a linked clone. Do i hear ‘linked clone, WHAT’? If it helps, take comfort in the fact that i don’t know much either . But i know what’s important to know, and here is a summary of it (http://www.vmware.com/support/ws55/doc/ws_clone_overview.html):
(1) “Changes made to a clone do not affect the parent virtual machine. Changes made to the parent virtual machine do not appear in a clone.”
(2) “A clone’s MAC address and UUID are different from those of the parent virtual machine.”
Also note that, “A linked clone must have access to the parent. Without access to the parent, a linked clone is disabled”. Now this is not a point that bothers me, so Yay to linked clone.

Now that VM2 is born, lets turn our attention to its network settings. We need to connect VM2 to VM1. Now VM1 already has a host-only connection with the host OS over vboxnet0. We could join this network but that would make VM1, VM2 and Host OS on the same LAN, which is not in accordance with our design. We need a separate connection between VM1 and VM2. This means that we need to create another adapter on VM1 specifically meant for connecting to VM2. Lets do that now. Go to network settings and enable adapter2 with ‘internal network’ (http://www.virtualbox.org/manual/ch06.html#network_internal). Also create an internal network on VM2 by following the same settings but remember that you need to enable adapter 3 (adapter 1,2 and 4 should not be enabled). So what’s the deal with adapter 3? We already enabled adapter 1 and 2 for VM1 and they have been assigned MAC addresses X and Y respectively. If you enable adapter 1 on VM2, it will get MAC address X (or Y for adapter 2). Because IP addresses are assigned based on MAC addresses, VM1 and VM2 will end up having the same IP address for their interfaces. Again, this will clash with our design.

Initial Testing
Do ifconfig on VM1 and note that it has two IP addresses, one for host OS and the other for VM2. Also note that the interface names are eth0 and eth1. Eth0 pairs with vboxnet0 on host OS, and eth1 joins with eth1 on VM2. Figure out IP addresses (do ifconfig) for interfaces of interest on host, VM1 and VM2 and go crazy pinging. In its current form, the following pings should take place.
Host<–>VM1
VM1<–>VM2

Some babble about interface naming
The names are prefixed by eth because VM doesn’t know what’s the underlying network card. All it sees is that it is connected to a (virtual) ethernet NIC. I have also noticed that the number after eth occurs in increasing order of the virtual network adapter number. For example, if adapter 2 got eth1, then adapter 3 will get the name eth2 and so on. It’s not that big a problem but it’s weird. I still can’t get over why my network adapter 3 on VM2 got the name eth1 when it was the first one and deserved to earn the title ‘eth0′. And if VM somehow wants me to believe that naming is in accordance with network adapter numbers, pray tell me how does network adapter 3 get the name eth1? By this logic, if we were to enable adapter 1 on VM2, it would have been named eth-1 (minus 1).

Bye bye Ipv4
On most modern OS (including Ubuntu), IPv6 is enabled by default. You can check by doing ifconfig and looking out for IPv6 addresses. Alternately, open terminal and  type in cat /proc/net/if_inet6. You should get something like:

————————————————————————————————-
fe8000000000000072f1a1fffe9710c1 03 40 20 80    wlan0
00000000000000000000000000000001 01 80 10 80       lo

————————————————————————————————-
If you don’t, then you need to manually enable it (http://www.cyberciti.biz/faq/check-for-ipv6-support-in-linux-kernel/).

You’ll notice on doing ifconfig that each interface is also allocated an IPv6 address alongwith IPv4 address. I want this to be a pure IPv6 network. The reason for my partiality to IPv6 is that i am going to have to analyze attack traffic and i want minimum noise. I don’t need IPv4, so how do i get rid of it? The answer is simple. Just turn it off in VB manager. Go to File->Preferences->Network. You’ll see your adapter(s) listed, now click on vboxnet0 and then click on what looks like a screwdriver (if you hover over it long enough, it says ‘Edit host-only network’). In the window that pops up, click on the tab DHCP server and uncheck Enable. Sweeeeeet! .

Do ifconfig and you’ll see only IPv6 addresses. But wait, we just turned off the DHCP server, who assigned these addresses? Welcome to IPv6. IPv6 is capable of what is called address auto-configuration. A more detailed discussion on Ipv6 is beyond the scope of this post which, by the way, is already getting too long for my taste So i’ll cover only some very basic info about IPv6 which we’ll need during our setup.

Welcome IPv6
Wikipedia has a nice page on IPv6 (http://en.wikipedia.org/wiki/IPv6). There are actually two points of interest here.

1. IPv6 has an in-built mechanism to configure addresses for its interfaces (Stateless address autoconfiguration (SLAAC)).

2. You’ll notice that the addresses start with fe80. That’s because fe80::/10 is the link-local scope. Also, note that the link-local IPv6 assigned to an interface will typically remain unchanged across reboots. This is because the link-local address is derived from MAC address, unless specifically specified otherwise. So what is link-local scope? This brings us to our second point. In IPv6, a single interface can have multiple IP addresses associated with it. These addresses differ from each other in their scopes. Now there are a bunch of scopes which you can read in detail in this very nice IPv6 cheat sheet (www.roesen.org/files/ipv6_cheat_sheet.pdf). The scopes we need to be familiar with are link-local and global. As the name implies, link-local addresses are valid only on the local link. On the other hand, a global scope address is globally identifiable. What does this mean to us? Well, SLAAC only takes care of link-local addresses, the global scope addresses have to be configured manually or through DHCP. Lets do it then, the manual global-scope IP address configuration, that is.

Manually assigning IPv6 global-scope addresses
The IPv6 global scope is 2000::/3. Keeping in mind our design, lets assign the addresses, as specified in figure 1, in this manner:
ifconfig [interface name] inet6 add [the global scope IPv6 address]
I’ll do the one for host, and the rest can be configured in a similar way:
———————————————————————————————–
ifconfig vboxnet0 inet6 add 2000:1000::/32
ifconfig
vboxnet0  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
inet6 addr: 2000:1000::/32 Scope:Global
inet6 addr: fe80::800:27ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:413 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B)  TX bytes:77497 (77.4 KB)

—————————————————————————————————–

It’s test time
I assume that all the interfaces have been assigned IPv6 addresses according to figure <x> by now. These pings for the global IPv6 addresses should be successful:
Host <–> VM1
VM1 <–> VM2
Note that the traditional ping won’t work for IPv6 addresses. You need to use ping6 like this:
ping6 -I vboxnet0 2000:1300::2

Sometimes the greatest journey is the distance between two hosts on different networks
Nice title, which by the way is a twist on the tagline of “The Painted Veil”. What we did so far was the easier part (at least for me). The nasty part is getting host OS to talk to VM2. lets just do a simple exercise. On your host OS, do:
—————————————————————-
ip -6 route show
fe80::/64 dev vboxnet0  proto kernel  metric 256
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
—————————————————————-
What does this information mean? Lets look at the first line. It tells the Kernel that packets that have their destination field set to be an address in the range fe80::/64 should be forwarded on interface vboxnet0. This command provides you a nice way to look at kernel’s routing table. This table is consulted to help Kernel decide which interface to forward a packet to based on its destination address. If the destination address does not match with any of the interfaces, the packet is dropped. Now do this on the host OS:
ping6 -I vboxnet0 2000:1200::1

As expected, this won’t succeed. Why? Because the Kernel looks at the interface in its routing table and notices that the interface is configured to accept packets addressed to 2000:1300::/32 only. Naturally, we want to tell the Kernel somehow that we *know* that 2000:1200::/32 can be reached through 2000:1300::2 on interface vboxnet0, so just pass it on. Lets state these steps systematically:

1. Tell host OS to pass on packets for 2000:1200::/32 to 2000:1300::2 on vboxnet0 t. The following command does exactly this:
sudo ip -6 route add 2000:1200::/32 via 2000:1300::2 dev vboxnet0

2. Tell VM1 that if it receives a packet on eth0 from 2000:1300::/32 and whose destination is 2000:1200::/32, then it should be forwarded to eth1. Now this isn’t default behavior for a host. In typical operation, hosts are information sinks. Information passing/forwarding is the routers’ job. Here we are making a host behave like router by defining a static path between two interfaces. So we need to do some extra work. The first one is to enable IPv6 forwarding. Note that in most Linux distros, IP forwarding is turned off by default.
sudo sysctl -w net.ipv6.conf.all.forwarding=1
Next, Linux has what’s called iptables, which is a fancy name for a firewall to be used by Kernel. Lets tell this firewall that we are ok with packets with certain destination address being forwarded to eth1. For more details on iptables, look at its manpage. The following commands di what we just discussed. The first command is to let host OS and VM2 talk over the path eth0->eth1 over VM1. The next command is the reverse of the first command to enable bidirectional communication, i.e., VM2 to host OS.

sudo ip6tables -A FORWARD -i eth0 -o eth1 -s 2000:1300::/32 -d 2000:1200::/32 -j ACCEPT
sudo ip6tables -A FORWARD -i eth1 -o eth0 -s 2000:1200::/32 -d 2000:1300::/32 -j ACCEPT   

Testing makes a network perfect
It’s ping time again. All possible ping combinations involving host OS (vboxnet0), VM1 and VM2 should work now.

Making life easier
There are two things i want to talk about.
1.  The IPv6 addresses are kind of difficult to remember so you can give a human readable name to each of your addresses. Just edit the file /etc/hosts. You’ll figure out the format by looking at the file. It’s something like this : <IP address>    <Name>

2-. Manually assigned IP addresses and forwarding information is not persistent across reboots. So it’s a good idea to write commands in bash scripts to run at boot up. Next make entry for the bash script in crontab.

“Cron job are used to schedule commands to be executed periodically. You can setup commands or scripts, which will repeatedly run at a set time.
Each user can have their own crontab file, and though these are files in /var/spool/cron/crontabs, they are not intended to be edited directly. You need to use crontab command for editing or setting up your own cron jobs that will be performed upon login.” (http://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses/). Execute ‘crontab -e‘ and add a line like:
@reboot path/to/myscript.sh

If there are permission issues, you might want to ‘chmod 777 myscript.sh‘ which is generally a bad idea but my last resort when i get into linux permissions mess Remember that while it’s a good idea to include these bash scripts in your crontab on the VMs, you’ll have to manually run the bash file in case of host OS. That’s because until you’ve run the VM, the host OS does’t recognize the interface vboxnet0 because it’s virtual. As crontab scripts are run at login time, nothing will happen if you include .sh file in crontab on host OS because the host OS simply doesn’t know what vboxnet0 is. So here is a typical sequence of events: start VM1, start VM2, manually run .sh file on host OS. For reference, i am including my .sh files here and yes, congratulations, we have reached the end of this post at last

—————————————————–
Host OS
—————————————————–
#!/bin/bash

sudo ifconfig vboxnet0 down
sudo ifconfig vboxnet0 inet6 add 2000:1300::1/32
sudo ifconfig vboxnet0 up

sudo ip -6 route add 2000:1200::/32 via 2000:1300::2 dev vboxnet0

—————————————————–
VM1
—————————————————–
#!/bin/bash

sudo ifconfig eth0 down
sudo ifconfig eth0 inet6 add 2000:1300::2/32
sudo ifconfig eth0 up

sudo ifconfig eth1 down
sudo ifconfig eth1 inet6 add 2000:1200::2/32
sudo ifconfig eth1 up

sudo sysctl -w net.ipv6.conf.all.forwarding=1

sudo ip6tables -A FORWARD -i eth0 -o eth1 -s 2000:1300::/32 -d 2000:1200::/32 -j ACCEPT
sudo ip6tables -A FORWARD -i eth1 -o eth0 -s 2000:1200::/32 -d 2000:1300::/32 -j ACCEPT

——————————————————-
VM2
——————————————————-
#!/bin/bash

sudo ifconfig eth1 down
sudo ifconfig eth1 inet6 add 2000:1200::1/32
sudo ifconfig eth1 up

sudo ip -6 route add 2000:1300::/32 via 2000:1200::2 dev eth1

Virtual Instinct

Let me warn you in advance that the post does not live up to its sensational title, sorry . Recently i had to do some work with virtual machines (VMs). In this post, i’ll briefly touch on some VM related topics and provide reference to the resources that i found useful. My host OS is Ubuntu 11.10, VM is VirtualBox and guest OS is also Ubuntu 11.10.

If you look forward to work in a standalone environment, then setting up a VM is so very easy. Things do get a little tricky when you need a bunch of VMs to talk to each other and also with the host OS. Actually, the process itself is not so difficult. Finding the right resources to help you jumpstart could be something of a pain. Lets dive into all things virtual, then:

1. Virtual Machine, WHAT?
This is very basic but it doesn’t hurt to start from the very basics Your native OS is your host OS and any other OSs you want to run with the help of VM (VirtualBox, VMWare or whatever) are the guest OS. The VM provides a layer of abstraction between the guest OS and the actual physical devices connected to your machine. Thanks to the VM, The guest OS doesn’t know that it is not directly interacting with the physical devices, e.g your NIC. For the more techy ones of us, there are a bunch of definitions of what a VM is. I prefer the one provided by VMWare:

“A virtual machine is a tightly isolated software container that can run its own operating systems and applications as if it were a physical computer. A virtual machine behaves exactly like a physical computer and contains it own virtual (ie, software-based) CPU, RAM hard disk and network interface card (NIC).”

2. Why need a VM?
There could be many reasons. The ones i can think of on top of my head are:
a) You’re curious about another OS and want to try it out. ( Actually you can use live cd for that purpose but VM is not a bad idea either, particularly if you happen to be the indecisive type )
b) You want to run an application that is supported by another OS than the one you like to use on day-to-day basis.
c) You’re running a nasty application but don’t want to wreak havoc on your native OS or that of others. It’s a good idea to run it in a VM and completely isolate it.
d) You need to test some network-related stuff but don’t have the time/resources to set up a physical network. The answer is…you guessed it…VM networking

3. How to install VirtualBox ?
Don’t be tempted to use Ubuntu’s software centre to install VirtualBox for you. I installed it this way first and when it was time to do some advanced tweaking, i found out that certain stuff was missing which cannot be downloaded independently…it comes as a package. To quote a forum post:
“I had this (embarrassing) problem too. I think it has to with the VBox version in the repos. I don’t think it comes with LinuxAdditions. I uninstalled it and got the deb from the VirtualBox site and all was cool.It “suddenly” appeared in /usr/share/virtualbox/ I mounted the iso in the guest, copied the LinuxAdditions run file to /home and ran it with sudo sh./ All is now well in my virtual world – I can even get a windowless VM! yay.”

Here is the right way to do it:
a) Download .deb file from here (https://www.virtualbox.org/wiki/Downloads).
b) Install it the ‘deb’ way, i.e., cd into Downloads and do sudo dpkg -i <your_package.deb>
c) That’s it. You can start it from terminal like this: virtualbox & or use GUI. It should be in Applications->Accessories or Applications->System Tools.

4. Installed. Now what?
a) Networking:
A good first step is to understand VM networking. By default, the NAT mode is enabled. This mode is alright if you intend to do some casual web browsing. You might need to reconsider your choice if you want others on the network to be able to directly communicate with you. In this mode, connections cannot be directly initiated to the VM (without elaborate tweaking–>port forwarding). Only outbound connections are allowed. Also, all outgoing traffic will bear the host OS’s IP address in ‘source’ and all incoming traffic will be addressed to the host OS’s IP (dest), which is then passed on to the VM.

Here is a good resource to get a basic overview of VM networking:
http://www.thegeekstuff.com/2012/03/virtualbox-guest-additions/

b) Guest Additions:
Installing this stuff can make your virtual life a breeze. You can do fun things like enabling clipboard (allows you to copy paste stuff between the guest and host OS) and shared folders between the guest and host OS. Again, you can read details in the following link. I skipped steps 8-14.
http://www.thegeekstuff.com/2012/03/virtualbox-guest-additions/

There are tons of videos on YouTube and web to help with clipboard and folder sharing stuff. I found the following useful:
Clipboard:

Shared Folder:
http://www.liberiangeek.net/2011/10/access-virtualbox-shared-folders-within-ubuntu-11-10-oneiric-ocelot/

5. What next?
I *might* be doing a post on simulating networking topologies using VMs. Stay tuned and enjoy the comic on what to do when your OS crashes (i didn’t make it, found it somewhere on the Internet). The one about Linux made my day

——————————————————————————————————————————————————————————————–

Playing with ALOT of network traffic

I was involved in a project that required analyzing and mining information of interest from large network traces (spanning TB’s of data). This was my first time with network data of this magnitude so i learnt quite a few things by hit and trial. I decided to do a quick post to summarize my experience and ‘humble’ insights!

1. Whenever you get a dataset, don’t take the data-provider’s word for it. Do the following:
a): Find out what are the time settings on the machine on which the capture was taken.

Reason: All capture files actually maintain timestamps in terms of the standard epoch (The time in seconds since epoch (Jan 1, 1970 00:00:00)). However, any tool you use to view the dump file will change this time according  to the local time settings on your machine. Most tools, including Wireshark even change the sequence of packets so don’t believe what you see in Wireshark and use your own script to verify timestamps.
b): Calculate the duration of capture by subtracting start timestamp from the end timestamp. If the duration makes sense, you are good to go, otherwise you’ll have to write a script to print interarrival times between packets and mark the ones in which this time is unacceptably large. You might want to investigate why.
c): Run a script to calculate incomplete handshakes. It will give you an idea what kind of data you have at hand.
d): Run a script to calculate data loss based on packet Seq and Ack numbers.

2. Split the data into 1 GB files. Can’t emphasize this enough.  Barring the initial overhead involved in splitting the file (took about 10 hours to split 238 GB), it saves so much time and effort and makes debugging alot easier. If a script halts, you know where to restart or where to look for problem. Also, many so called state of the art tools tend to crash when presented with a large chunk of data.

3. You *may* want to further split data based on ibnound and outb ound traffic. Alternately, instead of physically seperating inbound and outbound traffic, you can implement filters in your script (i prefer this .

4. Make provision for statefulness in all your scripts. Print out useful information in log files so that even if the script crashes, you can pinpoint the problem.

5. Always test your script on a small chunk of data before unleashing it on the data giant.

6. Python+Scapy is a bad bad choice for parsing pcap files. In high level languages, Java’s jpcap is probably the best bet. A python script that took above 72 hours to parse 238 GB data, did the same in 2.30 hours when reimplemented in Java.

7. Enable remote access on your machines. Saves a great deal of time. But be ‘careful’ — don’t turn it into a hacker fiesta!

8. Replicate data and results wherever they can fit. Hard disks will fail, computers will crash and all hell will break loose the moment you decide to do anything worthwhile with your data.

Now that i am familiar with Bro IDS, i intend to do a re- of this post mentioning things you can use Bro to do for you. Why reinvent the wheel? See you again, soon-

Dealing with vector graphics in LaTeX

There are a number of ways to include your drawings in your LaTeX file. I prefer to go the traditional route, i.e., convert the drawing to .eps format and include it in the LaTeX file. However, this too has the potential to turn into a real pain. The final .pdf might look ok at a first glance but as you zoom in, the figures don’t look as crisp as you’d like them to be.

To cut a long story short, vector graphics tend to throw tantrums when they need to be represented in raster form. I tried a number of methods to get around this problem. Finally, i reached the conclusion that nothing works like OLETeX. In the following paragraphs, i describe my journey in the graphics desert and how i found an oasis in OLETeX.

How to generate eps file from a visio file:

If you have a printable object (png img, word, visio drawing etc.), you can convert it to .eps format by following these steps:

1. Add a new printer and select MS Color Printer from the list.
2. Open your visio drawing and select print. Check the option print to file and click print. Save the file with the extension .ps. We’ll call our file drawing.ps
3. Download latest version of Ghostscript which is an interpreter for Adobe Systems’ PostScript and Portable Document Format (PDF) page description languages.
4. Download GSView which provides gui for handling ps files using Ghostscript. By the way, you can skip GSView and use command line to get things going with Ghostscript as your interpreter.
5. Now lets turn our attention to drawing.ps. Open it using GSView and from the file menu, select PS TO EPS option.  I do not particularly like GSView’s bounding box calculation, so I like to keep the ‘automatically calculate bounding box’ unchecked and do it manually when prompted. Save the file with the extension .eps.

What the hell is a bounding box?

Now if you open your newly created drawing.eps, go to options and select ‘show bounding box’, you’ll see a dotted box. So this is your bounding box, no points for guessing. More formally, bounding box is the smallest box that can contain your figure. Often, there is unnecessary white space around a figure which looks nasty, but really screws things up if you embed your figure inside LaTeX. Obviously, the main problem lies with your bounding box. This problem can be solved in two ways:

1. Specify the bounding box coordinates in your LaTeX file where you define your figure.
2. Change the eps source file.

I don’t prefer the first option as I believe that a LaTeX source file should not reflect its author’s idiosyncrasies. In simple words, it should be clean, with bare minimum frills. Coming to the second option, it may look tedious and freaky at a glance, but it’s not so difficult.

All you have to do is to open your drawing.eps with word, notepad or whatever text editor you can get your hands on. You should see something like %%bounding-box somewhere at the beginning of the file. You’ll see four numbers or coordinates following the %%bounding-box tag. These four points define your bounding box and correspond to <bbllx bblly bburx bbury> where the first two points correspond to (x,y) coordinates of the lower left corner of your bounding box and the last two relate to its upper right corner. You can change these to reflect your desired bounding box. Open drawing.eps with GSView and point the cursor to the lower left corner of your figure. Note down the coordinates displayed at the bottom of the window. Now do the same for the upper right corner of your figure.  Enter this information in the drawing.eps source file and save the changes. Congrats,  you’re done!

Come OLETeX

You can bypass all this hassle by using a tool called OLETeX. I could not install it on my Windows 7 (apparently it doesn’t go well with x64 architectures). But it works fine on Windows 9x, Vista, XP and the likes. Gives really great results.

Bring it on Bro!

I’ve been playing with Bro a little bit. No, Bro is not my pet. I am talking about Bro the open source NIDS. So I was scared to death about the prospect of working with Bro, thanks to the hype surrounding its complexity. Another downer was the note on their Wiki page saying “NOTE: This wiki is out of date and no longer maintained”. So my last option was to use their latest workshop material. After going through the workshop, I still felt kind of lost. After remaining on a heavy dose of Bro for about eight days, today I finally feel like I’ve seen light of the day after a long stormy night. I thought I’ll share my understanding of the tool with you to make it easier for you (read: how cool am I  ?).

To begin with, it is not so difficult after all. If you come from a programming background, you shouldn’t be worried at all. I’ll take you forward step by step. First, lets look at the basic architecture of Bro (the figure below is mine; it is based on the original figure on Bro’s website).

Network:

You have packets flowing in and out of your machine. You need a way to capture those packets for your own use. Bro does this by using libpcap. The nitty gritty details of how it captures network traffic don’t matter from user point-of-view. In some cases, you might not be interested in all the network traffic. Instead, you are interested only in traffic of certain kind, for example, HTTP. You can do this by (optionally) specifiying capture filters. Alternately, you can also set ‘rejection filters’ which essentially means: ‘I am interested in any traffic BUT this kind of traffic’. Basically, filtering enables you to relieve Bro of extra, unnecessary processing by capturing only meaningful network traffic;  ‘meaningful’ being subject to user requirements.

Analyzer/Event Engine:

Apart  from hard-wired filters that indiscriminately apply to all traffic, there can also be some tell-tale signs that indicate ‘fishiness’. This logic is built into analyzers which look for anomalies and known bad-behavior in different protocols. It doesn’t have to be related to protocols necessarily. For example, multiple login failures can hint at breaking into a computer via brute force. In a nutshell, fishy behavior is observed using analyzers. The output of an analyzer is one or more events. An event simply announces to a subscribing module that something has happened. It’s just like telling your secretary to give you a call if she sees a man with black jacket in the lobby. So, the criteria ‘man in the black jacket’ is defined in the analyzer, which is your secretary’s memory in this case. The event is secretary’s call informing you that she has indeed witnessed a man with a black jacket. You are the subscribing module. In Bro’s case, the subscribing module will reside in Scriptland as we’ll see..

If you are merely using Bro, you won’t be required to do anything here. You belong in Scriptland. However, if you are a developer meaning to extend the functionality of Bro, you may tweak existing analyzers or write new ones.

Scriptland:

Most users of Bro will find themselves messing around in Scriptland. Bro has a custom language which they call ‘policy scripting language’. By default, Bro is neutral. That is to say, Bro does not impose its own ideas of ‘abnormality in network traffic’ on its users. Instead, users are expected to define abnormality according to their situation. For example, one network administrator might say ‘Alert me if a spam email originates from my network’. Another might say ‘I am kind of lazy. Just let me know when there are a hundred spam emails from my network’. So users have the flexibility to specify what is interesting information to them. This is great. I repeat, this is GREAT. Most state-of-the-art tools lack this level of flexibility.

Most of the scripts in Scriptland merely handle events generated by the Analyzer/Event Engine. For example, Event Engine might tell you that it saw a failed http connection. In Scriptland, you’ll increment the value of a global variable, say, n_failed_conn. Within the next 30 minutes, your script in Scriptland receives about five hundred similar events and obviously, n_failed_conn=500, so by setting up a simple condition, you can do any of the following actions:

———————————————————————————————————————–

Write to a disk (e.g. Between the interval x-y I saw 500 failed http conn originating from w.x.y.z )

Update or maintain some global state data. We already did that with n_failed_conn when it received failed HTTP connection events while it was < 500.

Generate alarms for syslog. This is particularly useful for network forensics.

Call another function e.g. print the ip address of the responsible host

Generate another event.

Invoke shell commands to block flows, or pass information to routers to update their acl etc.

————————————————————————————————————————-

This is enough for now. More, later.