During the last few months the average ADSL bandwidth usage has grown to over 50 GB per month.  As bandwidth is still relatively expensive in SA (I use Afrihost’s excellent value for money ADSL at R29/GB) I needed to see where this actually goes.

The solution is a ubuntu based linux server with a collection of different open source products installed, plugged into the network just before the traffic enters the internet.

The ideal place for this server is between the network switch and the default gateway.  All traffic from devices on the inside of the bridge will now have to pass through the bridge server before entering the internet.

The server contains two network interfaces and is configured in bridge mode, allowing network traffic to pass through the server transparently, without even being aware of being scanned.

The key to this working is by bridging both ethernet interfaces together, using the ubuntu bridge-utils package and brctl command. Also make sure that http traffic  destined for the web gets redirected to dansguardian for filtering before leaving the network.

Using bridge mode has the added advantagew that it is not only completely invisible, it also require no changes on the existing netork pc’s. As long as it is placed just before the external gateway / firewall then all traffic will pass through it, including hardware devices like wifi routers, etc.

Once the server is installed, configured and switched into bridge mode the necessary monitoring software is installed. I use squid as a proxy server, dansguardian for traffic filtering and ntop for bandwith monitoring.

Dansguardian is great for rule based filtering, but comes with a static set of rules. I expanded my configuration by adding blacklists from shalla. The shalla lists are broken down by category and contains over 1.3 million entries.  As the lists are updated frequently I configured a cron job to download the lists nightly and copy it to the appropriate folders.

In dansguardian I then enable the specific categories that needs to be blocked, like porn, gambling, etc. Dansguard also contains exception lists for blocking or unblocking specific sites, keywords or ip addresses.

Another handy feature of dansguardian is that is uses clamav to automatically scan all downloaded files for virusses before letting it through.

Squid is configured as a transparent proxy server, allowing traffic caching and reporting. Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages.

Ntop is used to monitor and report on all network traffic passing through the server. When configuring ntop make sure to set it to watch the external interface only, so you do not pick up traffic not destined for the internet.

Iptables is used as a firewall for blocking specific protocols or destinations, and also for forwarding all HTTP traffic destined for the internet to dansguardian, which applies its rule based filters and then pass the traffic on to squid, which in turn passes it on to the internet if not served from the local cache.

Some outstanding issues is dealing effectively with bittorrent and peer to peer traffic, without having to create countless manual rules. For now the ntop reports give clear usage by source, target, protocol, etc. It is relatively easy to identify bandwith hogs and deal with them, but it takes time and is a re-active process rather than being pro-active.

Next step is to automate the creation of iptable firewall rules on the fly to filter traffic as it is identified as inappropriate. Another todo is to add is to scan mail for spam and virusses.