
Here we are again with another episode of "Running on OpenBSD" that describes how we configured a system with multiple gateway interfaces, allowing equal distribution of connections among them. The system was asked by an Internet Cafe with multiple ADSL connections that needed a better solution than the one already in place.
We will describe the requirements and setup of the system we finally installed along with some notes on the ideas behind each decision. As always the system was OpenBSD and the provided ports.
Our story starts in Greece where the price of leased lines is still prohibiting for small size businesses and the only means of getting a decent speed is by using ADSL connections. The Internet Cafe that we asked to help was using four ADSL connections, shared among 24 computers that needed to access the internet.
Each of those computers was logically segmented into groups of 6 computers. Each group was manually configured to use one of the four ADSL connections. However, this lead to a couple of obvious problems. A single group could utilize the entire internet connection while the rest of the ADSL connections was unused.
Furthermore, whenever one of the ADSL connections was having connectivity problems an entire group of computers was also unable to use the internet connection. This usually required to manually reconfigure the said systems to use one of the other available connections.
The internet usage was also another factor that needed to be considered. The users often times visited bandwidth consuming sites such as, YouTube, Google video and other similar services that utilised this form of streaming video and not only that, but as we all know most of the time were full of banners that delayed the overall loading time and caused pains to all the other systems that used the internet connection.
The requirements were straight forward for us but there were some extra things that we needed to take into consideration before we moved into the implementation of a solution that we would suggest.
We suggested an OpenBSD 4.2 system with 5 total Ethernet interfaces. Four of them used to balance the bandwidth utilisation with the external world and one to keep the system connected with the internal network. The OpenBSD PF took care of the balancing of the requests through different interfaces each time with a very simple round-robin route-to syntax. PF also redirected transparently all requests for websites through a squid proxy server which was installed from the ports.
Squid's configuration was minimalistic apart from some parts that were required to allow caching of special content such as:
A small Perl script was used along with Squid in order to filter the remaining of the advertisements that were hiding inside URL's and weren't block by the hosts blacklist.
We used the default ISC Bind that ships with OpenBSD to cache and provide name resolution services for the local machines. Used a hosts blacklist to avoid spending our time with bandwidth wasting, advertising hosts.
For network usage and utilization statistics we took advantage of PF's internal functions again and with the help of pfstat we were able to produce nice looking graphs representing all the required parameters. We used three different generation policies, one every hour, day and month.
A comparison of the two network topologies makes the benefits in simplicity obvius.
Topology before
and after 
The simplicity of PF made it a clear winner. Within minutes our gateway was able to distribute the requests among the 4 interfaces. Our configuration included mostly bits from the PF FAQ.
For the Squid configuration part you can find more of the configuration examples at Caching YouTube Content.
The whole project was completed in a matter of a couple of hours at which point we were enjoying our pizza and fast internet.
I hope you enjoyed the reading, 'till next time "Stop slacking you lazy bum".
PS: The system is now in place for almost 6 months. There have been no problems whatsoever related with the system. Even the owners, which come from a Microsoft Windows background, are impressed by the fact they have one less system to worry about.
Comments
Hi! This is great stuff.
Hi!
This is great stuff. I'm currently setting up an openbsd firewall with 2 external gateways (2 DSL lines) and I need some ideas on altq queueing like what you have. Could you post up your pf.conf or email it to me? (Feel free to mask the sensitive bits.) Thanks! :)
-T
Hi, Unfortunately we didn't
Hi,
Unfortunately we didn't use altq for this setup. We only used route-to and some specific rules in order for the daemons, which run on the local gateway, to be able to use specific gateways. For instance we wanted only one of the four ADSL lines (the one with lower bandwidth) to be used for DNS and Web queries.
Nevertheless, if you still need the pf.conf for some hints feel free to let me know.
Pantelis
___
Dont prohibit what you cant prevent.
I'm just curious. Did you
I'm just curious. Did you use tables to hold the external gateways? I found I had to use:
pass in on { $int_if lo0 } route-to { ($ext_if1 ), ($ext_if2 ) } round-robin proto tcp from ($int_if:network) to any flags S/SA modulate state
Then I had a pfctl --t extgw1 -T replace 0.0.0.1 in my /etc/hostname.pppoe0 (since that's handled by the kernel)
The PPP was trickier. I needed /etc/ppp/ppp.linkup to do:
isp:
!bg /sbin/pfctl -t extgw2 -T replace HISADDR
Since both gateways are dynamic and can change if the DSL line drops, this saves lots of headaches.
The only problem now is.. if one of the lines goes down out of the two, 50% of the traffic goes into a blackhole. Do you have any scripting to detect when a line fails over and change the route-to statements accordingly?
Hi, We did use tables but
Hi,
We did use tables but not for this purpose. Our gateways were static so we had no need for tables at that time.
Dont prohibit what you cant prevent.
Happy that you are back
Hi Peter, Thanks for you
Hi Peter,
Thanks for you comment, it's good to be back and what's more to be welcomed back. I have quite a lot of episodes yet to complete, although tight schedule makes it very hard.
However, I'll do my best to keep them coming.
___
Don't prohibit what you can prevent.
Wow that is a great system.
Post new comment