building a HA/LB solution

I’m currently in the process of trying to build a HA/LB solution for my forums.
Currently HA (HighAvailability) is created by running heartbeat on the two webserver-“nodes” with automatic ip-takeover and a mysql-slave which gets all the data from the main-db-server (but needs manual takeover).
LB (LoadBalancing) is done with FastCGI-Loadbalancing in LiteSpeed-Webserver but I’m not satisfied with the results as it seems that the first host is getting much more load than the second one.

Therefore I played with some Virtual Machines, one running haproxy ( http://haproxy.1wt.eu/ ), two running lighttpd with fcgi-php.
So far it worked good but taking down one of the webservers still gave me some failed requests if it was running under “siege”. Thats something I wanted to avoid.
Lighttpd was simply choosen because of mod_extforward so that I could keep the original hosts ip in the REMOTE_ADDR and its support for fcgi-php.

But as I wrote in an earlier post there is one feature I’d badly miss in lighttpd and which really keeps me from switching:
.htaccess-support or generally spoken: dynamic configuration changes without changing the main-configuration and the need for a webserver reload.
I found one thread in the lighttpd-forums which sounds promising.
Reading dynamic configuration from mysql is something I’d love to see. It would kick ass :).
Yeah, sure. Lighttpd would have to work without mysql-connection too, some fallback mechanism needs to be in place but that would solve at least most of my problems.
For my own DoS functionality I need a way to block connections on the webserver-level before it even reaches PHP.

So there are some problems or lets better call it “tasks” left to solve for my HA/LB solution:
– find the right webserver to implement that
– build a solution to merge the logs and process them for statistics
– find out how to get haproxy (or another loadbalancing solution) to send failed requests to another backend in case of one going down

And the big task:
– find some automatic solution for mysql-takeover (without DRBD, which I don’t trust because of its network-based nature ;))
Any ideas anyone?

Advertisements

5 comments so far

  1. ts77 on

    Nice try. It won’t solve the mysql-takeover problem nor any of the other issues I’m having I guess ;).

  2. sprewell on

    Have you looked at mod_magnet in much detail? It allows scripting functionality much more involved than htaccess. With a small tweak, I was able to feed it remote IPs in order to write a htaccess-style ip-blocking script. Admittedly, if lighttpd doesn’t yet feed the data you want to mod_magnet, you will have to tweak the lighttpd C source code to do so, but it was very easy to tweak in my case.

  3. ts77 on

    Hmm, that sounds like an interesting option.
    Did I read it right, that a “mod-magnet-script” (aka lua) is dynamic and lighttpd rereads it on changes?
    then it only needs your change to do what I want (and more :)).
    Too bad that the mod_extforward bug is not fixed yet either.

    Thanks for the hint!

  4. sprewell on

    Yep, it’s just lua so it’s compiled to bytecode and only reloaded if the lua/mod_magnet script changes. Glad to hear my small IP tweak is useful.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: