building a HA/LB solution
I’m currently in the process of trying to build a HA/LB solution for my forums.
Currently HA (HighAvailability) is created by running heartbeat on the two webserver-”nodes” with automatic ip-takeover and a mysql-slave which gets all the data from the main-db-server (but needs manual takeover).
LB (LoadBalancing) is done with FastCGI-Loadbalancing in LiteSpeed-Webserver but I’m not satisfied with the results as it seems that the first host is getting much more load than the second one.
Therefore I played with some Virtual Machines, one running haproxy ( http://haproxy.1wt.eu/ ), two running lighttpd with fcgi-php.
So far it worked good but taking down one of the webservers still gave me some failed requests if it was running under “siege”. Thats something I wanted to avoid.
Lighttpd was simply choosen because of mod_extforward so that I could keep the original hosts ip in the REMOTE_ADDR and its support for fcgi-php.
But as I wrote in an earlier post there is one feature I’d badly miss in lighttpd and which really keeps me from switching:
.htaccess-support or generally spoken: dynamic configuration changes without changing the main-configuration and the need for a webserver reload.
I found one thread in the lighttpd-forums which sounds promising.
Reading dynamic configuration from mysql is something I’d love to see. It would kick ass :).
Yeah, sure. Lighttpd would have to work without mysql-connection too, some fallback mechanism needs to be in place but that would solve at least most of my problems.
For my own DoS functionality I need a way to block connections on the webserver-level before it even reaches PHP.
So there are some problems or lets better call it “tasks” left to solve for my HA/LB solution:
- find the right webserver to implement that
- build a solution to merge the logs and process them for statistics
- find out how to get haproxy (or another loadbalancing solution) to send failed requests to another backend in case of one going down
And the big task:
- find some automatic solution for mysql-takeover (without DRBD, which I don’t trust because of its network-based nature ;))
Any ideas anyone?
Sure.
http://www.f5.com/products/bigip/ltm/
=)
Nice try. It won’t solve the mysql-takeover problem nor any of the other issues I’m having I guess ;).
Have you looked at mod_magnet in much detail? It allows scripting functionality much more involved than htaccess. With a small tweak, I was able to feed it remote IPs in order to write a htaccess-style ip-blocking script. Admittedly, if lighttpd doesn’t yet feed the data you want to mod_magnet, you will have to tweak the lighttpd C source code to do so, but it was very easy to tweak in my case.
Hmm, that sounds like an interesting option.
Did I read it right, that a “mod-magnet-script” (aka lua) is dynamic and lighttpd rereads it on changes?
then it only needs your change to do what I want (and more :)).
Too bad that the mod_extforward bug is not fixed yet either.
Thanks for the hint!
Yep, it’s just lua so it’s compiled to bytecode and only reloaded if the lua/mod_magnet script changes. Glad to hear my small IP tweak is useful.