Dell 2650/PERC 3/Di with kernel > 2.6.22 and XFS

As it took me a day to find out I wanted to post my findings here too.

I got a used Dell Poweredge 2650 and (as usual) installed Gentoo on it. First I got a faulty harddisk in the RAID5 and rebuilding took like 6 hours.
So I didn’t mind slow io performance which I accounted to the rebuild in process.
Unfortunately it still didn’t get better when the rebuild was finished. Taking seconds for a simple “ls”, installing gentoo-sources took more than a hour and the like. I did all firmware bios updates until none were available anymore. Still, no dice.
Searching around the Web I stumbled about this post and this post (from the same author) which are pointing to issues with the most recent aacraid driver but no relation to XFS yet.
Nearly convinced to downgrade the kernel or at least the aacraid driver I did a search in the gentoo forums and finally found the solution.
Mounting the XFS filesystems with nobarrier brought the speed back to normal. Personally I would have never thought of that solution but it seems like the newer aacraid doesn’t report back that write-barrier is a bad idea on the PERC 3/Di.

Now up for the task to try to get OpenManage running on gentoo … lets try if and exotic approach helps.

Nginx, finally!

Seeing the notice that the license on my Litespeed webserver is expiring again (yearly payments ) I finally started to move my sites to nginx (together with a move in datacenters so that webserver configuration was to be done anyway).
There were some more webservers in the run but I ended up with nginx.
Some others, lighttpd (got a bit silent over there and I don’t want to put my sites on a dying project), cherokee (now even with a webinterface!, but documentation is a bit sparse and the latest release seems inconsistent with the configuration – I simply couldn’t find out to do what I wanted to do) and the original Litespeed webserver.
In the end I wanted to come back to an open source webserver which doesn’t lock me in like that.
LSWS had some regressions in the last versions and one always has to wait for the developer team to fix them (even though they are quick) as no one else can dig into the code and also no one can write modules or enhancements because of the closed source.
Also there were some features which are now only available in the enterprise (aka paid) version which I don’t want to be forced to use forever. Also in the last year(s) its simply more directed to hosting companies or similar which are using native httpd.conf files and not doing the configuration in the webinterface they are offering. Some features are even only working with using httpd.conf entries.
Oh and the free version doesn’t offer x86-64 versions therefore I needed compat libs.
Therefore better do the cut now and use something else.
Nginx has the fastcgi loadbalancing I want, rewrite rules, great configurability and a very active community (and developers).
The only thing I’m really missing there is the possibility to use .htaccess files which forced me to search for the .htaccess files and turn their rules into native nginx configuration entries. Oh, one feature I forgot, reloading the configuration without doing a full restart of the webserver is neat too .
All issues I had could be quickly solved by either searching the maillist archive or posting there.

Don’t get me wrong. I still recommend LSWS to users who want to have an easy to use webserver with great performance as a drop in replacement for Apache supporting most of the previous features out of the box but its simply not for me anymore.

The next steps for Phorum

Now that Phorum-5.2 has finally gone stable there will be hopefully some better modules as the possibilities have been vastly increased. One of the new modules for 5.2 which show quite some of the abilities is the rewritten user-avatar module for 5.2.
With modules you can use now (not everything is new in 5.2!!!):

• use a supported API for files, users, custom profile fields and similar stuff
• ability to hook the module-css into the css loaded by phorum for valid (x)html pages and not loading it separately (saving requests)
• ability to hook the module-javascript into the javascript loaded by phorum for … see above
  (both can use raw files, templates, functions for including it)
• can do database calls without writing database dependent code (could still be because of the queries themselves)
• use module-templates which are included in the module itself, no need to copy them to the template folder(s)
• language files in the modules themselfes
• adding controlcenter panels without copying files around

Also our module list for 5.2 is now auto-generated from the modules posted into the 5.2-modules forum in the right format.
Make sure to add categories too as listed in the docs!

So, now that we (could) have better modules, whats next?

Dan Langille has been working on a postgresql-layer for 5.2/5.3 which will probably be included in one of the later 5.2 release as a beta of this layer.
The next big release will be Phorum-5.3. Our plan for Phorum-5.3 is “just” to add even more APIs, changing large parts of the backend without touching much of the frontend code.
Therefore templates from 5.2 should work without a hitch with 5.3. Maybe there will be added features missing in the old template but otherwise it will continue to work as before.
I know we made it hard for some admins with the switch from 5.0 to 5.1 and 5.1 to 5.2 but all these changes were done for flexibility in the templates and making them far more consistent and therefore easier to implement.
Some of the APIs will be about forum handling and similar stuff so that you can build a new admin or an admin in another page far easier than before.
As usual you can see the tickets on the table for 5.3 in our ticket-list (from the 5.3-milestone).

And further in the future?
There is lots and lots of stuff in the ever growing Ideas-milestone.
We’ll see if any of this will see the light in 5.3 already or in a later release but we surely won’t get bored .
I’m pretty sure that lots of stuff will be done at the MySQL Conference 2008 like last year where we’ve been coding and presenting there with lots of feature tickets closed for 5.2. You can help us to get there with donating to phorum.org!

laws and the use of logging IPs

in the light of recent court-decisions in germany ( german article ) which essentially disallows logging of IPs I’m wondering what one would really need it for?

I’m using IP-logging/-tracking in multiple ways:
1. statistics about visits and recurring users
2. storing it with forum-posts to allow law enforcement in case some user really goes over the line
3. tracking requests in a given time by IP to automatically block potential attacks

So what of that could be avoided?

For 1. , one could just ignore logging the ip but trying to count visits and recurring users would be impossible with that. What now? Maybe logging a md5/shaX of the ip to have some unique key per IP? Wouldn’t that still fall under the rule from the court as you could find out which was the actual IP?
Counting visits is an important tool for getting advertisers to advertise at a page (In my opinion). Any ideas?

For 2. , guess one could disable that but would I be responsible then for each and every forum-post because the real poster can’t be retrieved? (Yeah, laws in german are bad for the one offering the forum after all )
On the other hand there is the upcoming data retention ( german news collection about this topic ) which is planned for keeping all records for 6 months (!!!). So for now I should remove all tracking of ip-addresses just to be forced to store it for 6 months a while later?

For 3. , this behaviour gives me another problem too. Trying to load-balance over multiple webservers usually goes through a reverse proxy in front of the webservers which would always give the REMOTE_ADDR of the reverse-proxy to the apps. So the reverse-proxy would need to add this security layer. But I really failed to find one doing this up to know.
But is that really needed and I’m just oversensitive in this area? Do I need to accept any number of requests/s from any user?

Are there other use-cases for logging IPs?

How are other users handling this?

The editor of choice …

… yeah, everyone got his own idea of which editor he should or would use – thats the freedom of choice .
Brian loves his jedit, Maurice uses his VI (and can’t live wout vi-bindings and -code-folding) and I, I’m just going with the masses .
Currently I’m using Eclipse/PDT, coming right from the Zend IDE/Studio.
There were quite too many bugs in the current Zend Studio which I couldn’t live with (no, I don’t want to restart the IDE every half hour just because it forgets to show the content of the files) and PDT was just on its way to get to a final 1.0 so I used it.
Coming from Zend Studio its easy to use and for missing features in the IDE you can simply install some eclipse-extension – thats the power of using a generic IDE.
One thing I’m missing in PDT in relation to the Zend Studio is the line wrapping. There simply is NONE in Eclipse. Guess it was to teach coders to write 80cols code .
But for now I HAVE code which is far longer than 80 or even 160 cols and I don’t want to scroll around or reformat if I’m looking at a longish condition.

I also tried jEdit, Kommodo or the likes. I for one really want that project handling with function lists for the project, the possibility to just select a function and jump to its definition, having the comment of the function shown in a tooltip when using/typing it. Thats what I expect from an IDE.
Yeah, I know. These huge java apps can get slow sometimes but at least we got something to use our CPU’s for, eh?

A new post … REALLY!

Ok, I agree that it went a little bit silent in the last weeks but that was just because of an exam I had to do and which I really had a lot to learn for.
Now that one is done and I only got to finish (or at least start ) my diploma thesis to bring it to an end.

Lets see if I can get some life back into this blog.

whats up with lighttpd?

Is it just me or has development on lighttpd slowed down in the last months?
Last commit July 25th ( trac-timeline ) which was for a release
which opened a couple of problems ( http://www.lighttpd.net/2007/7/24/1-4-16-let-s-ship-it, blog-entry ) and none of them are fixed until now.
Also there are tickets which are (at least for me) showstopper bugs like the mod_extforward breaking url-matches which is open for 3 months now and got no comment by a developer.
When I read the page from the original author I see lots of comments about mysql-proxy and I just hope that its a temporary thing instead of him switching his attention completely to something different.
On the other hand – modlogan died silently when Jan started developing lighttpd .

Lighttpd got nice attention as is even mentioned by netcraft in their webserver-statistics but it should still be actively maintained and all open bugs which are not feature enhancements should be fixed as soon as possible – if there are enough developers on the project left to do this.

finally it has arrived: Phorum-5.2.0-alpha

It took quite some time but finally we made it and released phorum-5.2.0-alpha.
Brian already posted most of the new features in his blog .

One of the new things that don’t show up in the phorum-code itself are the revamped docs.
These are written in docbook-xml and are available (rebuild from trunk every hour) on phorum.org
as html-docs
and pdf: admin.pdf , developer.pdf , faq.pdf and the still empty user.pdf .
Therefore I renew my plea for help in this area.
It would be really great if you could help us to improve the docs. Every little thing helps.
Send us questions (and answers) for faq-items, texts for the user-manual and so on. Just send it in as plain text, we will convert it to docbook if you don’t want to mess with it.
Email-address for all docs-stuff is documentation@phorum.org (which will reply you at your first mail with a confirmation required).
If you want to play directly with the docs-source just checkout the trunk-tree as described in the wiki and look at docs/docbook in there.

Oh and before I forget: remember its alpha-quality. Don’t use it in production yet!

Thats one PHP-5.2.x feature for Phorum-5.2 I’d like to use …

Its the httpOnly Cookies support in the setcookie-call.
Now that Firefox 2.0.0.5 supports it too (as mentioned here and here)
the main browsers are supporting it.
Internet Explorer seems to have been the first one supporting it, with Firefox now and Opera meant to support it in 9.5.

Therefore it makes sense to use it now. Even browsers not supporting it are just ignoring the additional flag.
In PHP that flag was introduced in PHP-5.2 – another cause for going php-5.2 and up only .

The new db-layer in Phorum-5.2 kicks ass

Thanks for maurice’s pulling me into the db-layer I added my “own” mysql-mysnip-layer which handles the queries done.
The change from 5.1 to 5.2 brought up a split db-layer with one file containing all the queries for mysql and calling a function for actually running the query and (optionally)
returning the rows.
And that second part is simply extension specific like for mysql and mysqli extensions. I think it would be simple to add another for pdo but thats a different topic.
For now I added another “extension”-specific part for mysnip which looks into the queries to check if they need rewriting for the partitions used.
The partition-specific tables have some marker in there which tells where the partition number should be and that marker is replaced with the partition-id on querying.
If the partition-id is not yet available in for the current forum-id its retrieved from the database.

That functionality allows me finally to run through all the upgrades as we’ve added a flag to the queries which need to be run for each partition.
We stumbled about it when I was wondering how real-name upgrades are handled now as these have to run through all partitions where that user could have been posting to.
The upgrades from my early version to current 5.2 take a while but overall they are running fine.
Another nice thing is that I don’t have to hack the actual layer which contains the queries and can take that part from the distributed code, only the “extension-specific” code changed and is in a separate file. No more relying on a module to run and rewrite the table-names which would only run once on a page-view.

To cut a long story short: its great and should allow you all kinds of changes to the db-layers without touching the queries. It should even be much easier to write a layer for another database-system (anyone volunteering? ).