Merely as a note to myself I just looked around the Gentoo bugtracker to learn about the current state of MySQL in Gentoo.
So far I found two related bugs:
About 5.0.x
http://bugs.gentoo.org/show_bug.cgi?id=279493

What I learned from this bug:
Recent dev-db/mysql versions contain most of the percona patchsets already (neat!). Dunno about xtradb so far.
Latest in tree is 5.0.84 which I’m trying on a backup system now (even though is marked as ~x86/~amd64 aka testing.

About 5.1/5.4
http://bugs.gentoo.org/show_bug.cgi?id=194561
5.1 is going to be put into dev-db/mysql too (not in dev-db/mysql-community as it was before because of the changed development model)
Quoting Robin Johnson:

“I intend to issue a package move after we’ve had a few versions >=5.0.83 in stable, but there is no further need to make new -community ebuilds.”

So there will be 5.1.x in tree once some more recent 5.0.x versions were released as stable. Latest stable mysql version in tree is 5.0.70.

Seems like he’s also keeping back because of some more breakage in earlier 5.1 versions.
As another quote:

“I’m aware that 5.1.30 is out. However it’s still in bad bad shape. [...] It certainly ate some of my data when I tested it.”

For the topic whats keeping them from stabilizing later mysql (> 5.0.70) versions I found some quote from Robin Johnson too (who seems to be THE mysql maintainer in Gentoo):

My most defining test for putting MySQL builds in the tree has been that it
passes both of the following:
1. Passes it’s own testcases (upstream has been atrociously bad at this, see
status2 in 5.0.72 for example)
2. It doesn’t eat my data or break my systems.

#1 is pretty easy as a start point, seeing if it works.
#2 is a lot tougher:
- 5.0.70 is the best option for now.
- 5.0.72 breaks most of the statistics code out there really badly (I filed
upstream bug 41131) – changes to SHOW behaviour as well as the ‘Questions’
variable.
[...]
Having upstream do sane releases is part of why there has been so long between
my 5.0.x bumps, because they haven’t passed my personal testing.

So far its mostly general stuff and I don’t know if MySQL-5.0.x still doesn’t pass its own testcases or anything. I couldn’t find more detailed stuff in the bugtracker.

Long time no post but thats some stuff lingering in my head for a while looking at the mysql ecosystem lately.

For a long time I had to stay with MySQL-4.0 (sick I know) but there’s a lot of software to adapt to the new version(s) but lately I’m pushing it more and more into mysql 5.0.
MySQL 5.1 would be also interesting and even MySQL-5.4 … but thats where the trouble starts.
MySQL-5.4 came out of the dark, no one expected it and it gave numerous improvements … though its still in beta.
With that release (or at least around this time) MySQL started to change its release model to something new where the version numbers matter far less and where there should be regularly released. Well, Oracle has bought Sun which owns MySQL … so we will see what the next “release model” will be.

On the other hand there are some “forks” of mysql out there which offer more improvements or at least they are supposed to do this.
For one there is XtraDB which is supposed to be just a replacement for the InnoDB-plugin now … while it had more patches to the main mysql server before as far as I remember – so its not really a fork, just another storage engine.
Edit: just found a newer release – its called “MySQL with Percona patches”

Then there is MariaDB which should be a “… community developed branch of the MySQL database that uses the Maria engine by default.” (quote from the linked page) which is developed by Monty Program AB and the OpenDatabaseAlliance.

Being “only” a collection of patches and builds of MySQL with patches is OurDelta.
I’m pretty sure that there are more forks or patch collections out there, please forgive me if yours isn’t listed.

But here is the question. Which MySQL version / patchset / fork should one use?
Previously it was just the question of using the commercial enterprise or the community version. Now I’m just confused.

Next problem is the distribution on linux …
Nearly all my servers are using Gentoo as the linux distribution but have you ever looked at the status of mysql in there?
The latest “stable” version is dev-db/mysql-5.0.70. dev-db/mysql-community-5.1.21_beta is in there, but not marked as stable (5.1.39 being the last one on the mysql homepage), 5.4.x is nowhere to be seen, same for XtraDB, MariaDB … . I don’t blame it on the maintainers – who should keep up with that flow of releases, different branches and/or forks. I also heard rumours that all the latest versions are failing numerous tests for the maintainer(s) and therefore won’t go in.
If I want to use a recent stable version or one of the enhancements I will have to do my own builds instead of using the great gentoo infrastructure for that. I could, sure, but time is low and I would get far further if I could use the regular way to install (and update) software on Gentoo. I don’t have a full blown infrastructure team to keep the systems going.

If I wouldn’t have too many software using raw database calls in php or perl I would seriously consider switching to some other database, PostgreSQL or the like. I heard even Maurice considering switching to Postgresql in the future and supporting the Postgresql layer for Phorum.

If there is no one stopping this proliferation of the mysql ecosystem and providing reliable and regular stable builds of a tree, I think Oracle won’t have to worry about MySQL anymore. There will be not too much left of its userbase and community.
But maybe its just me, painting things black, who knows .

As it took me a day to find out I wanted to post my findings here too.

I got a used Dell Poweredge 2650 and (as usual) installed Gentoo on it. First I got a faulty harddisk in the RAID5 and rebuilding took like 6 hours.
So I didn’t mind slow io performance which I accounted to the rebuild in process.
Unfortunately it still didn’t get better when the rebuild was finished. Taking seconds for a simple “ls”, installing gentoo-sources took more than a hour and the like. I did all firmware bios updates until none were available anymore. Still, no dice.
Searching around the Web I stumbled about this post and this post (from the same author) which are pointing to issues with the most recent aacraid driver but no relation to XFS yet.
Nearly convinced to downgrade the kernel or at least the aacraid driver I did a search in the gentoo forums and finally found the solution.
Mounting the XFS filesystems with nobarrier brought the speed back to normal. Personally I would have never thought of that solution but it seems like the newer aacraid doesn’t report back that write-barrier is a bad idea on the PERC 3/Di.

Now up for the task to try to get OpenManage running on gentoo … lets try if and exotic approach helps.

Seeing the notice that the license on my Litespeed webserver is expiring again (yearly payments ) I finally started to move my sites to nginx (together with a move in datacenters so that webserver configuration was to be done anyway).
There were some more webservers in the run but I ended up with nginx.
Some others, lighttpd (got a bit silent over there and I don’t want to put my sites on a dying project), cherokee (now even with a webinterface!, but documentation is a bit sparse and the latest release seems inconsistent with the configuration – I simply couldn’t find out to do what I wanted to do) and the original Litespeed webserver.
In the end I wanted to come back to an open source webserver which doesn’t lock me in like that.
LSWS had some regressions in the last versions and one always has to wait for the developer team to fix them (even though they are quick) as no one else can dig into the code and also no one can write modules or enhancements because of the closed source.
Also there were some features which are now only available in the enterprise (aka paid) version which I don’t want to be forced to use forever. Also in the last year(s) its simply more directed to hosting companies or similar which are using native httpd.conf files and not doing the configuration in the webinterface they are offering. Some features are even only working with using httpd.conf entries.
Oh and the free version doesn’t offer x86-64 versions therefore I needed compat libs.
Therefore better do the cut now and use something else.
Nginx has the fastcgi loadbalancing I want, rewrite rules, great configurability and a very active community (and developers).
The only thing I’m really missing there is the possibility to use .htaccess files which forced me to search for the .htaccess files and turn their rules into native nginx configuration entries. Oh, one feature I forgot, reloading the configuration without doing a full restart of the webserver is neat too .
All issues I had could be quickly solved by either searching the maillist archive or posting there.

Don’t get me wrong. I still recommend LSWS to users who want to have an easy to use webserver with great performance as a drop in replacement for Apache supporting most of the previous features out of the box but its simply not for me anymore.

Now that Phorum-5.2 has finally gone stable there will be hopefully some better modules as the possibilities have been vastly increased. One of the new modules for 5.2 which show quite some of the abilities is the rewritten user-avatar module for 5.2.
With modules you can use now (not everything is new in 5.2!!!):

• use a supported API for files, users, custom profile fields and similar stuff
• ability to hook the module-css into the css loaded by phorum for valid (x)html pages and not loading it separately (saving requests)
• ability to hook the module-javascript into the javascript loaded by phorum for … see above
  (both can use raw files, templates, functions for including it)
• can do database calls without writing database dependent code (could still be because of the queries themselves)
• use module-templates which are included in the module itself, no need to copy them to the template folder(s)
• language files in the modules themselfes
• adding controlcenter panels without copying files around

Also our module list for 5.2 is now auto-generated from the modules posted into the 5.2-modules forum in the right format.
Make sure to add categories too as listed in the docs!

So, now that we (could) have better modules, whats next?

Dan Langille has been working on a postgresql-layer for 5.2/5.3 which will probably be included in one of the later 5.2 release as a beta of this layer.
The next big release will be Phorum-5.3. Our plan for Phorum-5.3 is “just” to add even more APIs, changing large parts of the backend without touching much of the frontend code.
Therefore templates from 5.2 should work without a hitch with 5.3. Maybe there will be added features missing in the old template but otherwise it will continue to work as before.
I know we made it hard for some admins with the switch from 5.0 to 5.1 and 5.1 to 5.2 but all these changes were done for flexibility in the templates and making them far more consistent and therefore easier to implement.
Some of the APIs will be about forum handling and similar stuff so that you can build a new admin or an admin in another page far easier than before.
As usual you can see the tickets on the table for 5.3 in our ticket-list (from the 5.3-milestone).

And further in the future?
There is lots and lots of stuff in the ever growing Ideas-milestone.
We’ll see if any of this will see the light in 5.3 already or in a later release but we surely won’t get bored .
I’m pretty sure that lots of stuff will be done at the MySQL Conference 2008 like last year where we’ve been coding and presenting there with lots of feature tickets closed for 5.2. You can help us to get there with donating to phorum.org!

in the light of recent court-decisions in germany ( german article ) which essentially disallows logging of IPs I’m wondering what one would really need it for?

I’m using IP-logging/-tracking in multiple ways:
1. statistics about visits and recurring users
2. storing it with forum-posts to allow law enforcement in case some user really goes over the line
3. tracking requests in a given time by IP to automatically block potential attacks

So what of that could be avoided?

For 1. , one could just ignore logging the ip but trying to count visits and recurring users would be impossible with that. What now? Maybe logging a md5/shaX of the ip to have some unique key per IP? Wouldn’t that still fall under the rule from the court as you could find out which was the actual IP?
Counting visits is an important tool for getting advertisers to advertise at a page (In my opinion). Any ideas?

For 2. , guess one could disable that but would I be responsible then for each and every forum-post because the real poster can’t be retrieved? (Yeah, laws in german are bad for the one offering the forum after all )
On the other hand there is the upcoming data retention ( german news collection about this topic ) which is planned for keeping all records for 6 months (!!!). So for now I should remove all tracking of ip-addresses just to be forced to store it for 6 months a while later?

For 3. , this behaviour gives me another problem too. Trying to load-balance over multiple webservers usually goes through a reverse proxy in front of the webservers which would always give the REMOTE_ADDR of the reverse-proxy to the apps. So the reverse-proxy would need to add this security layer. But I really failed to find one doing this up to know.
But is that really needed and I’m just oversensitive in this area? Do I need to accept any number of requests/s from any user?

Are there other use-cases for logging IPs?

How are other users handling this?

… yeah, everyone got his own idea of which editor he should or would use – thats the freedom of choice .
Brian loves his jedit, Maurice uses his VI (and can’t live wout vi-bindings and -code-folding) and I, I’m just going with the masses .
Currently I’m using Eclipse/PDT, coming right from the Zend IDE/Studio.
There were quite too many bugs in the current Zend Studio which I couldn’t live with (no, I don’t want to restart the IDE every half hour just because it forgets to show the content of the files) and PDT was just on its way to get to a final 1.0 so I used it.
Coming from Zend Studio its easy to use and for missing features in the IDE you can simply install some eclipse-extension – thats the power of using a generic IDE.
One thing I’m missing in PDT in relation to the Zend Studio is the line wrapping. There simply is NONE in Eclipse. Guess it was to teach coders to write 80cols code .
But for now I HAVE code which is far longer than 80 or even 160 cols and I don’t want to scroll around or reformat if I’m looking at a longish condition.

I also tried jEdit, Kommodo or the likes. I for one really want that project handling with function lists for the project, the possibility to just select a function and jump to its definition, having the comment of the function shown in a tooltip when using/typing it. Thats what I expect from an IDE.
Yeah, I know. These huge java apps can get slow sometimes but at least we got something to use our CPU’s for, eh?

Ok, I agree that it went a little bit silent in the last weeks but that was just because of an exam I had to do and which I really had a lot to learn for.
Now that one is done and I only got to finish (or at least start ) my diploma thesis to bring it to an end.

Lets see if I can get some life back into this blog.

Is it just me or has development on lighttpd slowed down in the last months?
Last commit July 25th ( trac-timeline ) which was for a release
which opened a couple of problems ( http://www.lighttpd.net/2007/7/24/1-4-16-let-s-ship-it, blog-entry ) and none of them are fixed until now.
Also there are tickets which are (at least for me) showstopper bugs like the mod_extforward breaking url-matches which is open for 3 months now and got no comment by a developer.
When I read the page from the original author I see lots of comments about mysql-proxy and I just hope that its a temporary thing instead of him switching his attention completely to something different.
On the other hand – modlogan died silently when Jan started developing lighttpd .

Lighttpd got nice attention as is even mentioned by netcraft in their webserver-statistics but it should still be actively maintained and all open bugs which are not feature enhancements should be fixed as soon as possible – if there are enough developers on the project left to do this.

It took quite some time but finally we made it and released phorum-5.2.0-alpha.
Brian already posted most of the new features in his blog .

One of the new things that don’t show up in the phorum-code itself are the revamped docs.
These are written in docbook-xml and are available (rebuild from trunk every hour) on phorum.org
as html-docs
and pdf: admin.pdf , developer.pdf , faq.pdf and the still empty user.pdf .
Therefore I renew my plea for help in this area.
It would be really great if you could help us to improve the docs. Every little thing helps.
Send us questions (and answers) for faq-items, texts for the user-manual and so on. Just send it in as plain text, we will convert it to docbook if you don’t want to mess with it.
Email-address for all docs-stuff is documentation@phorum.org (which will reply you at your first mail with a confirmation required).
If you want to play directly with the docs-source just checkout the trunk-tree as described in the wiki and look at docs/docbook in there.

Oh and before I forget: remember its alpha-quality. Don’t use it in production yet!