Re: Forum Software Benchmarks: http://bb-bench.com
Um, wow. I’ll admit I’m quite astounded and I’m not quite sure what to say, but I’ll try.
Actually the PHP version kinda proved my point about content caching.
It’s taking 3-4 ms.
The front page index on most of those forums is taking 20-25ms
Uh… no they aren’t. http://bb-bench.com/benchmark/1#section_p_index
That’s impossibly fast, it definitely means content is not being regenerated. If the cache was defeated it would have to re-render the whole thing and the page time would be significantly higher.
No, it’s not impossibly fast. I have not perused the code of most systems, but I can guarantee phpBB does not cache the output of its pages. They are retrieved every page load. Sure, there is a caching system in place, but it doesn’t catch the output of a page, only stores a few variables that don’t even improve performance particularly. As for caching at the apache level, you would see a much bigger difference in speed than you do see. Without special configuration, apache should never cache the output of a php page, since that would entirely defeat the purpose of a dynamically generated page.
Caching page output is near impossible for forums in general, due to forum permissions and session specific information. It would be kind of bad if my “One private message unread” were cached for you. The only other form of caching is the sql cache, which is contained entirely within MySQL, and is automatically activated for all queries, regardless of software.
bbPress doesn’t even save the tag cloud between pages, it will re-render it each and every time, which is at least 1/4th of the page render time (that can be changed via a plugin).
Sounds like that’s a big area for bbPress to improve. But it still doesn’t make the result invalid: bbPress is slower.
So 265ms vs 78ms
bbPress 0.9 is “only” 187ms slower than statically served PHP.
Do your math again. It’s 260ms slower than a static page. It’s still damn slow, and if someone installs bbpress and *pick another board* side by side, the bbpress one will not perform better. While technically correct, it does nothing to affect the overall results.
@ckwalsh, Your benchmarks are incorrect this way. _ck_ is right about it
Can you please paraphrase what his arguments are, and perhaps add a little bit more? I’m getting the feeling that you posted here to support _ck_ since the results I found were unfavorable to bbPress, and you don’t have anything to add, or even understand the discussion. Not to say I only want to talk to experts, but blind bandwagonning always frustrates me on the internet.
Basically I’d want to see a simulated load with logged in users being served different content – ie. unread posts for different users
Certainly apache bench isn’t perfect, however, it does show the relative performance of those 3 pages, which are almost guaranteed to be responsible for the majority of requests to a forum. I have been considering how to build a better tool, but have not gotten a chance to do so yet.
But plugins can still do content caching themselves. For example I realized awhile back that the Hot Tag cloud that bbPress renders is taking at least 1/4th of the total page render time, and it does it each and every time, regardless if there are new tags added or deleted. So my Hot Tags Plus plugin caches the tag cloud as static html (as well as add many other features to it at the same time).
If you think that will drop load time, it seems to be something anyone can add – I’d certainly be willing to install it for my tests.
Overall, you seem to be upset that bbPress performed so badly, leading you to say the benchmark is wrong. While it isn’t perfect, what it does show it shows rather precisely. In the current state of forum software, bbPress does appear to have a long way to go. It doesn’t matter if it was rewritten. It doesn’t matter if it doesn’t have caching (if so, that is a failure of bbPress, not an unfair advantage to other software). When comparing the same functionality between software, those are the results, like them or not.
