I’m using the excellent bb-benchmark plugin, and here’s what I get on my front page when logged in as admin:

01:24:43 up 147 days, 2:25, 2 users, load average: 0.08, 0.07, 0.01

total page time: 0.325 seconds.

query count: 17

total query time: 0.0258 seconds

page rendering time: 0.2992 seconds

slowest call was # 1 : 0.0077 seconds

What I found most strange was that the rendering time is so much higher than I would expect. Even with a guest account and much less queries going on I see this in the footer:

0.219 – 8 queries

Am I wrong, or is that render time taking too long? I notice that many of the other pages render in 0.005 seconds or so.

[@_ck_ Participant 17 years, 4 months ago]

Depending on your host and how accurate their microtime is (for example I actually get negative numbers sometimes on Dreamhost?!!?) it’s fairly accurate but look at the page after a few tries and round out the numbers.

I have even more improvements in the newest bb-benchmach version (can’t remember if I put it into the SVN yet but I don’t think so). With a slight modification to bb-settings.php you can now get a detailed report like this:

=== benchmark & query results ===



load average: 1.64, 1.23, 1.14



total page time: 0.696 seconds.



time to reach each section:

bb_underscore_plugins_loaded =  0.412

bb-polls.php loaded =  0.421

bb-signatures.php loaded =  0.431

report-post.php loaded =  0.434

user-topics-to-favorites.php loaded =  0.436

tweaks.php loaded =  0.449

bb-topic-views.php loaded =  0.472

my-views.php loaded =  0.479

notification.php loaded =  0.481

bb-gzip.php loaded =  0.488

my-views-installed-available-plugins.php loaded =  0.491

my-views-started-participated-topics.php loaded =  0.493

my-views-most-least-views.php loaded =  0.495

plugin-browser.php loaded =  0.504

bb_plugins_loaded =  0.506

pluggable.php loaded =  0.523

locale.php loaded =  0.543

bb_init =  0.548

bb_index.php_pre_db =  0.603

bb_index.php =  0.616

front-page.php =  0.632

header.php =  0.655

logged-in.php =  0.675

footer.php =  0.694



time to render page: 0.6623 seconds (query time subtracted)



total query count: 11



total query time: 0.0337 seconds



slowest call was # 1 : 0.0169 seconds

SELECT meta_key, meta_value FROM bb_topicmeta WHERE topic_id = '0' server:dbh_local connect: 10.59ms



=== mysql queries used ===



# 1 : 0.0169 seconds

SELECT meta_key, meta_value FROM bb_topicmeta WHERE topic_id = '0' server:dbh_local connect: 10.59ms



etc. etc.

[@_ck_ Participant 17 years, 4 months ago]

Oh and bbPress’s (and WordPress’s) biggest weakness is not mysql queries – it’s actually all the plugins loading.

Unless you use an opcode cache or have really good disk caching on your host, loading a dozen or more plugins can cause those longer delays.

If you are on a VPS, you need eaccelerator or some other opcode cache. A few shared hosts like dreamhost let you install your own copy of php+fastcgi and eaccelerator too (but I do not recommend DreamHost, quite the opposite).

[@howtogeek Member 17 years, 4 months ago]

I’m running xcache opcode caching, and I have a dedicated virtual with 1GB of ram from Mediatemple. I am very committed to making sure the site loads instantly… the site in question is the one linked on my username here.

So loading the plugins causes the speed problems? I’ve tried running phped profiler and couldn’t find where the speed issue was.

Is it the actual parsing of the files, or loading in the functions into the filter/actions list?

I’m considering just concatenating all the plugins into a single plugin, just to see if that makes a difference.

[@_ck_ Participant 17 years, 4 months ago]

If you are running xcache, the plugins are not slowing you down, they get cached.

So “dedicated virtual” = VPS ? Interesting way to put it.

If so, you are at the mercy of the disk load on the rest of the node.

Your own average load rates have nothing to do with the rest of the server node, the VPS software tricks you.

(the big myth about VPS is that you are you are isolated from your neighbors, you’re not – any high disk use by them will slow you down)

However if you have 1gb guaranteed, then you have quite a huge VPS. If that’s 1gb burst, then it’s another story.

But if you have your own vps you can be using an opcode cache (xcache is very good) and then make sure you are using mysql cache.

Not much else can be done. You could in theory replace apache with LiteSpeed unless you are savvy enough to setup Lighttpd (which I myself won’t bother with)

[@_ck_ Participant 17 years, 4 months ago]

I’ve put 0.16 into the SVN, I think it updates on the hour.

Once you have 0.16 installed, go edit bb-settings.php in the bbpress root and around line 157 change that section so you add the line bb_benchmark_template_timer('',$plugin.' loaded');

like so:

if ( $plugins = bb_get_option( 'active_plugins' ) )

foreach ( (array) $plugins as $plugin ) {

if ( file_exists(BBPLUGINDIR . $plugin) ) {

require( BBPLUGINDIR . $plugin );

bb_benchmark_template_timer('',$plugin.' loaded');

} }

do_action( 'bb_plugins_loaded' );

unset($plugins, $plugin);

.

That will give you the super detailed output, so if there is a single plugin that is consistently slowing your page load down, you might be able to spot it.

[@howtogeek Member 17 years, 4 months ago]

This is interesting… because bbpress loads the plugins in order, the bb-benchmark plugin is getting loaded nearly last on my system, so I only see the output from the onlinelist plugin.

It looks like the vast majority of the execution time is before the plugins even get loaded.

[@howtogeek Member 17 years, 4 months ago]

After doing some more research, the plugin loading time is virtually instant when you an opcode cache.

I added separate timestamp logging which shows that all the plugins are loaded at 0.019 but the bb_plugins_loaded action doesn’t get fired until 0.1 seconds later.

bb-benchmark.php loaded = 0.019

bb_underscore_plugins_loaded = 0.019

bb_plugins_loaded = 0.102

bb_init = 0.103

bb_index.php_pre_db = 0.106

front-page.php = 0.106

header.php = 0.107

logged-in.php = 0.107

search-form.php = 0.116

footer.php = 0.302

The majority of the time is taken between search form and footer…. I’m not sure yet what gets executed between them.

[@howtogeek Member 17 years, 4 months ago]

I think I figured it out… I added my own timing mechanism to virtually every line of code on the front page, and noticed the extra 0.2 seconds was occurring at random places…

So I put ob_start(); at the beginning of bb-settings.php to turn on output buffering, and now here’s the times I get with like 16 plugins installed, logged in as admin with a ton of custom stuff going on and everything is opcached:

total page time: 0.078 seconds.

time to reach each section:

bb_underscore_plugins_loaded = 0.018

bb_plugins_loaded = 0.021

bb_init = 0.021

bb_index.php_pre_db = 0.024

front-page.php = 0.025

header.php = 0.025

logged-in.php = 0.026

search-form.php = 0.035

footer.php = 0.078

Now that is more like it! Now to remove all my logging….

And thanks for that benchmark plugin… it’s great, I can’t live without it now

[@_ck_ Participant 17 years, 4 months ago]

Note that microtime is off by at least 15ms on every call by nature, sometimes more depending what the cpu is doing and if you have a multiple core cpu. Sometimes the core clocks can be out of sync! (This is the problem on Dreamhost I believe)

By the way, make sure you notice the new benchmark.php starts with an underscore and delete the old one. But if you are done benchmarking you should delete it altogether or the underscore version will auto-load. The underscore was necessary so it could time the other benchmarks as it loaded before it.

It’s well known in any programming language that constant little echos to the output are much less efficient than buffering and outputting it all at the end. So that’s something like what you were running into.

If you use ob_start permanently, make sure you add the second optional chunked output option so the visitor gets periodic output from it, otherwise they won’t see anything until the entire page is flushed. Visually (and mentally) it’s more rewarding to start seeing some kind of page render the moment you click on something.

(my bb-gzip plugin enforces the chunked output option if you want to see it in action – note that certain server options will de-chunk the output however and ruin the effect like mod_gzip and most proxy caches like squid)

[@howtogeek Member 17 years, 4 months ago]

Thanks for that… good thought.

I changed it to ob_start(null,8192) instead of using the 4096 value you were using in the plugin. I noticed that when I had it set to 4096, I was still getting the really slow page render time for some reason, but 8192 works perfectly.

I think I’ve mentioned it before, but you rule

Oh, and dreamhost sucks.

[@_ck_ Participant 17 years, 4 months ago]

Yeah dreamhost certainly does suck. I just have a $20 per year promo account that I completely forgot about until recently when I wanted to test bbpress on various hosts (I’m looking for other super cheap hosts to try it on if you know of any, especially pay-as-you go like nearlyfreespeech.net)

What’s really strange is that some other forum software works MUCH faster on dreamhost and I am trying to figure out why. Take a look at WP plugin author GamerZ’s SMF (simple machines forum, evolved from YaBB) which is on another Dreamhost node:

http://forums.lesterchan.net

It’s incredibly responsive. Not even running an opcode cache. Of course SMF has half a decade of development behind it, but still, it’s impressive. I suspect they use partial page caching somehow. Meanwhile GamerZ’s WP blog running on the same account, crawls at times.

[@howtogeek Member 17 years, 4 months ago]

With dreamhost, it really depends on what server you end up on, and what time of the day. If you are lucky enough to get on a decent server without too many others, then you’ll have fast performance, or at least tolerable.

I couldn’t live without the VPS… and to answer your question way above somewhere, it’s 1GB dedicated. Mediatemple is great.

[@_ck_ Participant 17 years, 4 months ago]

That’s exactly right about Dreamhost.

I am working on a technique to try to load all the bbpress includes and plugins directly from the eaccelerator memcache instead of seeking on the disk first to check if they have changed (and just let the mysql plugins registry manage that).

I suspect I can eliminate the 400ms lag if I can pull that off. Not an easy feat though. Requires a replacement bb-settings.php

Oh and you’re probably paying per month for your dedicated what I pay per year for my VPS (I’ve migrated twice to get better neighbors):

=== benchmark & query results ===



12:08:28 up 25 days,  3:37,  0 users,  load average: 0.07, 0.04, 0.00



total page time: 0.045 seconds.



time to reach each section:

bb_underscore_plugins_loaded =  0.005

bb-polls.php loaded =  0.005

bb-signatures.php loaded =  0.005

report-post.php loaded =  0.006

my-views.php loaded =  0.006

bb-topic-views.php loaded =  0.006

user-topics-to-favorites.php loaded =  0.006

admin-post-anything.php loaded =  0.006

tweaks.php loaded =  0.006

my-views-most-least-views.php loaded =  0.007

my-views-started-participated-topics.php loaded =  0.008

my-views-installed-available-plugins.php loaded =  0.008

ignore-member.php loaded =  0.009

bb_plugins_loaded =  0.009

bb_init =  0.011

bb_index.php_pre_db =  0.011

bb_index.php =  0.016

front-page.php =  0.016

header.php =  0.017

logged-in.php =  0.018

footer.php =  0.045



time to render page: 0.0429 seconds (query time subtracted)



total query count: 12



total query time: 0.0021 seconds



slowest call was # 1 : 0.0005 seconds

SELECT * FROM bb_forums LIMIT 1 server:dbh_local connect: 0.38ms

[@howtogeek Member 17 years, 4 months ago]

Sweet stats there…. I’m still working on trimming some time off the page loads.

Yeah, MediaTemple’s best dedicated virtual server is $150 / month, and that’s what I’m using. I don’t pay that much because I’m in their partner program, but I wouldn’t mind paying it to make sure that my site is fast enough.

I’m really quite happy with them.

I’ll be very interested to see what you come up with on the eaccellerator front… I’ve been considering switching from xcache to that.

[@_ck_ Participant 17 years, 4 months ago]

eaccellerator and xcache are derived from the same original core – eaccellerator is a tad slower than both xcache and apc, but it also has the fewest problems and fewest incompatibilities compared to both. I believe it also has several people working on it, unlike xcache which means the project doesn’t grind to a halt when one person becomes distracted. No opcode cache project can in theory leapfrog in performance over another, it’s just little tweaks here and there.

The big secret is replacing Apache with LiteSpeed or if you really have alot of time on your hands and don’t need virtual hosts, Lighttpd. Apache is very inefficient, especially running PHP.

[@_ck_ Participant 17 years, 4 months ago]

I just installed SMF on the my dreamhost account next to bbpress.

SMF blows away bbpress’s performance, every time.

I have to investigate how the heck they do that. It feels like they are using partial page caching but their performance stats say they are using the same number of queries.

Then again I know bbpress’s weakness is not the queries, it’s all those plugins getting included and then firing up when the filters/actions first trigger.

Bugs the heck out of me. bbPress should be as fast!

[@_ck_ Participant 17 years, 4 months ago]

Ah, here’s SMF’s secret against bbPress – sheer number of files + size.

This definitely makes me want to go ahead with making a “quick loader” that will concat all the includes and then all the plugins as two blobs that can be either loaded from disk or mysql at user discretion based on the weakness of their shared server:

Front pages compared:

(keep in mind SMF actually has even more features than the dozen plugins loaded with bbPress, online user tracker, etc)

SMF 1.1.3

files for front page: 8 files, 338.45 Kb



index.php : 14.31 Kb

Settings.php : 3.71 Kb

Sources/QueryString.php : 18.84 Kb

Sources/Subs.php : 132.37 Kb

Sources/Errors.php : 15.94 Kb

Sources/Load.php : 92.47 Kb

Sources/Security.php : 29.88 Kb

Sources/MessageIndex.php : 30.93 Kb

=====================================================

`

bbPress 0.8.3 alpha

files for front page: 40 files, 437.15 Kb

index.php : 0.44 Kb

bb-load.php : 0.46 Kb

config.php : 2.28 Kb

bb-settings.php : 7.14 Kb

bb-includes/db-mysqli.php : 12.77 Kb

bb-includes/functions.php : 71.73 Kb

bb-includes/wp-classes.php : 3.37 Kb

bb-includes/classes.php : 37.98 Kb

bb-includes/formatting-functions.php : 8.73 Kb

bb-includes/template-functions.php : 59.12 Kb

bb-includes/capabilities.php : 13.15 Kb

bb-includes/cache.php : 8.52 Kb

bb-includes/deprecated.php : 8.17 Kb

bb-includes/wp-functions.php : 36.06 Kb

bb-includes/kses.php : 17.65 Kb

bb-includes/l10n.php : 1.76 Kb

bb-includes/bozo.php : 10.1 Kb

bb-includes/akismet.php : 8.14 Kb

bb-includes/default-filters.php : 4.45 Kb

bb-includes/script-loader.php : 9.2 Kb

bb-includes/compat.php : 0.57 Kb

my-plugins/_bb-benchmark.php : 4.79 Kb

my-plugins/_cookie-year.php : 0.92 Kb

my-plugins/bb-polls.php : 21.04 Kb

my-plugins/bb-signatures.php : 11.52 Kb

my-plugins/report-post.php : 3.92 Kb

my-plugins/user-topics-to-favorites.php : 0.59 Kb

my-plugins/bb-topic-views.php : 5.91 Kb

my-plugins/my-views.php : 10.13 Kb

my-plugins/notification.php : 2.26 Kb

my-plugins/my-views-started-participated-topics.php : 1.52 Kb

my-plugins/my-views-most-least-views.php : 2.7 Kb

my-plugins/plugin-browser.php : 21.72 Kb

my-plugins/my-views-installed-available-plugins.php : 6.61 Kb

bb-includes/pluggable.php : 8.86 Kb

bb-includes/locale.php : 6.91 Kb

bb-templates/kakumei/front-page.php : 3.03 Kb

bb-templates/kakumei/header.php : 1.6 Kb

bb-templates/kakumei/logged-in.php : 0.21 Kb

bb-templates/kakumei/footer.php : 1.13 Kb

=====================================================

[@sambauers Participant 17 years, 4 months ago]

Here are a couple of other reasons why SMF might be faster.

* SMF doesn’t use php-gettext for translation, it uses an array of text that can be replaced. This method approaches the speed of a native gettext implementation.

* bbPress has a lot of functions to register.

* SMF has (auto-detecting) built in support for memcached, eAccelerator, MMCache, AlternativePHP Cache and the Zend Optimiser

* Most (all?) of SMF is written inside PHP, i.e. the HTML is in strings that are echoed out of PHP, this is slightly quicker than dropping in and out of PHP to output to the browser.

* Many bbPress plugins don’t need to load on every page, they could start with some sort of conditional that requests the current bbPress location and then just send a “return” when they aren’t required.

That’s all I can think of after a quick look.

[@_ck_ Participant 17 years, 4 months ago]

Well every millisecond on the output does matter and the reasons you listed are very valid, but on heavily loaded servers, I have to believe the biggest weakness is the sheer number of files that have to seek and load. If they aren’t in the server’s file cache (dreamhost might have 1000 virtual hosts per node or more) that can cause long delays. Even if using an opcode cache, the cache still has to check the file date/size in the OS before it allows execution.

But even bbpress’s output template requires 3+ files to be included. SMF does it entirely differently in one single file.

I’ve built a prototype “fastload” plugin that concatinates two pools of files:

1. includes (except a few outside the general block in bb-settings)

2. all plugins

It actually works to some degree to speed things up on dreamhost when they are are at insane loads.

By reducing the number of files to seek from 40 to less than 10 the page load times are much better.

[@sambauers Participant 17 years, 4 months ago]

SMF is even smaller than that because a lot of the content in those files is comments.

[@_ck_ Participant 17 years, 4 months ago]

I’ve reduced the number of files included to:

=== executed files ===

/index.php : 0.44 Kb

/bb-load.php : 0.46 Kb

/config.php : 2.28 Kb

/bb-settings.php : 7.49 Kb

/bb-includes/db-mysqli.php : 12.77 Kb

/my-plugins/fastload-includes.php : 298.75 Kb

/my-plugins/fastload-plugins.php : 93.66 Kb

/bb-includes/pluggable.php : 8.86 Kb

/bb-includes/locale.php : 6.91 Kb

/bb-templates/kakumei/front-page.php : 3.03 Kb

/bb-templates/kakumei/header.php : 1.6 Kb

/bb-templates/kakumei/logged-in.php : 0.21 Kb

/bb-templates/kakumei/footer.php : 1.13 Kb

==================================================

With a little work I could make pluggable & locale include properly too.

[@sambauers Participant 17 years, 4 months ago]

File system hits are particularly bad on Dreamhost because from what I can tell the user home directories are actually based on file servers that are separate to the machine running apache.

[@sambauers Participant 17 years, 4 months ago]

You could probably drop the deprecated functions altogether. Would save a few milliseconds.

[@_ck_ Participant 17 years, 4 months ago]

Yeah I’ve seem some really interesting hosting methods by running bbpress on several cheap shared hosts just to see how it behaves.

SMF still freaks me out though. Then again they have like 5 years of development or much more if you include YaBB experience.

I have this clever little idea to trick dreamhost by storing the “fastload” plugins and includes as mysql blobs and using eval. Their mysql stays consistently fast.

[@sambauers Participant 17 years, 4 months ago]

> I have this clever little idea to trick dreamhost by storing the “fastload” plugins and includes as mysql blobs and using eval. Their mysql stays consistently fast.

Eeeek!

Let us know how that goes, I’ve never found any aspect of Dreamhost “fast”, cheap – yes, roomy – yes, fast – no.