Question about rendering time « bbPress support forums

bbPress support forums » Troubleshooting

Question about rendering time

(36 posts)

Started 1 year ago by HowToGeek
Latest reply from HowToGeek
This topic is not resolved

Tags:

HowToGeek
Member

I'm using the excellent bb-benchmark plugin, and here's what I get on my front page when logged in as admin:

01:24:43 up 147 days, 2:25, 2 users, load average: 0.08, 0.07, 0.01
total page time: 0.325 seconds.
query count: 17
total query time: 0.0258 seconds
page rendering time: 0.2992 seconds
slowest call was # 1 : 0.0077 seconds

What I found most strange was that the rendering time is so much higher than I would expect. Even with a guest account and much less queries going on I see this in the footer:

0.219 - 8 queries

Am I wrong, or is that render time taking too long? I notice that many of the other pages render in 0.005 seconds or so.

Posted 1 year ago #

Depending on your host and how accurate their microtime is (for example I actually get negative numbers sometimes on Dreamhost?!!?) it's fairly accurate but look at the page after a few tries and round out the numbers.

I have even more improvements in the newest bb-benchmach version (can't remember if I put it into the SVN yet but I don't think so). With a slight modification to bb-settings.php you can now get a detailed report like this:

=== benchmark & query results === 

load average: 1.64, 1.23, 1.14

total page time: 0.696 seconds.

time to reach each section:
   bb_underscore_plugins_loaded =  0.412
   bb-polls.php loaded =  0.421
   bb-signatures.php loaded =  0.431
   report-post.php loaded =  0.434
   user-topics-to-favorites.php loaded =  0.436
   tweaks.php loaded =  0.449
   bb-topic-views.php loaded =  0.472
   my-views.php loaded =  0.479
   notification.php loaded =  0.481
   bb-gzip.php loaded =  0.488
   my-views-installed-available-plugins.php loaded =  0.491
   my-views-started-participated-topics.php loaded =  0.493
   my-views-most-least-views.php loaded =  0.495
   plugin-browser.php loaded =  0.504
   bb_plugins_loaded =  0.506
   pluggable.php loaded =  0.523
   locale.php loaded =  0.543
   bb_init =  0.548
   bb_index.php_pre_db =  0.603
   bb_index.php =  0.616
   front-page.php =  0.632
   header.php =  0.655
   logged-in.php =  0.675
   footer.php =  0.694

time to render page: 0.6623 seconds (query time subtracted)

total query count: 11 

total query time: 0.0337 seconds 

slowest call was # 1 : 0.0169 seconds
SELECT meta_key, meta_value FROM bb_topicmeta WHERE topic_id = '0' server:dbh_local connect: 10.59ms

 === mysql queries used === 

 # 1 : 0.0169 seconds
SELECT meta_key, meta_value FROM bb_topicmeta WHERE topic_id = '0' server:dbh_local connect: 10.59ms

etc. etc.

Posted 1 year ago #

_ck_
Moderator

Oh and bbPress's (and WordPress's) biggest weakness is not mysql queries - it's actually all the plugins loading.

Unless you use an opcode cache or have really good disk caching on your host, loading a dozen or more plugins can cause those longer delays.

If you are on a VPS, you need eaccelerator or some other opcode cache. A few shared hosts like dreamhost let you install your own copy of php+fastcgi and eaccelerator too (but I do not recommend DreamHost, quite the opposite).

Posted 1 year ago #
HowToGeek
Member

I'm running xcache opcode caching, and I have a dedicated virtual with 1GB of ram from Mediatemple. I am very committed to making sure the site loads instantly... the site in question is the one linked on my username here.

So loading the plugins causes the speed problems? I've tried running phped profiler and couldn't find where the speed issue was.

Is it the actual parsing of the files, or loading in the functions into the filter/actions list?

I'm considering just concatenating all the plugins into a single plugin, just to see if that makes a difference.

Posted 1 year ago #
_ck_
Moderator

If you are running xcache, the plugins are not slowing you down, they get cached.

So "dedicated virtual" = VPS ? Interesting way to put it.
If so, you are at the mercy of the disk load on the rest of the node.

Your own average load rates have nothing to do with the rest of the server node, the VPS software tricks you.

(the big myth about VPS is that you are you are isolated from your neighbors, you're not - any high disk use by them will slow you down)

However if you have 1gb guaranteed, then you have quite a huge VPS. If that's 1gb burst, then it's another story.

But if you have your own vps you can be using an opcode cache (xcache is very good) and then make sure you are using mysql cache.

Not much else can be done. You could in theory replace apache with LiteSpeed unless you are savvy enough to setup Lighttpd (which I myself won't bother with)

Posted 1 year ago #
_ck_
Moderator
I've put 0.16 into the SVN, I think it updates on the hour.
Once you have 0.16 installed, go edit bb-settings.php in the bbpress root and around line 157 change that section so you add the line bb_benchmark_template_timer('',$plugin.' loaded');
like so:
```
if ( $plugins = bb_get_option( 'active_plugins' ) )
	foreach ( (array) $plugins as $plugin ) {
		if ( file_exists(BBPLUGINDIR . $plugin) ) {
			require( BBPLUGINDIR . $plugin );
			bb_benchmark_template_timer('',$plugin.' loaded');
		} }
do_action( 'bb_plugins_loaded' );
unset($plugins, $plugin);
```
.
That will give you the super detailed output, so if there is a single plugin that is consistently slowing your page load down, you might be able to spot it.
Posted 1 year ago #
HowToGeek
Member

This is interesting... because bbpress loads the plugins in order, the bb-benchmark plugin is getting loaded nearly last on my system, so I only see the output from the onlinelist plugin.

It looks like the vast majority of the execution time is before the plugins even get loaded.

Posted 1 year ago #
HowToGeek
Member

After doing some more research, the plugin loading time is virtually instant when you an opcode cache.

I added separate timestamp logging which shows that all the plugins are loaded at 0.019 but the bb_plugins_loaded action doesn't get fired until 0.1 seconds later.

bb-benchmark.php loaded = 0.019
bb_underscore_plugins_loaded = 0.019
bb_plugins_loaded = 0.102
bb_init = 0.103
bb_index.php_pre_db = 0.106
front-page.php = 0.106
header.php = 0.107
logged-in.php = 0.107
search-form.php = 0.116
footer.php = 0.302

The majority of the time is taken between search form and footer.... I'm not sure yet what gets executed between them.

Posted 1 year ago #
HowToGeek
Member

I think I figured it out... I added my own timing mechanism to virtually every line of code on the front page, and noticed the extra 0.2 seconds was occurring at random places...

So I put ob_start(); at the beginning of bb-settings.php to turn on output buffering, and now here's the times I get with like 16 plugins installed, logged in as admin with a ton of custom stuff going on and everything is opcached:

total page time: 0.078 seconds.
time to reach each section:
bb_underscore_plugins_loaded = 0.018
bb_plugins_loaded = 0.021
bb_init = 0.021
bb_index.php_pre_db = 0.024
front-page.php = 0.025
header.php = 0.025
logged-in.php = 0.026
search-form.php = 0.035
footer.php = 0.078

Now that is more like it! Now to remove all my logging....

And thanks for that benchmark plugin... it's great, I can't live without it now =)

Posted 1 year ago #
_ck_
Moderator

Note that microtime is off by at least 15ms on every call by nature, sometimes more depending what the cpu is doing and if you have a multiple core cpu. Sometimes the core clocks can be out of sync! (This is the problem on Dreamhost I believe)

By the way, make sure you notice the new benchmark.php starts with an underscore and delete the old one. But if you are done benchmarking you should delete it altogether or the underscore version will auto-load. The underscore was necessary so it could time the other benchmarks as it loaded before it.

It's well known in any programming language that constant little echos to the output are much less efficient than buffering and outputting it all at the end. So that's something like what you were running into.

If you use ob_start permanently, make sure you add the second optional chunked output option so the visitor gets periodic output from it, otherwise they won't see anything until the entire page is flushed. Visually (and mentally) it's more rewarding to start seeing some kind of page render the moment you click on something.

(my bb-gzip plugin enforces the chunked output option if you want to see it in action - note that certain server options will de-chunk the output however and ruin the effect like mod_gzip and most proxy caches like squid)

Posted 1 year ago #
HowToGeek
Member

Thanks for that... good thought.

I changed it to ob_start(null,8192) instead of using the 4096 value you were using in the plugin. I noticed that when I had it set to 4096, I was still getting the really slow page render time for some reason, but 8192 works perfectly.

I think I've mentioned it before, but you rule =)

Oh, and dreamhost sucks.

Posted 1 year ago #
_ck_
Moderator

Yeah dreamhost certainly does suck. I just have a $20 per year promo account that I completely forgot about until recently when I wanted to test bbpress on various hosts (I'm looking for other super cheap hosts to try it on if you know of any, especially pay-as-you go like nearlyfreespeech.net)

What's really strange is that some other forum software works MUCH faster on dreamhost and I am trying to figure out why. Take a look at WP plugin author GamerZ's SMF (simple machines forum, evolved from YaBB) which is on another Dreamhost node:
http://forums.lesterchan.net
It's incredibly responsive. Not even running an opcode cache. Of course SMF has half a decade of development behind it, but still, it's impressive. I suspect they use partial page caching somehow. Meanwhile GamerZ's WP blog running on the same account, crawls at times.

Posted 1 year ago #
HowToGeek
Member

With dreamhost, it really depends on what server you end up on, and what time of the day. If you are lucky enough to get on a decent server without too many others, then you'll have fast performance, or at least tolerable.

I couldn't live without the VPS... and to answer your question way above somewhere, it's 1GB dedicated. Mediatemple is great.

Posted 1 year ago #

_ck_
Moderator

That's exactly right about Dreamhost.
I am working on a technique to try to load all the bbpress includes and plugins directly from the eaccelerator memcache instead of seeking on the disk first to check if they have changed (and just let the mysql plugins registry manage that).

I suspect I can eliminate the 400ms lag if I can pull that off. Not an easy feat though. Requires a replacement bb-settings.php

Oh and you're probably paying per month for your dedicated what I pay per year for my VPS (I've migrated twice to get better neighbors):

=== benchmark & query results === 

 12:08:28 up 25 days,  3:37,  0 users,  load average: 0.07, 0.04, 0.00

total page time: 0.045 seconds.

time to reach each section:
   bb_underscore_plugins_loaded =  0.005
   bb-polls.php loaded =  0.005
   bb-signatures.php loaded =  0.005
   report-post.php loaded =  0.006
   my-views.php loaded =  0.006
   bb-topic-views.php loaded =  0.006
   user-topics-to-favorites.php loaded =  0.006
   admin-post-anything.php loaded =  0.006
   tweaks.php loaded =  0.006
   my-views-most-least-views.php loaded =  0.007
   my-views-started-participated-topics.php loaded =  0.008
   my-views-installed-available-plugins.php loaded =  0.008
   ignore-member.php loaded =  0.009
   bb_plugins_loaded =  0.009
   bb_init =  0.011
   bb_index.php_pre_db =  0.011
   bb_index.php =  0.016
   front-page.php =  0.016
   header.php =  0.017
   logged-in.php =  0.018
   footer.php =  0.045

time to render page: 0.0429 seconds (query time subtracted)

total query count: 12 

total query time: 0.0021 seconds 

slowest call was # 1 : 0.0005 seconds
SELECT * FROM bb_forums LIMIT 1 server:dbh_local connect: 0.38ms

Posted 1 year ago #

HowToGeek
Member

Sweet stats there.... I'm still working on trimming some time off the page loads.

Yeah, MediaTemple's best dedicated virtual server is $150 / month, and that's what I'm using. I don't pay that much because I'm in their partner program, but I wouldn't mind paying it to make sure that my site is fast enough.

I'm really quite happy with them.

I'll be very interested to see what you come up with on the eaccellerator front... I've been considering switching from xcache to that.

Posted 1 year ago #
_ck_
Moderator

eaccellerator and xcache are derived from the same original core - eaccellerator is a tad slower than both xcache and apc, but it also has the fewest problems and fewest incompatibilities compared to both. I believe it also has several people working on it, unlike xcache which means the project doesn't grind to a halt when one person becomes distracted. No opcode cache project can in theory leapfrog in performance over another, it's just little tweaks here and there.

The big secret is replacing Apache with LiteSpeed or if you really have alot of time on your hands and don't need virtual hosts, Lighttpd. Apache is very inefficient, especially running PHP.

Posted 1 year ago #
_ck_
Moderator

I just installed SMF on the my dreamhost account next to bbpress.

SMF blows away bbpress's performance, every time.

I have to investigate how the heck they do that. It feels like they are using partial page caching but their performance stats say they are using the same number of queries.

Then again I know bbpress's weakness is not the queries, it's all those plugins getting included and then firing up when the filters/actions first trigger.

Bugs the heck out of me. bbPress should be as fast!

Posted 1 year ago #
_ck_
Moderator
Ah, here's SMF's secret against bbPress - sheer number of files + size.

This definitely makes me want to go ahead with making a "quick loader" that will concat all the includes and then all the plugins as two blobs that can be either loaded from disk or mysql at user discretion based on the weakness of their shared server:

Front pages compared:
(keep in mind SMF actually has even more features than the dozen plugins loaded with bbPress, online user tracker, etc)
```
SMF 1.1.3
files for front page: 8 files, 338.45 Kb

index.php : 14.31 Kb
Settings.php : 3.71 Kb
Sources/QueryString.php : 18.84 Kb
Sources/Subs.php : 132.37 Kb
Sources/Errors.php : 15.94 Kb
Sources/Load.php : 92.47 Kb
Sources/Security.php : 29.88 Kb
Sources/MessageIndex.php : 30.93 Kb
=====================================================
```
`
bbPress 0.8.3 alpha
files for front page: 40 files, 437.15 Kb

index.php : 0.44 Kb
bb-load.php : 0.46 Kb
config.php : 2.28 Kb
bb-settings.php : 7.14 Kb
bb-includes/db-mysqli.php : 12.77 Kb
bb-includes/functions.php : 71.73 Kb
bb-includes/wp-classes.php : 3.37 Kb
bb-includes/classes.php : 37.98 Kb
bb-includes/formatting-functions.php : 8.73 Kb
bb-includes/template-functions.php : 59.12 Kb
bb-includes/capabilities.php : 13.15 Kb
bb-includes/cache.php : 8.52 Kb
bb-includes/deprecated.php : 8.17 Kb
bb-includes/wp-functions.php : 36.06 Kb
bb-includes/kses.php : 17.65 Kb
bb-includes/l10n.php : 1.76 Kb
bb-includes/bozo.php : 10.1 Kb
bb-includes/akismet.php : 8.14 Kb
bb-includes/default-filters.php : 4.45 Kb
bb-includes/script-loader.php : 9.2 Kb
bb-includes/compat.php : 0.57 Kb
my-plugins/_bb-benchmark.php : 4.79 Kb
my-plugins/_cookie-year.php : 0.92 Kb
my-plugins/bb-polls.php : 21.04 Kb
my-plugins/bb-signatures.php : 11.52 Kb
my-plugins/report-post.php : 3.92 Kb
my-plugins/user-topics-to-favorites.php : 0.59 Kb
my-plugins/bb-topic-views.php : 5.91 Kb
my-plugins/my-views.php : 10.13 Kb
my-plugins/notification.php : 2.26 Kb
my-plugins/my-views-started-participated-topics.php : 1.52 Kb
my-plugins/my-views-most-least-views.php : 2.7 Kb
my-plugins/plugin-browser.php : 21.72 Kb
my-plugins/my-views-installed-available-plugins.php : 6.61 Kb
bb-includes/pluggable.php : 8.86 Kb
bb-includes/locale.php : 6.91 Kb
bb-templates/kakumei/front-page.php : 3.03 Kb
bb-templates/kakumei/header.php : 1.6 Kb
bb-templates/kakumei/logged-in.php : 0.21 Kb
bb-templates/kakumei/footer.php : 1.13 Kb
=====================================================
Posted 1 year ago #
sambauers
Key Master

Here are a couple of other reasons why SMF might be faster.

* SMF doesn't use php-gettext for translation, it uses an array of text that can be replaced. This method approaches the speed of a native gettext implementation.

* bbPress has a lot of functions to register.

* SMF has (auto-detecting) built in support for memcached, eAccelerator, MMCache, AlternativePHP Cache and the Zend Optimiser

* Most (all?) of SMF is written inside PHP, i.e. the HTML is in strings that are echoed out of PHP, this is slightly quicker than dropping in and out of PHP to output to the browser.

* Many bbPress plugins don't need to load on every page, they could start with some sort of conditional that requests the current bbPress location and then just send a "return" when they aren't required.

That's all I can think of after a quick look.

Posted 1 year ago #
_ck_
Moderator

Well every millisecond on the output does matter and the reasons you listed are very valid, but on heavily loaded servers, I have to believe the biggest weakness is the sheer number of files that have to seek and load. If they aren't in the server's file cache (dreamhost might have 1000 virtual hosts per node or more) that can cause long delays. Even if using an opcode cache, the cache still has to check the file date/size in the OS before it allows execution.

But even bbpress's output template requires 3+ files to be included. SMF does it entirely differently in one single file.

I've built a prototype "fastload" plugin that concatinates two pools of files:
1. includes (except a few outside the general block in bb-settings)
2. all plugins

It actually works to some degree to speed things up on dreamhost when they are are at insane loads.

By reducing the number of files to seek from 40 to less than 10 the page load times are much better.

Posted 1 year ago #
sambauers
Key Master

SMF is even smaller than that because a lot of the content in those files is comments.

Posted 1 year ago #

_ck_
Moderator

I've reduced the number of files included to:

=== executed files ===
/index.php : 0.44 Kb
/bb-load.php : 0.46 Kb
/config.php : 2.28 Kb
/bb-settings.php : 7.49 Kb
/bb-includes/db-mysqli.php : 12.77 Kb
/my-plugins/fastload-includes.php : 298.75 Kb
/my-plugins/fastload-plugins.php : 93.66 Kb
/bb-includes/pluggable.php : 8.86 Kb
/bb-includes/locale.php : 6.91 Kb
/bb-templates/kakumei/front-page.php : 3.03 Kb
/bb-templates/kakumei/header.php : 1.6 Kb
/bb-templates/kakumei/logged-in.php : 0.21 Kb
/bb-templates/kakumei/footer.php : 1.13 Kb
==================================================

With a little work I could make pluggable & locale include properly too.

Posted 1 year ago #

sambauers
Key Master

File system hits are particularly bad on Dreamhost because from what I can tell the user home directories are actually based on file servers that are separate to the machine running apache.

Posted 1 year ago #
sambauers
Key Master

You could probably drop the deprecated functions altogether. Would save a few milliseconds.

Posted 1 year ago #
_ck_
Moderator

Yeah I've seem some really interesting hosting methods by running bbpress on several cheap shared hosts just to see how it behaves.

SMF still freaks me out though. Then again they have like 5 years of development or much more if you include YaBB experience.

I have this clever little idea to trick dreamhost by storing the "fastload" plugins and includes as mysql blobs and using eval. Their mysql stays consistently fast.

Posted 1 year ago #
sambauers
Key Master

> I have this clever little idea to trick dreamhost by storing the "fastload" plugins and includes as mysql blobs and using eval. Their mysql stays consistently fast.

Eeeek!

Let us know how that goes, I've never found any aspect of Dreamhost "fast", cheap - yes, roomy - yes, fast - no.

Posted 1 year ago #
_ck_
Moderator

Feel free to play with page load times:
http://bbpress.dreamhosters.com/forums/
vs.
http://bbpress.dreamhosters.com/smf/
The server is absolutely crawling right now so it's a great time to see how bad each one is. With "fastload" bbpress is respectable against smf.

I've unblocked the hidden bb-benchmarks so any user can see them if you do a view source and scroll to the bottom if you want to see all the stats.

Posted 1 year ago #
_ck_
Moderator

I've never seen their mysql take longer than 16-20ms to connect and never, ever more than 100ms to load all the queries. Storing code in mysql is crazy but just may work in weird cases like that. Of course if 1000 people did it, they'd ban the technique.

What's really interesting is their microtime never gives accurate results and frequently negative numbers. Either their multi-cores have clocks that are out of sync or parts of the code are somehow executing on different servers???? I posted a question about it on their support forum and it was mysteriously deleted the next day.

Posted 1 year ago #
HowToGeek
Member

They are both slower than all hell for me.

I'm very interested in your fastload plugin... but I wonder if it would matter much for opcode cached sites.

Seems like the majority of the problems in speed I'm getting are from the repeated hooks in tight loops.

Posted 1 year ago #
_ck_
Moderator

No "fastload" is not very useful for servers with a proper opcode cache and responsive disk system (and files are fetched and rendered on the same node). On my VPS it actually adds a few ms on average.

The load on dreamhost has been over 20 for most of today so it's not surprising anyone will find the bbpress+smf test sites slow.

I'm probably going to switch the showcase back to nearlyfreespeech shortly which is still on the slower side and uses only http 1.0 as it's forces though a squid proxy.

Unless someone can find a $20 or less per year host that has a better setup as my budget is maxed out. Can't even afford to replace my mouse that's acting up right now.

Posted 1 year ago #

RSS feed for this topic

1 2 Next »

Reply »

You must log in to post.

bbPress

Simple, Fast, Elegant

bbPress support forums » Troubleshooting

Question about rendering time

Reply »