Skip to:
Content
Pages
Categories
Search
Top
Bottom

Question about rendering time

  • I’m using the excellent bb-benchmark plugin, and here’s what I get on my front page when logged in as admin:

    01:24:43 up 147 days, 2:25, 2 users, load average: 0.08, 0.07, 0.01

    total page time: 0.325 seconds.

    query count: 17

    total query time: 0.0258 seconds

    page rendering time: 0.2992 seconds

    slowest call was # 1 : 0.0077 seconds

    What I found most strange was that the rendering time is so much higher than I would expect. Even with a guest account and much less queries going on I see this in the footer:

    0.219 – 8 queries

    Am I wrong, or is that render time taking too long? I notice that many of the other pages render in 0.005 seconds or so.

Viewing 25 replies - 1 through 25 (of 35 total)

  • _ck_
    Participant

    @_ck_

    Depending on your host and how accurate their microtime is (for example I actually get negative numbers sometimes on Dreamhost?!!?) it’s fairly accurate but look at the page after a few tries and round out the numbers.

    I have even more improvements in the newest bb-benchmach version (can’t remember if I put it into the SVN yet but I don’t think so). With a slight modification to bb-settings.php you can now get a detailed report like this:

    === benchmark & query results ===

    load average: 1.64, 1.23, 1.14

    total page time: 0.696 seconds.

    time to reach each section:
    bb_underscore_plugins_loaded = 0.412
    bb-polls.php loaded = 0.421
    bb-signatures.php loaded = 0.431
    report-post.php loaded = 0.434
    user-topics-to-favorites.php loaded = 0.436
    tweaks.php loaded = 0.449
    bb-topic-views.php loaded = 0.472
    my-views.php loaded = 0.479
    notification.php loaded = 0.481
    bb-gzip.php loaded = 0.488
    my-views-installed-available-plugins.php loaded = 0.491
    my-views-started-participated-topics.php loaded = 0.493
    my-views-most-least-views.php loaded = 0.495
    plugin-browser.php loaded = 0.504
    bb_plugins_loaded = 0.506
    pluggable.php loaded = 0.523
    locale.php loaded = 0.543
    bb_init = 0.548
    bb_index.php_pre_db = 0.603
    bb_index.php = 0.616
    front-page.php = 0.632
    header.php = 0.655
    logged-in.php = 0.675
    footer.php = 0.694

    time to render page: 0.6623 seconds (query time subtracted)

    total query count: 11

    total query time: 0.0337 seconds

    slowest call was # 1 : 0.0169 seconds
    SELECT meta_key, meta_value FROM bb_topicmeta WHERE topic_id = '0' server:dbh_local connect: 10.59ms

    === mysql queries used ===

    # 1 : 0.0169 seconds
    SELECT meta_key, meta_value FROM bb_topicmeta WHERE topic_id = '0' server:dbh_local connect: 10.59ms

    etc. etc.


    _ck_
    Participant

    @_ck_

    Oh and bbPress’s (and WordPress’s) biggest weakness is not mysql queries – it’s actually all the plugins loading.

    Unless you use an opcode cache or have really good disk caching on your host, loading a dozen or more plugins can cause those longer delays.

    If you are on a VPS, you need eaccelerator or some other opcode cache. A few shared hosts like dreamhost let you install your own copy of php+fastcgi and eaccelerator too (but I do not recommend DreamHost, quite the opposite).

    I’m running xcache opcode caching, and I have a dedicated virtual with 1GB of ram from Mediatemple. I am very committed to making sure the site loads instantly… the site in question is the one linked on my username here.

    So loading the plugins causes the speed problems? I’ve tried running phped profiler and couldn’t find where the speed issue was.

    Is it the actual parsing of the files, or loading in the functions into the filter/actions list?

    I’m considering just concatenating all the plugins into a single plugin, just to see if that makes a difference.


    _ck_
    Participant

    @_ck_

    If you are running xcache, the plugins are not slowing you down, they get cached.

    So “dedicated virtual” = VPS ? Interesting way to put it.

    If so, you are at the mercy of the disk load on the rest of the node.

    Your own average load rates have nothing to do with the rest of the server node, the VPS software tricks you.

    (the big myth about VPS is that you are you are isolated from your neighbors, you’re not – any high disk use by them will slow you down)

    However if you have 1gb guaranteed, then you have quite a huge VPS. If that’s 1gb burst, then it’s another story.

    But if you have your own vps you can be using an opcode cache (xcache is very good) and then make sure you are using mysql cache.

    Not much else can be done. You could in theory replace apache with LiteSpeed unless you are savvy enough to setup Lighttpd (which I myself won’t bother with)


    _ck_
    Participant

    @_ck_

    I’ve put 0.16 into the SVN, I think it updates on the hour.

    Once you have 0.16 installed, go edit bb-settings.php in the bbpress root and around line 157 change that section so you add the line bb_benchmark_template_timer('',$plugin.' loaded');

    like so:

    if ( $plugins = bb_get_option( 'active_plugins' ) )
    foreach ( (array) $plugins as $plugin ) {
    if ( file_exists(BBPLUGINDIR . $plugin) ) {
    require( BBPLUGINDIR . $plugin );
    bb_benchmark_template_timer('',$plugin.' loaded');
    } }
    do_action( 'bb_plugins_loaded' );
    unset($plugins, $plugin);

    .

    That will give you the super detailed output, so if there is a single plugin that is consistently slowing your page load down, you might be able to spot it.

    This is interesting… because bbpress loads the plugins in order, the bb-benchmark plugin is getting loaded nearly last on my system, so I only see the output from the onlinelist plugin.

    It looks like the vast majority of the execution time is before the plugins even get loaded.

    After doing some more research, the plugin loading time is virtually instant when you an opcode cache.

    I added separate timestamp logging which shows that all the plugins are loaded at 0.019 but the bb_plugins_loaded action doesn’t get fired until 0.1 seconds later.

    bb-benchmark.php loaded = 0.019

    bb_underscore_plugins_loaded = 0.019

    bb_plugins_loaded = 0.102

    bb_init = 0.103

    bb_index.php_pre_db = 0.106

    front-page.php = 0.106

    header.php = 0.107

    logged-in.php = 0.107

    search-form.php = 0.116

    footer.php = 0.302

    The majority of the time is taken between search form and footer…. I’m not sure yet what gets executed between them.

    I think I figured it out… I added my own timing mechanism to virtually every line of code on the front page, and noticed the extra 0.2 seconds was occurring at random places…

    So I put ob_start(); at the beginning of bb-settings.php to turn on output buffering, and now here’s the times I get with like 16 plugins installed, logged in as admin with a ton of custom stuff going on and everything is opcached:

    total page time: 0.078 seconds.

    time to reach each section:

    bb_underscore_plugins_loaded = 0.018

    bb_plugins_loaded = 0.021

    bb_init = 0.021

    bb_index.php_pre_db = 0.024

    front-page.php = 0.025

    header.php = 0.025

    logged-in.php = 0.026

    search-form.php = 0.035

    footer.php = 0.078

    Now that is more like it! Now to remove all my logging….

    And thanks for that benchmark plugin… it’s great, I can’t live without it now =)


    _ck_
    Participant

    @_ck_

    Note that microtime is off by at least 15ms on every call by nature, sometimes more depending what the cpu is doing and if you have a multiple core cpu. Sometimes the core clocks can be out of sync! (This is the problem on Dreamhost I believe)

    By the way, make sure you notice the new benchmark.php starts with an underscore and delete the old one. But if you are done benchmarking you should delete it altogether or the underscore version will auto-load. The underscore was necessary so it could time the other benchmarks as it loaded before it.

    It’s well known in any programming language that constant little echos to the output are much less efficient than buffering and outputting it all at the end. So that’s something like what you were running into.

    If you use ob_start permanently, make sure you add the second optional chunked output option so the visitor gets periodic output from it, otherwise they won’t see anything until the entire page is flushed. Visually (and mentally) it’s more rewarding to start seeing some kind of page render the moment you click on something.

    (my bb-gzip plugin enforces the chunked output option if you want to see it in action – note that certain server options will de-chunk the output however and ruin the effect like mod_gzip and most proxy caches like squid)

    Thanks for that… good thought.

    I changed it to ob_start(null,8192) instead of using the 4096 value you were using in the plugin. I noticed that when I had it set to 4096, I was still getting the really slow page render time for some reason, but 8192 works perfectly.

    I think I’ve mentioned it before, but you rule =)

    Oh, and dreamhost sucks.


    _ck_
    Participant

    @_ck_

    Yeah dreamhost certainly does suck. I just have a $20 per year promo account that I completely forgot about until recently when I wanted to test bbpress on various hosts (I’m looking for other super cheap hosts to try it on if you know of any, especially pay-as-you go like nearlyfreespeech.net)

    What’s really strange is that some other forum software works MUCH faster on dreamhost and I am trying to figure out why. Take a look at WP plugin author GamerZ’s SMF (simple machines forum, evolved from YaBB) which is on another Dreamhost node:

    http://forums.lesterchan.net

    It’s incredibly responsive. Not even running an opcode cache. Of course SMF has half a decade of development behind it, but still, it’s impressive. I suspect they use partial page caching somehow. Meanwhile GamerZ’s WP blog running on the same account, crawls at times.

    With dreamhost, it really depends on what server you end up on, and what time of the day. If you are lucky enough to get on a decent server without too many others, then you’ll have fast performance, or at least tolerable.

    I couldn’t live without the VPS… and to answer your question way above somewhere, it’s 1GB dedicated. Mediatemple is great.


    _ck_
    Participant

    @_ck_

    That’s exactly right about Dreamhost.

    I am working on a technique to try to load all the bbpress includes and plugins directly from the eaccelerator memcache instead of seeking on the disk first to check if they have changed (and just let the mysql plugins registry manage that).

    I suspect I can eliminate the 400ms lag if I can pull that off. Not an easy feat though. Requires a replacement bb-settings.php

    Oh and you’re probably paying per month for your dedicated what I pay per year for my VPS (I’ve migrated twice to get better neighbors):

    === benchmark & query results ===

    12:08:28 up 25 days, 3:37, 0 users, load average: 0.07, 0.04, 0.00

    total page time: 0.045 seconds.

    time to reach each section:
    bb_underscore_plugins_loaded = 0.005
    bb-polls.php loaded = 0.005
    bb-signatures.php loaded = 0.005
    report-post.php loaded = 0.006
    my-views.php loaded = 0.006
    bb-topic-views.php loaded = 0.006
    user-topics-to-favorites.php loaded = 0.006
    admin-post-anything.php loaded = 0.006
    tweaks.php loaded = 0.006
    my-views-most-least-views.php loaded = 0.007
    my-views-started-participated-topics.php loaded = 0.008
    my-views-installed-available-plugins.php loaded = 0.008
    ignore-member.php loaded = 0.009
    bb_plugins_loaded = 0.009
    bb_init = 0.011
    bb_index.php_pre_db = 0.011
    bb_index.php = 0.016
    front-page.php = 0.016
    header.php = 0.017
    logged-in.php = 0.018
    footer.php = 0.045

    time to render page: 0.0429 seconds (query time subtracted)

    total query count: 12

    total query time: 0.0021 seconds

    slowest call was # 1 : 0.0005 seconds
    SELECT * FROM bb_forums LIMIT 1 server:dbh_local connect: 0.38ms

    Sweet stats there…. I’m still working on trimming some time off the page loads.

    Yeah, MediaTemple’s best dedicated virtual server is $150 / month, and that’s what I’m using. I don’t pay that much because I’m in their partner program, but I wouldn’t mind paying it to make sure that my site is fast enough.

    I’m really quite happy with them.

    I’ll be very interested to see what you come up with on the eaccellerator front… I’ve been considering switching from xcache to that.


    _ck_
    Participant

    @_ck_

    eaccellerator and xcache are derived from the same original core – eaccellerator is a tad slower than both xcache and apc, but it also has the fewest problems and fewest incompatibilities compared to both. I believe it also has several people working on it, unlike xcache which means the project doesn’t grind to a halt when one person becomes distracted. No opcode cache project can in theory leapfrog in performance over another, it’s just little tweaks here and there.

    The big secret is replacing Apache with LiteSpeed or if you really have alot of time on your hands and don’t need virtual hosts, Lighttpd. Apache is very inefficient, especially running PHP.


    _ck_
    Participant

    @_ck_

    I just installed SMF on the my dreamhost account next to bbpress.

    SMF blows away bbpress’s performance, every time.

    I have to investigate how the heck they do that. It feels like they are using partial page caching but their performance stats say they are using the same number of queries.

    Then again I know bbpress’s weakness is not the queries, it’s all those plugins getting included and then firing up when the filters/actions first trigger.

    Bugs the heck out of me. bbPress should be as fast!


    _ck_
    Participant

    @_ck_

    Ah, here’s SMF’s secret against bbPress – sheer number of files + size.

    This definitely makes me want to go ahead with making a “quick loader” that will concat all the includes and then all the plugins as two blobs that can be either loaded from disk or mysql at user discretion based on the weakness of their shared server:

    Front pages compared:

    (keep in mind SMF actually has even more features than the dozen plugins loaded with bbPress, online user tracker, etc)

    SMF 1.1.3
    files for front page: 8 files, 338.45 Kb

    index.php : 14.31 Kb
    Settings.php : 3.71 Kb
    Sources/QueryString.php : 18.84 Kb
    Sources/Subs.php : 132.37 Kb
    Sources/Errors.php : 15.94 Kb
    Sources/Load.php : 92.47 Kb
    Sources/Security.php : 29.88 Kb
    Sources/MessageIndex.php : 30.93 Kb
    =====================================================

    `

    bbPress 0.8.3 alpha

    files for front page: 40 files, 437.15 Kb

    index.php : 0.44 Kb

    bb-load.php : 0.46 Kb

    config.php : 2.28 Kb

    bb-settings.php : 7.14 Kb

    bb-includes/db-mysqli.php : 12.77 Kb

    bb-includes/functions.php : 71.73 Kb

    bb-includes/wp-classes.php : 3.37 Kb

    bb-includes/classes.php : 37.98 Kb

    bb-includes/formatting-functions.php : 8.73 Kb

    bb-includes/template-functions.php : 59.12 Kb

    bb-includes/capabilities.php : 13.15 Kb

    bb-includes/cache.php : 8.52 Kb

    bb-includes/deprecated.php : 8.17 Kb

    bb-includes/wp-functions.php : 36.06 Kb

    bb-includes/kses.php : 17.65 Kb

    bb-includes/l10n.php : 1.76 Kb

    bb-includes/bozo.php : 10.1 Kb

    bb-includes/akismet.php : 8.14 Kb

    bb-includes/default-filters.php : 4.45 Kb

    bb-includes/script-loader.php : 9.2 Kb

    bb-includes/compat.php : 0.57 Kb

    my-plugins/_bb-benchmark.php : 4.79 Kb

    my-plugins/_cookie-year.php : 0.92 Kb

    my-plugins/bb-polls.php : 21.04 Kb

    my-plugins/bb-signatures.php : 11.52 Kb

    my-plugins/report-post.php : 3.92 Kb

    my-plugins/user-topics-to-favorites.php : 0.59 Kb

    my-plugins/bb-topic-views.php : 5.91 Kb

    my-plugins/my-views.php : 10.13 Kb

    my-plugins/notification.php : 2.26 Kb

    my-plugins/my-views-started-participated-topics.php : 1.52 Kb

    my-plugins/my-views-most-least-views.php : 2.7 Kb

    my-plugins/plugin-browser.php : 21.72 Kb

    my-plugins/my-views-installed-available-plugins.php : 6.61 Kb

    bb-includes/pluggable.php : 8.86 Kb

    bb-includes/locale.php : 6.91 Kb

    bb-templates/kakumei/front-page.php : 3.03 Kb

    bb-templates/kakumei/header.php : 1.6 Kb

    bb-templates/kakumei/logged-in.php : 0.21 Kb

    bb-templates/kakumei/footer.php : 1.13 Kb

    =====================================================


    Sam Bauers
    Participant

    @sambauers

    Here are a couple of other reasons why SMF might be faster.

    * SMF doesn’t use php-gettext for translation, it uses an array of text that can be replaced. This method approaches the speed of a native gettext implementation.

    * bbPress has a lot of functions to register.

    * SMF has (auto-detecting) built in support for memcached, eAccelerator, MMCache, AlternativePHP Cache and the Zend Optimiser

    * Most (all?) of SMF is written inside PHP, i.e. the HTML is in strings that are echoed out of PHP, this is slightly quicker than dropping in and out of PHP to output to the browser.

    * Many bbPress plugins don’t need to load on every page, they could start with some sort of conditional that requests the current bbPress location and then just send a “return” when they aren’t required.

    That’s all I can think of after a quick look.


    _ck_
    Participant

    @_ck_

    Well every millisecond on the output does matter and the reasons you listed are very valid, but on heavily loaded servers, I have to believe the biggest weakness is the sheer number of files that have to seek and load. If they aren’t in the server’s file cache (dreamhost might have 1000 virtual hosts per node or more) that can cause long delays. Even if using an opcode cache, the cache still has to check the file date/size in the OS before it allows execution.

    But even bbpress’s output template requires 3+ files to be included. SMF does it entirely differently in one single file.

    I’ve built a prototype “fastload” plugin that concatinates two pools of files:

    1. includes (except a few outside the general block in bb-settings)

    2. all plugins

    It actually works to some degree to speed things up on dreamhost when they are are at insane loads.

    By reducing the number of files to seek from 40 to less than 10 the page load times are much better.


    Sam Bauers
    Participant

    @sambauers

    SMF is even smaller than that because a lot of the content in those files is comments.


    _ck_
    Participant

    @_ck_

    I’ve reduced the number of files included to:

    === executed files ===
    /index.php : 0.44 Kb
    /bb-load.php : 0.46 Kb
    /config.php : 2.28 Kb
    /bb-settings.php : 7.49 Kb
    /bb-includes/db-mysqli.php : 12.77 Kb
    /my-plugins/fastload-includes.php : 298.75 Kb
    /my-plugins/fastload-plugins.php : 93.66 Kb
    /bb-includes/pluggable.php : 8.86 Kb
    /bb-includes/locale.php : 6.91 Kb
    /bb-templates/kakumei/front-page.php : 3.03 Kb
    /bb-templates/kakumei/header.php : 1.6 Kb
    /bb-templates/kakumei/logged-in.php : 0.21 Kb
    /bb-templates/kakumei/footer.php : 1.13 Kb
    ==================================================

    With a little work I could make pluggable & locale include properly too.


    Sam Bauers
    Participant

    @sambauers

    File system hits are particularly bad on Dreamhost because from what I can tell the user home directories are actually based on file servers that are separate to the machine running apache.


    Sam Bauers
    Participant

    @sambauers

    You could probably drop the deprecated functions altogether. Would save a few milliseconds.


    _ck_
    Participant

    @_ck_

    Yeah I’ve seem some really interesting hosting methods by running bbpress on several cheap shared hosts just to see how it behaves.

    SMF still freaks me out though. Then again they have like 5 years of development or much more if you include YaBB experience.

    I have this clever little idea to trick dreamhost by storing the “fastload” plugins and includes as mysql blobs and using eval. Their mysql stays consistently fast.


    Sam Bauers
    Participant

    @sambauers

    > I have this clever little idea to trick dreamhost by storing the “fastload” plugins and includes as mysql blobs and using eval. Their mysql stays consistently fast.

    Eeeek!

    Let us know how that goes, I’ve never found any aspect of Dreamhost “fast”, cheap – yes, roomy – yes, fast – no.

Viewing 25 replies - 1 through 25 (of 35 total)

You must be logged in to reply to this topic.