[riddle]

I’m in the process of setting up a bbPress installation (my first) at a new hosting provider (also my first time with them).

Subjectively, things seem a bit slow. Is there a tool for testing response time on a generic PHP hosting provider? Maybe a PHP app for very gentle stress testing?

Failing that, what sort of response time should be considered normal in bbPress’s self report in the comments at the bottom of the page? My top page currently says, “2.490 – 8 queries.” (That’s with a nearly-empty database.)

If there are serious problems with this host, I’d like to figure it out now, before I get settled in and launch.

Thanks.

[fel64]

Use the port of a pretty cool wp server diagnostics plugin. https://bbpress.org/forums/topic/front-page-takes-50-mysql-queries?replies=15#post-9140

Just add define( 'SAVEQUERIES', true ); to your config.php and activate this. Then go to View > Page Source and scroll to the bottom to see the diagnostics. (You can ignore all the stuff about queries. Just check out query time, page time, page render time. This tells you about the server bottlenecks.)

I just checked the numbers on my server, and I had 3.399 once and about twenty minutes later I had 0.436. This sort of thing can vary a lot, especially on shared hosting.

[riddle]

Thanks! I’ll check it out.

[chrishajer]

I would say if it seems slow to you now, it is going to seem slow to your users. especially when you have more of them. There’s a lot more to a good host than just the speed it takes to execute a bunch of queries, although that can point to a problem. My point is, there can be other problems that will not show up using just these diagnostics.

Do you have any information about the host, their connection to the Internet, the hardware they are using, the number of other websites on the shared machine? Do you have a link to your forum so others can see the speed you’re seeing? So you have shell access where you can run a command like top?

[_ck_]

Since there’s no real way to cache bbpress as static pages like wordpress (and not that there ever really can be, defeats the purpose) if you plan to have anything bigger than a “micro-forum” it’s important your host has mysql caching and optionally but recommend, some kind of php opcode cache (ie. eaccelerator)

Otherwise you’ll be getting emails from your host about your account creating all of the mysql load, etc.

The problem is I think most hosts do not have the two features I mentioned enabled. At least most shared hosting that’s done on cpanel “out of the box” on defaults. They have to be a bit more tech-savvy. And then if they are that clever, sometimes they will “oversell” the box thinking it can do more work than average.

Last but not least, on all shared hosting and VPS, you can have great performance for the moments you test it, but then a “bad neighbour” comes along and your performance will become quite bad for no reasons of your own, until the host does something about it, which can be days. Sometimes even the host can make poor decisions that will cripple your site, like backing it up during times of higher activity or allowing the statistics programs to run on dozens of clients at the exactly same time.

I wish I could afford dedicated hosting myself! Solves most of these problems. But spending $100 per year instead of $100 a month is no contest.

ps. 2.490 – 8 queries is not good at all if you are expecting high traffic – I have mysql cache and opcode cache and get always get under 0.300 for 10-30 queries no matter how many visitors. But I suspect I pay much more for my hosting. My guess is there’s definitely no mysql cache on there or you are paying very very little for that hosting.

[Ziyphr]

Maybe it’s worth posting a link to your install so we can give feedback on speed.

It sounds to me like you have a poor host – who is it? Many cheaper ones have poor database performance.

[riddle]

Thanks for all the comments!

I’m using WestHost. I am (naively, I know) trying to keep my site out of the search engines for another week or so, so please don’t link to it:

http:// muellerfever dot com

(I’m not using robots.txt because I don’t know how long crawlers cache the “do not harvest” status and I do want to announce the site soon.)

WestHost seems not to have top, w or uptime where I can get at them. I don’t know anything about the size of their pipes.

I chose WestHost because they came highly recommended and support Rails, which I may use for another project. I also have a HostingMatters account if I have reason to believe that would be faster.

[_ck_]

Page transfer time alone was not-so-hot.

Front page, non-cached, no-images, no external js/css was 2.84 seconds. Loading a fresh topic took over 6 seconds in my browser.

No gzip web compression enabled.

PHP5 (~10%+ slower than PHP4)

There’s definitely no mysql cache – way, way too slow.

Can you put make yourself a phpinfo.php page and put <? phpinfo(); ?> in there for us to check for other stuff? I don’t think it’s a security issue but others will have to agree or not.

Another neat way to look at unknown servers is with this perlinfo cgi I found and hacked at a bit a few years ago. Rename that to perlinfo.cgi upload to your cgi-bin and chmod it to 755. Then we can maybe see a bit more. If you don’t know how to do all this, then nevermind.

I need at least one static page to compare it all to.

If you don’t mind saying, how much are you paying for this hosting? According to westhost’s page ALL clients are on their own VPS, even @ $4/mo which is kinda strange. If true, that means you can customize mysql yourself and install eaccelerator.

top w and uptime are all done from a “shell” via SSH. You need to email them and ask them how you access your shell. If you are indeed on a VPS you should have access to one. Many shared hosting hosts block shell access because of security issues.

[riddle]

Thanks, CK. I’m a trusting soul, so here you go (on the same server and database):

http://riddle.whsites.net/bbpress/phpinfo.php

http://riddle.whsites.net/bbpress/static.html

http://riddle.whsites.net/cgi-bin/perlinfo.cgi

If I’ve just opened a big security hole, I hope someone will tell me!

I’m paying about $12 a month so this shouldn’t be absolute bottom-of-the-barrel service.

As for the shell: yes, I’m an old Unix guy, I built the site in SSH and vi. There’s no top, w or uptime in my path. If there’s a Red Hat-specific bin directory where I should look, I’ll do so.

[Ziyphr]

It felt slow to me.. there shouldn’t be that 2-3 second pause on each page.

If it has been over 48 hours I’d find another host.

[_ck_]

Well its a P4 (single core) @ 3ghz with 4gb of ram.

The question is how many clients is it hosting, and since you have your own ip, there’s no real way to determine that.

It’s taking almost a full second for 9 queries. Something is very, very, very wrong. You might want to enquire. If you do have access to /etc/my.cnf, you can turn on the mysql query cache and that should cause a radical improvement.

Not related to the mysql speed but just in general, any external javascript that you don’t need to execute immediately on a page, you should add the word DEFER to it. There are two flickr scripts you should try adding it to, if it still works with it in both IE and Firefox, it will make your pages seem a little faster.

ie.

<script DEFER type="text/javascript" ...blah...blah

Last but not least if they can’t fix it, you can do better for your almost $150 a year.

[riddle]

I do have access to my.cnf so I’m now off educating myself about the basics of mySQL configuration so I can enable caching without screwing something up.

Meanwhile, though, I have some more data points. In phpMyAdmin > Runtime Information the following variables are highlighted in red, meaning they are outside desirable range.

Slow_queries 5

Handler_read_rnd 39k

Handler_read_rnd_next 437k

Slow_launch_threads 1

Created_tmp_disk_tables 56

Opened_tables 84

The last is supposed to be a strong indication that my table cache value is too small (no surprise since it’s zero!). phpMyAdmin provides useful tips in interpreting these values, but as a mySQL newbie I’m still having trouble deciding how to act on them. Suggestions welcome.

As always, thanks!

[riddle]

Well, drat, while trying to get mysqld to reread my.cnf, I managed to kill it. So any of you kind souls reading this, please try back in a while (or tell me how to restart mysqld!).

[chrishajer]

/etc/rc.d/init.d/mysqld restart or /etc/init.d/mysqld restart ?

[chrishajer]

If you have shell access, I imagine you would have access to top ? My guess is the load on the server is pretty crazy, with lots of sites hosted there.

edit: just read where you don’t have top in your path. Hmm. But you have access to my.cnf? That’s odd.

[chrishajer]

How about apache bench?

http://httpd.apache.org/docs/1.3/programs/ab.html

[chrishajer]

And those files you posted aren’t a security threat per se, but they do give a lot of info about the server. I would take them down so you don’t forget about them. If someone were looking to exploit your server, they could start by finding out what options your php and perl were compiled with, and attack from there. Just take them down so you don’t forget about them.

[riddle]

Chris, thanks for the tips. Yes, WestHost is a little peculiar in what it chooses to make available to shell customers — there’s no tcsh, either.

“/etc/rc.d/init.d/mysqld restart” was what killed my MySQL and “/etc/rc.d/init.d/mysqld start” is a no-op. Sigh. I’ve got a query in to WestHost support.

And thanks for the warning about the diagnostics. Makes sense. I’ll disable them for now.

[_ck_]

I should point out that even your static.html serves up fairly slowly, though a little faster than the dynamic pages. Getting mysql caching may not be as big a boost as it hopefully should be. I can’t even see what your guaranteed VPS ram is.

If you are on a VPS there is usually a seperate VPS control panel that will also let you restart services, including mysql.

my setup lets me do a “service mysql restart” but that’s probably not universal. This is what I have in my.cnf

[mysqld]



# myisam-recover = BACKUP

# delay-key-write = ALL



max_connections = 400

key_buffer = 16M

myisam_sort_buffer_size = 32M

join_buffer_size = 1M

read_buffer_size = 1M

sort_buffer_size = 2M

table_cache = 1024

thread_cache_size = 286

interactive_timeout = 25

wait_timeout = 1000

connect_timeout = 10

max_allowed_packet = 16M

max_connect_errors = 10

query_cache_limit = 1M

query_cache_size = 16M

query_cache_type = 1

tmp_table_size = 16M

skip-innodb

old-passwords = 1

basedir = /var/lib/mysql

datadir = /var/lib/mysql

long_query_time = 1

log-slow-queries = /var/log/mysql/mysql-slow.log

log-error = /var/log/mysql/mysqld.err



[mysqld_safe]

open_files_limit = 8192



[mysqldump]

quick

max_allowed_packet = 16M



[myisamchk]

key_buffer = 32M

sort_buffer = 32M

read_buffer = 16M

write_buffer = 16M

[_ck_]

Looks like they are really trying to hide “load” from you.

I know of only two ways to grab it from php (there might be more?)

echo "method 1: ".file_get_contents('/proc/loadavg');

echo "method 2: ".shell_exec("uptime");

Really though, should not be this hard.

It’s possible to get better getting hosting for $150/year.

[riddle]

Restart mystery solved: WestHost changed something so mysqld required an IP bind address in my.cnf. Once I formally reported the problem to WestHost support, they had it fixed in minutes and were even kind enough to apologize. (I sure wish that init.d gave some diagnostics!)

That done, I turned on query caching:

query_cache_type=1

query_cache_limit=1M

query_cache_size=32M

The result is significant improvement but there may be more I can do. I’m now getting a median response time of about 0.125 seconds for 8 queries on the top page and 0.205 seconds for 12 queries on a topic page.

In phpMyAdmin the indicators are greatly improved but I still have a couple in the red zone:

Handler_read_rnd_next 15

(“The number of requests to read the next row in the data file. This is high if you are doing a lot of table scans. Generally this suggests that your tables are not properly indexed or that your queries are not written to take advantage of the indexes you have.”)

Key_reads 6

(“The number of physical reads of a key block from disk. If Key_reads is big, then your key_buffer_size value is probably too small. The cache miss rate can be calculated as Key_reads/Key_read_requests.” My key buffer size is 16M.)

Opened_tables 2

(“The number of tables that have been opened. If opened tables is big, your table cache value is probably too small.” My table cache is 64.)

I’m inclined to say this is good enough for now, but I’m always open for further suggestions.

[riddle]

P.S. I wrote a quick and dirty perl script to do a bunch of page retrievals, grep for bbPress’s self-reporting string in the footer, and report median times. Let me know if anybody wants it.

[_ck_]

I’m now getting a median response time of about 0.125 seconds for 8 queries on the top page and 0.205 seconds for 12 queries on a topic page.

That’s like night and day compared to what you had before. It was taking almost a full second in some cases, look at the bottom source of the static.html !!!

Now can you believe most shared hosting environments have mysql cache turned off by default!!! Even your VPS neighbours may have it turned off which is causing high disk load that’s affecting you. This is where all those who claim how VPS is superiour are wrong. Disk access is the achilles heel of VPS (and shared hosting).

Next performance improvement you can do is install eaccelerator. Though it’s nowhere near as easy as editing mysql and restarting services.

Do you know how much guaranteed ram you have on your vps? Because the mysql and eacellerator cache will eat into that. If you have less that 128mb it will limit the sheer number of simultaneous connections you can have. Though for your needs I don’t think it’s a big deal unless all the search engines start scanning you at once.

ps. don’t worry any more about mysql tuning and phpmyadmin status reports, it’s really all you can do for now – the rest is a limit of how wordpress/bbpress works and can’t be avoided

pps. most of your pages are now coming up in 1.5 seconds for me when they were taking up to 6 seconds before – there are still some weird delays sometimes though which must be OS loads

[riddle]

Thanks for all your help, CK! I’ll look into eaccelerator — although I think I’ll finish building the site first.