Oh and looking at a MySQL performance guide for text search, apparently bbPress’s search is using ALL of the slowdown triggers:
order by
group by
and worst of all SQL_CALC_FOUND_ROWS
How long does this query take without the modifiers:
SELECT * FROM bb_posts WHERE post_text LIKE '%test%' LIMIT 5;
because that’s as fast as it can possibly get without an index and should be used as a baseline.
Looking at the original query, I believe they are causing a FULL scan of the entire table to rank and group it first. That’s going to be insanely slow.
You might be able to just use LIKE with a simple sort by post date DESC.
The largest bbPress install I have access to has nearly 100,000 rows and still only takes 0.04 seconds for that trimmed query. The original queries you posted take 0.5 seconds so that’s 10x slower.
@Nekita
Don’t modify your bb-config file until after you have installed bbPress.
Even then I can’t guarantee it will work with Trunk as it isn’t tested for deep integration yet.
If you click on the profile link (the title under your username) you will get a list of all posts you’ve made and threads you’ve started. It’s not search, but it will all be there. I don’t believe the member name is searched by default (could be completely wrong there and haven’t tested it in a long while.) I think it just shows “recent posts” and “relevant posts” (at least in my old version that’s what happens.) If you click the profile link, you get topics started and recent replies.
That roadmap is wildly inaccurate and just guesses.
Not only does 1.0 gut huge sections of code that were perfectly working under 0.9 and need to be replaced but it also replaces entire concepts (like the new object cache). There is a huge amount of work to do and I don’t envy Sam and Michael’s jobs right now. Then it will need lots of testing and debugging.
Then there is plugin stability since everything is in a state of flux. I can’t even keep up with the changes anymore, I’ve stopped updating my plugins until things settle down a bit.
Sam is currently working on trackbacks (pingbacks) now which was suppose to be the big new “feature” for 1.0 IMHO it’s a huge waste of time that could be spent elsewhere in the code but pingbacks on forums was something Matt wanted, so he gets what he wants obviously. However it will be the first feature I delete (not just disable, but delete) since XML-RPC was the #1 security problem with WordPress over the years. It’s also going to be a spammer’s delight.
Actually, the best thing to do is completely replace the search facility. That’s what I do in Super Search. bbPress’s search is very weak, so weak that there wasn’t even a link to it on any page in 0.9 (it’s similar to WordPress’s search, which also sucks, it’s the ugly truth no one seems to talk about).
I assume you are using the bb-benchmark plugin to watch those queries happen (if not, you should be).
Stupid question but you DO have the mysql cache turned on? I only ask because on many server configs (like CPANEL) it’s turned off by default. What does your my.cnf look like? (do a cat /etc/my.cnf in your shell)
Try going into phpmyadmin (or command line) and test that first test query against adding AND post_text REGEXP ':<:%test%:>:' to the query like so:
SELECT p.*, 0 AS search_score, MAX(post_time) AS post_time FROM bb_posts AS p JOIN bb_topics as t ON ( t.topic_id = p.topic_id ) WHERE p.post_text LIKE '%test%' AND p.post_text REGEXP ':<:%test%:>:' AND p.post_status = '0' AND t.topic_status = '0' GROUP BY t.topic_id ORDER BY p.post_time DESC LIMIT 5;
I suspect in the end due to your huge db size you are going to need to completely replace the search functions with something like this http://sphinxsearch.com which has a PHP api. Fortunately in bbPress it’s very simple to hook the internal search and bypass it entirely without even template hacks. A quick google shows that a few WordPress plugins have sphinxsearch support so that would be easy to copy over to bbPress. If you find the keyword “sphinxsearch” on this source for example, you can see how it’s done: http://svn.scriblio.net/plugin/trunk/scriblio.php
I tested this on a very large database under very low load. The standard bbpress search creates two queries, namely:
SELECT p.*, 0 AS search_score, MAX(post_time) AS post_time FROM bb_posts AS p JOIN bb_topics as t ON ( t.topic_id = p.topic_id ) WHERE p.post_text LIKE '%test%' AND p.post_status = '0' AND t.topic_status = '0' GROUP BY t.topic_id ORDER BY p.post_time DESC LIMIT 5;
and
SELECT t.*, MIN(p.post_id) as post_id, GROUP_CONCAT(p.post_text SEPARATOR ' ') AS post_text, 0 AS search_score FROM bb_topics AS t JOIN bb_posts as p ON ( t.topic_id = p.topic_id ) WHERE t.topic_status = '0' AND p.post_status = '0' AND ( (t.topic_title LIKE '%test%') OR p.post_text LIKE '%test%' ) GROUP BY t.topic_id ORDER BY t.topic_time DESC LIMIT 30
I ran both queries five times with different search terms. On my very large database, with no optimisation, the first query took (167s, 104s, 36s, 29.7s, 24s), the second (33s, 21s, 22s, 20s, 21s).
To give you an idea of what I’m talking about:
http://chocolatebydeath.com/forum/
User: Rauko
PW: IIlCuxOmYRpn
This is a test user I just registered. Once you log in with this data, you’ll see that this account is extremely restricted for whatever reason. It’s the same with my own admin account and other user accounts.
Well you’ll have to test to make sure a wikipost isn’t created yet on that topic (and that topic hasn’t been made yet, and they don’t edit the link to change the topic name,link, etc.).
But adding bbcode style parsing to posts is very straightforward. You’ll need to use preg_match_all on the $post->post_text
something like this:
add_filter('post_text', 'make_wiki_links'); // you can also try 'pre_post' which will make it only process the text once during save and not everytime it's displayed
function make_wiki_links($text) {
if (preg_match_all("/[wiki](.*?)[/wiki]/sim", $text, $wiki_links)) {
foreach ($wiki_links[0] as $wiki_link) {
// do whatever you want to each $wiki_link here
}
}
return $text;
}
You’ve got about a dozen problems to handle with this technique, including replacing the [wiki] parts afterwards with another preg_replace, good luck.
I’m curious about this issue as performance problems always intrigue me. You must be using 0.9 as 1.0 has an index on stickies by default.
As far as fulltext search there is a trick I use because of the multiple issues with fulltext (not only speed but fulltext can’t do words less than 4 characters until you customize and rebuild mysql). The trick is to use regex and do a two pass query where you first exclude all the posts without the words and then allow mysql to do a regular scan of the remaining posts.
Query example from my Super-Search plugin:
WHERE post_text LIKE '%".$term."%'" AND post_text REGEXP ':<:".$term.":>:[^']' "
Compare the performance of that against a fulltext search that uses "MATCH post_text AGAINST $term" I don’t have enough data to do a huge benchmark but some simple tests with the cache off shows 0.4 seconds for the trick and 0.9 for the fulltext.
The only downside to the trick is you cannot do partial word searches that way. ie. $term="cat" will only return posts with the exact word “cat” and not “cats” or “category”. But it should be way faster.
Searching a huge number of posts is a non-trivial problem. It’s been known to crush other forums like vbulletin which has fancy code to prevent users from searching too often/too quickly and even disable search temporarily on high server load. Sites like Wikipedia have to go through several technically complex tricks to keep the search fast on that much data.
Many large sites end up using sphinxsearch to replace fulltext search. You could interface it to bbPress via their PHP api.
You can read more workarounds on the mysql fulltext search page (with far more knowledgeable people than me) http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
I successfully integrated bbpress alpha with the latest wordpress. The install is pretty flawless.
Now, I want to pull in the header and sidebar from wordpress. Does anyone have any tips for me on how to do this?
http://www.punaji.com/
It’s a bit hard to understand at first, but given time I think the layout is much more usable than normal message boards. I would appreciate giving it that time, but if you want to know a quick rundown of how things work read on.
The left-most links on the “ribbons” are the categories (art and other). Running down the ribbons from left to right are the latest topics that have been posted in. The topic titles are on the ribbons while the last post of the topic shows beneath, and the username of whomever posted that message. Yeah, that’s about it.
There are quirks still, but they’re being worked on. Any insight on how you found the experience to be would be greatly appreciated.
Hi
I have WP 2.6 installed and the latest svn version (updated yesterday) of bbPress. The two are integrated with each other. I had a look at the users and roles in WP and noted that bbPress had changed all the roles to the bbPress role types. The WP roles are now all gone. The problem is that no one of my users can login, only the first admin user which I created when installed WP and bbPress. If I change another user from member to admin he cannot login.
Can I change back the roles to the default WP roles or do you have any clue what have happened?
Can add bbPress poll into wordpress like Bbpress Latest Discussion and embed wordpress post?
The test registered name is not in my database, just the original keymaster user.
I installed the alpha on its own database. When I test register, I am not sent a password, nor password sent when I try to recover password.
I just finished a test integration of bbpress into word-press and all is working very well with one exception. The static front page for word press has stopped functioning when I add the following to the end of the wp-config.php file:
if ( !defined(‘BBDB_NAME’) )
{
require_once(ABSPATH.’forum/bb-load.php’);
}
If I remove this and the couple of bbpress calls that I’m using which require this, the static front page works again. I think I’m just too tired to find the answer in the source.
Has anyone solved this? BTW — this is WP version 2.5.1 and bbpress 0.9.
Thanks in advance for any help.
yes – creating the autoload plugin apperently did the trick !!!
(I still need to test if all functionality works, but at least I can now see the posts 
)
thank, this issue is resolved for now
erf, the bbcodes are already not displayed :'( :'(
i try to make:
code strong test /strong /code
but i can’t see the strong balise.
with pre or code, bbcode are not displayed…
i need to see bbcodes in code tags because on my forum i use pre-formatted posts, for many uses
in example for:
pre strong test /strong /pre
i would like to see strong and /strong …
there is no way?
++
changes I made were related to the info being repeated twice and I was messing with the code to get the graphic bar working
I somehow worked out the first problem and the latest version of Karma plugin (0.0.5) solved the second problem so now everything is working perfectly
thank you for the update of the plugin and quick response
Thanks for your replies. After reading / testing your links, I found a quick working fix. Here’s the plugin code :
<?php
/*
Plugin Name: UTF-8 usernames
Plugin URI:
Description: This plugin enable utf-8 characters in usernames. Mbstring must be enabled in the php.ini.
Author: Amir Habibi
Author URI: http://www.residence-mixte.com/
Version: 0.1
*/
function sanitize_user_mbstring( $raw_username, $username, $strict = false ) {
$raw_username = $username;
$username = strip_tags($username);
// Kill octets
$username = preg_replace('|%([a-fA-F0-9][a-fA-F0-9])|', '', $username);
$username = preg_replace('/&.+?;/', '', $username); // Kill entities
// Usage of 'mb_ereg_replace' instead of 'preg_replace' to preserve accents.
if ( $strict )
$username = mb_ereg_replace('|[^a-z0-9 _.-@]|i', '', $username);
return apply_filters('sanitize_user_mbstring', $username, $raw_username, $strict);
}
add_action('sanitize_user', 'sanitize_user_mbstring', 0, 3);
?>
To work, you must have mstring enabled in your php.ini.
Hope this will help !
chrishajer : our board is in french
This is the latest discussion I heard about the issue:
https://bbpress.org/forums/topic/diacritic-letters-i-username-like-goran
What is the primary language of your forum?
bbPress is beta software right now. The latest beta release is 0.9.0.2. That release is not compatible with WordPress 2.6. The decisions are up to you.
The compatibility is only an issue if you want integration. If you don’t need integrated users between bbPress and WordPress, then install bbPress 0.9.0.2.
There is a discussion of changing the number of front page topics here:
https://bbpress.org/forums/topic/how-to-restrict-number-of-latest-discussions-on-front-page
To put the categories first and the latest topics after, you just need to move those things around in your front page template (front-page.php in your template directory).
Never use an alpha of any product for anything other than testing. Otherwise it’s 100% at your own risk.
bbPress 0.9 is very stable.
There are sometimes daily updates to 1.0 alpha (via the SVN of the trunk) but it will not be “finished” for many weeks.
Who is reviewing 1.0 alpha? You can’t review 1.0 alpha, it’s not finished. That’s like reviewing a car that has no interior.
