Somethings holding us back (random thoughts)…
-
Last Thursday, after an email conversation with 3 people I respect from this board, i decided to take a look at the old bbProgress code. With a bit of dusting off, the weekend has seemed quite fruitful. there are though, still a few things that bother me greatly, and the more I delve into bbPress and the ability to theme it differently or extract data in a different manner, the more questions it raises.
Specifically, bbPress is effectively set up to meet singular criteria, and really struggles to be adaptable. I’d go as far as to say it’s far to blog like in data structure. I’m confident that if i had a few days off work (and with no kids) that i could easily make a wordpress theme that would look and work exactly the same as bbPress. That’s fairly disappointing.
It’s disappointing because forums and blogs are very different in their make up. Forums have genealogy and many to many relationships, while blogs do not. It’s not just a case of a feature wishlist, or replicating features on other forum solutions, it’s rather the mindset that bbPress has evolved with. The “this works for us” mentality. Here are some examples:
Genealogy
bbPress makes no attempt to update parent forums. It’s a major flaw that leads to forum X having no information on forum Y at all, even though they are directly related. Instead, in order to find this out, a plugin that makes numerous SQL calls every time a forum is loaded (and for every forum listed) must be created to fake this basic functionality.
Should your forum have no parent/child relationships (such as this forum and the WordPress support forums), then the issue probably hasn’t arisen.
This somewhat forces bbPress forums into linking via topics/posts rather than forum structure. I’m neither for nor against this, it definitely has its advantages for some forums, but for larger more structure forums where parent/child/child relationships are helpful, this quickly becomes very difficult when it doesn’t need to be.
More importantly, by only storing information in this blog-esque flat-file scenario, we’re keeping theme development very tied into plugins for even the most basic of information such as:
Forum – > Last Post
I’ve raised this one for a year and a bit now, but its a great example (imo).
Lets say, i run a review forum for restaurants in Pennsylvania (i don’t):
Forum
Topics — Posts — Last Post=======================================
USA
0 topics – 0 posts – No posts– Pennsylvania
0 topics – 0 posts – No posts– – Philadelphia
0 topics – 0 posts – No posts– – Pittsburg
0 topics – 0 posts – No posts– – Allentown
0 topics – 0 posts – No posts– – Scranton
0 topics – 0 posts – No posts– – – chinese restaurants– 1 topics – 1 posts – nom nom nom by fat albert
– – – french restaurants— 3 topics – 42 posts – i hate snails by Jody
– – – italian restaurants— 9 topics – 99 posts – i love pasta by Bob
Ok, so i hope this will illustrate the problem. According to bbPress USA, Pennsylvania, Scranton all have 0 posts in them, but that’s not true. They are parents in a forum sense, and have 13 topics and 142 posts under them.
The issue as a theme developer is that if you attempt to emulate the look/feel of any other forum software out there, then you have to make crazy complex SQL calls just to display the data that your users are used to. Or worse, (as i’ve shown in some real world examples) display “No Posts” on a forum that actually has loads of posts, just divided into their children.
I’m not saying that one way is better than another, but right now we’re stuck with only having data that fits one type of forum, and that’s one that has no need for parentage (such as this forum and the WordPress support forums).
Definition of Forums, Categories etc.
bbPress, in an attempt to roll in some backward compatibility of a plugin, included the “forum_is_category” meta. The issue here is that it’s a 1 to 1 relationship, a boolean, a yes/no; and yet we’re holding this data in the meta table, rather than the forum table.
What this means is that anyone pulling a list of the forums in a plugin, then has to compare each one to a different SQL against the meta table. Why isn’t their just a column in the forums table “is_a_category”? or even better, “forum_is_category”, and then no code would need to be changed in the existing “bb_forums()” loop. Why? well because if a forum doesn’t use categories because it has no need for parentage (such as this forum and the WordPress support forums) its not something they’re going to run into.
But of course, there is a bigger issue. A forum can only be a category or not a category. A Category is described (and categorised as) being Read Only and having Sub-Forums. The thing is, what if I want to make a forum read-only, but it already has child forums (for example, i want to make Philadelphia read only because i’ve divided the city into North/South as sub forums), would that mean that i want Philadelphia as a category? no. But that’s the bbPress definition.
bbPress has no way to add any form of settings or meta data to a forum. Forum A is treated the exact same way as forum B and forum Z, when in reality for forums rather than blogs, that’s not always the case.
There is no way to set any forum as read only, or hidden, or anything else. Forums don’t have settings, and why is that? because they’re exactly the same as categories in wordpress. Which suits the current need of one type of forum, and that’s one that has no need for parentage (such as this forum and the WordPress support forums).
A forum (n to n) not a blog (1 to n)
This is where the problem starts to expand. bbPress’s 1 to 1 reliance does not play well when attempting to assign other meta data. This really starts to throw issues as soon as your board becomes a certain size. Moderation is a nightmare because the whole n-to-n abilities of every other forum is non existent. Users to groups, groups to permissions on a forum per forum basic, rules on a forum per forum basis and per user and/or group.
Outside of WordPress integration the most common questions on this board are “how do i do X basic forum feature (read only, hidden forums, assign moderators etc)” and the answer is never good.
These are not issues for bbPress’s major role (the WordPress support forums), a forum with 5 sections, no parentage, and everyone is a WPuser already. But if you look around any of the “powered by bbPress” forums, and then look around any of the not-powered by bbPress forums, there are amazing differences. Those differences suit Automattic, and they may suit you (awesome if they do), but why do we continue to code/design/develop in a static flat-file based way that only meets the needs of one system, instead of assigning data in an N-to-N structure so that people can theme/develop/forum their own way?
My name is Kevinjohn Gallagher, and this is the 3rd February in a row i’ve written this (or v similar) post. I fully expect to be called an Angry Celt, “concern trolling” or whatever else is popular these days. In honesty, i’m holding my hands up to not being a great IA or Developer, and hoping that if i can raise these issues maybe we as a community can solve them.
Have a great day!
-
*life is too short
*
Hi John,
I’m not proposing anything right now (maybe in the long term), but it’s not my purpose.
Its really jsut one of the topics i wanted to discuss at “Bob’s no show of an IRC meet” (it’s now official title until we hear from bob – who i hope is ok btw). I’m not a developer, i can code, but i’m very poor at it. I am though not bad at the bigger picture stuff, and this year i’ve had to move 2 forums away from bbPress (we’re only 7 weeks in), and i’m not sure that by the end of the year my “convertin to” will be more than my “moved away from”.
I’m totally cool if people dont agree, and i’m cool if people do. I’ve been wrong many times before, and will be again
But, um, please right more than “report it in trac”
Cheers
Kind of mate
In fact, I’m sure that a lot if not everything bbPress related could be done with plugins, but it comes to a point in time where a certain number of plugins are needed as standard for the forum to function properly.
I mean, “forum_is_category” and “topic_voices” are in the core now, but rather than being in the core tables, are still added and fiddled around with as meta data, so there’s always an INNER JOIN needed or additional SQL calls.
Call up a topic, in SQL, simple. How many Posts? there’s the data. How many tags? there’s the data. Heck, last poster’s name? sure! How many voices? oh hell no! I don’t keep that data here. It’s a secret!! Even though it’s a one to one relationship and it’s in the core, we keep that somewhere totally different, and you need to come up with a totally different method of getting that info. Ha, i mean, just cos it’s in the core, doesn’t mean it’s laid out prof… well we don’t use it on the WP support forums.
I’m not suggesting anything overtly Machiavellian at all, please don’t assume that and if I’ve given that impression I’m sorry (sometimes i jokes dont translate well), but if you give the data-structure to a DBA or heck, a basic developer they’ll scratch their heads. There is no really good reason for something like “forum_is_category” or “topic_views” to be held in a separate table as a data row, other than the one major forum where it’s not used. And that’s ok, he who pays the piper calls the tune, but that’s a core issue.
Realistically, thats not a coding issue, it’s a perspective issue. In order to get at the data, we need to call additional database information that we don’t need. Why do we need that overhead, even if it’s small.
Deep down, I can’t believe I need a plugin for my forum to be able to say it has posts underneath it though. That isn’t a bug and it isn’t intended, it just wasn’t thought of.
Same with Mass deleting of users / topics / posts. Or the ability to be able to add any form of ordering or filtering to any of them. Just wasn’t thought of.
I can understand a plugin with a specific purpose in this field (I’ve written one that marks any “.ru” and “.cn” user as bozo automatically), that’s cool. But I still think it’s daft that I have to manually search the user list every day for members with certain words in it – that stuff became bog standard on the internet a long time ago.
You know, once i search for (and find) a user, it then takes me 7 clicks and 4 different pages to delete the user. And there is no en-mass option. 7 clicks and 4 pages!!!! That’s not a ‘plugin or not plugin’ issue, that’s no PM/BA/IA/common-sense issue. I mean, really, having to leave the admin area to interact with a user? They got rid of that idea in wordpress almost 3 years ago.
I don’t want to flog a dead horse, but now that WordPress isn’t being ported to backPress (at least in the next version minimum), can you imagine the difference that rolling in a moderation suite or really any of _ck_’s non-fluff (smilies, bbcode) plugins into the core would have made in the 14 months between 0.9 and 1.0 being released? Hey has anyone used XML_RPC to post to their forums yet? it’s awesome…
=====================================================================
Is the concern that since the data design isn’t optimized for inherited forums, the query count will go up?
Yeah, kind of. It’s more that it’s already gone up, and as themes become more complex, it’ll go up again.
Let me give an example, and it’s a walkthrough – there could be code optimization.
To load the last forum poster/post (and btw this is for the singualar, not the child, which would multiply this by each child forum), you effectively are looking up the topic with the last date that’s attatched to that forum, and then look up the last post that’s attatched to that topic. Ok, so that’s 2 additional SQL calls for every forum thats displayed in the forum list.
(EDIT: actually, I’m confident it’s just 1 INNER JOIN SQL call now, but it’s still not needed really).
Now on a forum like this, or on WP support forums, thats only 6 forums. ok, so 6 additional calls total. Meh, not fussed.
But do you see how this doesn’t scale? On one of my forums, a football one, we’ve a “forum” for every football team in the UK. There’s no “latest” list, as we want people to talk in their own club’s section. Thing is that’s:
England
– Prem
– – 22 teams
– (Another 4 divisions of 24 teams)
Scotland
– SPL
– – 12 teams
– (Another 2 divisions of 16 teams)
It was facing meltdown every now and then, and I realised why, I’m placing an additional 150 SQL calls every time I want to display the list of forums, simply to find out the last post in a forum (and that’s NOT including the code needed if I was to respect genealogy).
Here’s the thing though, the TOPIC table has a column called: topic_last_post_id. Do you know why? Because it allows an inner join rather than an additional SQL call. Very sensible. Of course, it just wasn’t transposed to the bit that a certain support forums didn’t use.
If we applied the same logic to “forums” as to the topics, heck even put it in the meta data table, we’d add 1 SQL command in total when a new post was added, instead of 1 SQL per forum listed per pageload per user. Now, i could write a plugin to do that, but it’s not solving the issue, it’s just papering over the cracks in the datastructure.
I’m sure there will be some people reading this and they wont get that that i’m talking about data structure and optomization, and not specifically about “last forum poster”. I’m not advocaating category heavy forums or flat file forums, i’m saying that we’ve got some real data holes, and i think we should discuss them. How long until i’m told to leave bbPress because on some of my websites I use it differently to someone who has relatively few uesers and so isn’t hitting any of these issues? I bet not long.
I am not in any way having a go at any current/ Past developer/ development team/ Santa nor the Easter bunny. But the data is set up in a way in which it is ideally suited for 1 type of forum, and not for any of the others. Thats cool, but if bbPress is going to be a WPplugin, then this doesn’t matter. If there’s going to be a standalone version, then this sort of thing really needs to be looked at.
There’s countless examples of this; I’m just going with the one that people can replicate easily.
=========================================
I’d like to stress i’m not advocating a massive change to the core, honestly I’m just up for discussion. I’m really just wanting to know people’s feelings or thoughts on this. Far too much data is not readily/intrinsically/obviously available especially when said data is on a predefined 1-to-1 relationship.
Maybe if the data structure wasn’t so wishy-washy at points, there might be more plugin developers, or themes that looked a little more like the themes available on any of the other forums.
=========================================
John you’re also right that a “moderation suite” could be done via a plugin, my thoughts on that though would be “XML-RPC is a standard feature, but moderating my forum is a 3rd party plugin”?
I don’t think that making Forums read only on a selective basis is the work of a plugin, thats integral to how a forum works. Same with certain forums being visible to certain people.
The ability to move/sticky/resolved a topic/post from inside the admin area isn’t the job of a plugin, thats the job of the admin area. otherwise, what the point of the admin area
I’m sure there are things that I think should be standard that many other don’t; and vice versa. I don’t want bloatware, i’m not suggesting we copy phpBB or anything else, but i also think that just because we can have a plugin to do a job that we should always have a plugin to do a job. Also, just because something works with the current datastructure, doesn’t mean it’s even close to optimal.
=================================================================
Phew, longer than i wanted. Probably wrote many things which people will get annoyed at. Then again, i’m sure some idiot will be along in a minute demanding that we stop talking about this and include Facebook Connect, Polls, Smileys etc. I wonder how long that will take before my data structure conversation gets it’s first “ZOMG i cant code so ignore data structurz & make me XXX feature”.
So my wish to BBPress is to keep it simple, and make the hard things even simpler.
The hard things right now is adding new features, polls and many of the plugins requires alot of work to plug in.
Yes, thats what we’re talking about
Things are harder right now, and we’re trying to discuss them to make them easier.
Why not make these into widgets instead, as a sidebar?
I would love to have a dynamic sidebar I could add or remove things with ease. Have polls on the sidebar, not arrested in a topic, and maybe a chatbox instead of having an “Off Topic” category or thread.
Who had under 2 minutes in the “ZOMG i cant code so ignore data structurz & make me XXX feature” poll?
Yeah, i’m going to hell, i understand. Well i tried.
I suppose what I might not have put across well is that I don’t want bbPress to become the monster that some of the other boards out there are. I’m very cool with lightweight and fast, it’s more that… bbPress has never had a “fixit” release y’know.
- 0.8 added some new functionality while removing alot of the hardcoded stuff (remember when you HAD to set the time displayed to the time zone you server was physically located in?).
- 0.9 added loads of new functionality
- 1.0 added backPress and new admin section.
- 1.1 is adding Anon posting and Email notification.
For me, having things like https://trac.bbpress.org/changeset/2377 sitting in trac (for the 2nd time btw) for over 5 months is crazy. Why do we have to have a year between releases?
I guess the way i’d put it is that there’s a difference between “basic” features and “standard” features in terms of forums. Simleys, BBcode, private messaging are fairly standard these days. I can absolutely live with them being a plugin. But the ability to delete a user from the backend inside the ‘3 click rule’ is a “basic” feature (why isn’t it one click?).
There’s a whole lot of places, especially in the backend, where there is legacy code or integrated plugin code that’s just made to work with 1.0. Take editing your forums placement, it uses an old AJAX routine from 0.8 to send to a page that’s not actually part of the current admin set up, and has no hooks/actions/filters/triggers. No plugin can interact with it. (I’m not against AJAX or the use of javascript, but there’s only 1 place in the whole admin area where there’s no actual posting back of info, and it just so happens to also be the one place where there’s hooks/action/fiters).
In my opinion, at some stage someone more intelligent than myself (not difficult) needs to have a look at bbPress on the whole, not in terms of X bits of code, but more in terms of the coding philosophy behind it. Not to make radical changes, but to make sure that it’s all going in the same direction.
==========================================================
Thanks, i’d be very keen to know how you got round that particular issue, i personally got round it by writing a plugin after abandoning _ck_’s approach in her plugin.
Finally,
That said, please don’t (even jokingly) refer to anyone as an idiot!
You’re right of course, apologises to anyone i did offend there. I write on these forums at about 3-4am my time as that’s the busiest time for the rest of [the board], and sometimes i forget to edit myself. I am sorry
- You must be logged in to reply to this topic.
