[Tom Dyer]

Hi,

After reading the importer
I’ve recently migrated my phpBB forum to bbPress, but I have noted a few issues with the import/repair process. Does anybody have any suggestions as to what’s going wrong with any/all of these:

Forum import – The documentation states that stickies will not be imported, however I after doing the import I have around 20 ‘super-stickies’ which, looking at the old forum, are a mixture of announcements and stickies. I’d rather these weren’t converted to anything, and as far as I’m aware they shouldn’t be (https://bbpress.trac.wordpress.org/ticket/2126)

User import – During the import, I (the main WordPress admin user) seem to have had quite a lot of forums, topics and replies linked to my user. I wasn’t a user on the previous phpBB forum at all so I’m not sure why I’ve adopted these posts. Or any posts for that matter.

Repair tools – All the repair tasks seem to work ok, except for two:
– Recalculate the position of each reply
– Remap existing users to default forum roles
These two tasks end up white-screening with no status messages at all.

User Login Conversion – User logins are converted correctly when using the standard wordpress login screen but now when using my custom frontend ajax login – does anyone know if there’s a way to get that to work? A seperate topic has been started about that here: http://bbpress.org/forums/topic/user-login-authentication-migrated-phpbb-users/

Any advice on any of these steps would be very much appreciated.
Thanks!

[Stephen Edgar]

Cool, whack your issues here and I’ll check than out tomorrow.

Quick answers:

– stickies are now converted so I’ll update the docs for that.

– Repair tools, as I stated in the other thread shouldn’t do this, will look closer tomorrow

– Users is odd, seems maybe some users aren’t getting imported, job for tomorrow also

– User login, bbPress doesn’t actually convert the phpBB password until the migrated user logs in for the first time. This will most likely be the issue in that the ajax login is stepping around this. Presumably after an imported user has logged in and their password converted the Ajax login will work.

[Tom Dyer]

Thanks,

Stickies – is there a way to ignore stickies during the import? They don’t appear to have come through correctly so it would be nice to simply ignore them (as was previously the case).

Users – looking at the phpbb data, the amount of imported users seemed to match up, but I wonder if I’ve been assigned posts that were previously changed to the ‘Anonymous User’ that phpBB has at user ID 0… the amount of posts don’t seem to match up with that theory but a ‘majority’ of the posts that have been assigned to me seem to match the logic to an extent (ie, posts that have been moved into a ‘moderators only’ forum for further discussion about actions to be taken).

User Login – that’s correct – I’m just using the wp-signon method so I guess I’ll need to add some kind of do_action() call or something to trigger the conversion? If you can point me in the right direction, that would be great 🙂

Thanks for your help

[Stephen Edgar]

Forum import – The documentation states that stickies will not be imported, however I after doing the import I have around 20 ‘super-stickies’ which, looking at the old forum, are a mixture of announcements and stickies. I’d rather these weren’t converted to anything, and as far as I’m aware they shouldn’t be (https://bbpress.trac.wordpress.org/ticket/2126)

The above ticket you reference was the ‘issue’ being created to track the changes and progress of implementing this feature.

In comment 12 that is where the ticket was to updated to ‘fixed’ and ‘closed’ via changeset r5170. Trac can be a little daunting at first trying to understand everything that goes on in a ticket 😉

With only your ~20 ‘stcikies’ it shouldn’t take more than a couple of minutes to ‘unstick’ the ones you do not want after your import has finished and the repair tools run.

I have updated the docs to reflect that also https://codex.bbpress.org/import-forums/phpbb/

Stickies – is there a way to ignore stickies during the import? They don’t appear to have come through correctly so it would be nice to simply ignore them (as was previously the case).

How so, could you explaing that a little more please? Do the topics ‘look right’ and are just not ‘stuck’ or something else going on?

User import – During the import, I (the main WordPress admin user) seem to have had quite a lot of forums, topics and replies linked to my user. I wasn’t a user on the previous phpBB forum at all so I’m not sure why I’ve adopted these posts. Or any posts for that matter.

If there is a username match the importer will try to match the posts being imported with any current WordPress user, thus if phpBB has a user named ‘admin’ and you use the default the default WordPress username ‘admin’ that will be the cause.

The best work around for this would be to rename the phpBB username in the database bvefore importing.

Users – looking at the phpbb data, the amount of imported users seemed to match up, but I wonder if I’ve been assigned posts that were previously changed to the ‘Anonymous User’ that phpBB has at user ID 0… the amount of posts don’t seem to match up with that theory but a ‘majority’ of the posts that have been assigned to me seem to match the logic to an extent (ie, posts that have been moved into a ‘moderators only’ forum for further discussion about actions to be taken).

You should notice that you now actually have a WordPress username ‘Anonymous’, so any ‘Anonymous’ posts should be attributed to that user.

Your ‘moderators only’ forum after import you should change the bbPress forum permission to ‘hidden’ and that will make that forum the equiv bbPress ‘moderators only’ forum.

Repair tools – All the repair tasks seem to work ok, except for two:
– Recalculate the position of each reply
– Remap existing users to default forum roles
These two tasks end up white-screening with no status messages at all.

What size is the phpBB database your importing? (Approx. Forums, Topics, Posts & Users please 🙂 )

User Login Conversion – User logins are converted correctly when using the standard wordpress login screen but now when using my custom frontend ajax login – does anyone know if there’s a way to get that to work? A seperate topic has been started about that here: https://bbpress.org/forums/topic/user-login-authentication-migrated-phpbb-users/

I’ve pretty much still got the same opinion as I wrote last night here.

User Login – that’s correct – I’m just using the wp-signon method so I guess I’ll need to add some kind of do_action() call or something to trigger the conversion? If you can point me in the right direction, that would be great 🙂

There is nothing to point you to sorry, when a user logs in for the first time after the import their legacy salt and hash password get’s converted to the native WordPress password format. Without knowing each legacy password their is no way to do this programmatically.

bbPress’ login forms actually are hooked into the WordPress login forms, so theoretically it shouldn’t be too difficult for either the plugin you are using to add support for bbPress or possibly look for another similar plugin that already supports bbPress.

Passwords and all their associated algorithms are hard and this an area where I just manage to skate by with the basics, actually doing any of what I mentioned in the previous paragraph is way beyond my skill level and pay grade 😉

[Tom Dyer]

Hi @netweb, thanks for the response. I’ve gathered some more information to elaborate on the issues…

Stickies – the issue here seems to be that the phpBB stickies and announcements have all been imported as ‘super stickies’. There is actually a lot more than 20 posts (my estimate in the original post was waaay out – there was actually about 100 when I went in to un-stick them). These might have been fine had they imported as normal stickies – but having them as super-stickies had a hug impact on forum pages load times.

User import – I thought at the time that it was probably a conflict with username and/or ID but having looked at the posts I’ve adopted, they are all from different users. I have 62 forum topics/replies under my name but looking at the live site, the original authors of these are all different users which makes no sense to me. What’s more is, I haven’t adopted all of the users posts, just like one or two from each user…

…Following on, I now wonder if these are posts that should have been assigned to the ‘Anonymous’ user because I think I may have deleted that user at some point not realising it was a result of the import… I’ll add it back in and see if it fixes my problem.

I didn’t realise that adding a moderators only forum was that easy… I’ve been working with RobinW to integrate the bbp-private-groups plugin, maybe I don’t need that after all.

Repair Tools – our forum has imported 72 Forums, 71,007 Topics, and 564,453 Replies. It’s a fairly large one, but I think I’ve heard of larger forums being imported into bbPress…

User login – I haven’t used a plugin for this, I’ve written it myself. It just does a simple ajax call using the admin-ajax and then calls wp_signon() to attempt to log you in. It’s a fairly simple one, I just need to know which hooks bbPress is using to run the conversions on and hopefully I can just call that action before/after trying to log them in.

[Stephen Edgar]

Stickies – the issue here seems to be that the phpBB stickies and announcements have all been imported as ‘super stickies’. There is actually a lot more than 20 posts (my estimate in the original post was waaay out – there was actually about 100 when I went in to un-stick them). These might have been fine had they imported as normal stickies – but having them as super-stickies had a hug impact on forum pages load times.

Hmmmm… I see where your coming from, not sure what to do about this though, I think we should continue to support ‘stickies’ as part of the import process, I’ll have a think about this some more.

User import – I thought at the time that it was probably a conflict with username and/or ID but having looked at the posts I’ve adopted, they are all from different users. I have 62 forum topics/replies under my name but looking at the live site, the original authors of these are all different users which makes no sense to me. What’s more is, I haven’t adopted all of the users posts, just like one or two from each user…

I think this will come down to ‘data integrity’ the queries are quite accurate so if the results are inconsistent I can only really think that.

Can you check via MySQL a couple of those if your happy playing with MySQL that is?

…Following on, I now wonder if these are posts that should have been assigned to the ‘Anonymous’ user because I think I may have deleted that user at some point not realising it was a result of the import… I’ll add it back in and see if it fixes my problem.

That is one possibility though if as you say they are inconsistent a bit more debugging will be needed. Adding the user ‘back in’ so to speak most likely won’t help as the existing topics/replies are already assigned to the admin user.

I didn’t realise that adding a moderators only forum was that easy… I’ve been working with RobinW to integrate the bbp-private-groups plugin, maybe I don’t need that after all.

I didn’t realise from those posts this is what your intentions were 😉

Repair Tools – our forum has imported 72 Forums, 71,007 Topics, and 564,453 Replies. It’s a fairly large one, but I think I’ve heard of larger forums being imported into bbPress…

Yes, there have been a handful (I can count them on one hand ) larger that this that I know of, but again theoretically it should not be an issue.

That said because phpBB does not support ‘threaded replies’ there is nothing to be gained by running the repair tool ‘Recalculate the position of each reply’, so for now just avoid that tool.

User login – I haven’t used a plugin for this, I’ve written it myself. It just does a simple ajax call using the admin-ajax and then calls wp_signon() to attempt to log you in. It’s a fairly simple one, I just need to know which hooks bbPress is using to run the conversions on and hopefully I can just call that action before/after trying to log them in.

I’ll put this on my ‘todo’ list, which is open up bbPress source and read the inline docs, bbPress is extremely well documented source code, so feel free to beat me to that. 😉

[Stephen Edgar]

Aside from what I wrote above I have spend most of today fixing a couple of niggling issues with the main converter.php that will cause some odd data import validation for users with a particular setup. This doesn’t affect your imports though because if it did you would not have had any stickies 😉

I didn’t know these bugs were there until today as I am typically not affected by the issue either. I hope to get these patched tomorrow so I can continue working on what I had actually planned for the day.

I am adding support to phpBB and SMF imports to fully support users who have been a) deleted, b) anonymous or c) guests users from these forums. They will be imported as bbPress ‘Anonymous Users’ keeping any legacy username with posts correctly attributed to these users.

If you willing to check a couple of the integrity issues I mentioned above we can make sure that the phpBB importer is doing what it is supposed to and ‘data integrity’ is all good and you want to do the import with the above support included I’ll add some links back here on where to get the bits for this so you can add them to your setup and have another go at a ‘clean import’ with the above support.

[Tom Dyer]

Sure thing, I’ll see what I can dig out. I’m going to be running the import a few more times over the coming weeks anyway because I need to make sure that I have the process down to a t before we decide to go live. The plan of action is to shut down the existing forum 48 hours before go live. That *should* give me time to grab a dump from the forum and import it into the new site before we switch it on…. that’s the plan anyway, I can only hope it goes as smoothly as that! lol

I’m happy to poke around with MySQL and PHP, I’ll see what I can find.

One other thing, if I ‘Reset’ my would that be enough to start an import from scratch without running into any issues? And can the reset script ‘remove imported users’ if the wp_bbp_converter_translator is missing? Not entirely sure where it’s gone – I certainly wouldn’t have deleted it myself, I was hoping to use it to setup a redirect from the old forum topics…

[Stephen Edgar]

Thanks extra sets of eyes with all of this ‘import’ stuff is always handy, boat loads of data all just starts to look the same after a while 😉

Indeed, ‘Reset Forums’ will remove everything bbPress, as long as you also check the ‘Delete Imported Users’ they will also be removed.

The wp_bbp_converter_translator is not explicitly required, if your site can create this table bbPress uses it as part of the import process, if not it will just use the ‘out of the box’ WordPress database tables. It also gets deleted when resetting the forums. It also relates to the bugs I found yesterday that I am working on fixing and as I stated above I am pretty sure your imports are not using wp_bbp_converter_translator as if you were you wouldn’t have had any stickies.

As to redirects you shouldn’t need that table for those, depending on your old phpBB permalinks
some simple right up to some advanced .htaccess will get you what you want.

A week or so back whilst messing about with splitting and merging topics I happened upon a WordPress feature for 301 redirection of old URL’s to new URL’s. I am yet to try this as it has only popped into my head as I wrote the previous paragraph that this may be something that could work so that imported topics and replies include full redirect support.

So much so I just gave it a whirl, if I store and old topic slug e.g. viewtopic.php?f=3&t=9 I can query on the fly that topics new post ID and 301 redirect to it’s new permalink e.g. phpbb-v3-standard-topic 😉

It still doesn’t quite work though due to some URL weirdness presumably with one or a combination of the .php, ? and & characters, I am quite sure we can do something here though 🙂

[Tom Dyer]

@netweb – I’m happy to give it a try, but the problem(s) I think I’ll face are:

1) The ID’s for the Forums, Topics and Replies are all different now that I’ve imported them into bbPress. This is because we’ve had to migrate a whole site over too which meant that around 5000 ID’s were already taken by other posts.

2) The migrated data for bbPress has 68 Forums, 71,006 Topics, and 564,453 Replies so it’s going to be a big job to set up redirects for these individually.

What I was thinking of doing, was grabbing the migration helper data, and writing a PHP script that would look up these old forum and topic ID’s and redirect to the correct place without me having to input any manual redirects.

You mention that if there is no ‘wp_bbp_converter_translator’ table, then it uses the built in WordPress tables – do you know which ones I should be looking in? I’d presume it to be wp_postmeta or something for these particular bits?

[Stephen Edgar]

You wrote in your previous post:

Sure thing, I’ll see what I can dig out. I’m going to be running the import a few more times over the coming weeks anyway because I need to make sure that I have the process down to a t before we decide to go live.

Does this mean you have now gone live and a ‘few more imports’ has already happened and ‘coming weeks’ is time flies when having fun?

You should find in your wp_postmeta table primarily two reference meta keys, _bbp_old_forum_id and _bbp_old_topic_id

In the case below the new forum ID in wp_posts is 8789 for the old phpBB forum ID 53 and the new topic id in in wp_posts is 8803 for the old phpBB topic id 25743

meta_id	post_id	meta_key	        meta_value	
57514	8789	_bbp_old_forum_id	53
57626	8803	_bbp_old_topic_id	25743

With the above data in hand you should be able to map all your old phpBB forums and topic ID’s to the new bbPress forum and topic ID’s.

[Stephen Edgar]

Oh…. And you should also be able to map the Anonymous/Deleted phpBB users topics and replies also via those same mappings, I’ve just finished adding support for this with phpBB and bbPress around an hour ago.

I’ll come back tomorrow and show you the SQL query for that, for now it’s late here and I’m done for the night, anything else I will look at tomorrow 🙂

[Tom Dyer]

No we haven’t gone live yet – I’m still developing the new site and haven’t got around to doing another import test yet. I’ve got plenty of tasks to complete before I carry on with the forum so I’ll be using an updated version of bbpress by that time I’d imagine. When you say you’ve started adding support for this, are you talking about the redirects in particular?
Cheers

[Stephen Edgar]

I added support for the anonymous/deleted phpBB user topics and replies, just finalizing all of this for the any other forums that also have this feature, SMF, Drupal, Vanilla.

For the redirects I need to find a workaround to have WordPress correctly verify that viewtopic.php?f=3&t=9 is a valid URL, if we can do that then redirects could be included for imported forums and that would be awesome.