[sdocpublishing]

Hi Stephen–
I’m taking you up on your invite to talk about phpBB imports.
In advance of upgrading a client’s site from (very VERY outdated WP and phpBB) I created a sandbox with WP 3.5.1 and phpBB 3.
There’s not a whole lot here, as I’m just using it to find the best way to integrate the bulletin board w/ the WP site, so the BB can be access-controlled for paid members.
Site: sdocplayground.com/wordpress
Imported forum: http://sdocplayground.com/wordpress/index.php/forums/
bbPress v 2.3 beta 2
Theme = twenty twelve
The phpBB install has about 1400 users and at least 50K posts and was created in mid-2004. My client wants to keep *everything*.

Here’s what happens when I run the import:
Default settings are left alone (pause after 100 lines, 1 second between, etc
All users are imported seamlessly.
When the import process gets to “calculating board hierarchy”, it hangs. I let the import run for an hour then check the results in the “forum” admin tab.
It appears that all of the posts have been imported. (The numbers in each of the categories is right.) But when you try to visit any of the forums/categories, they’re empty.
So then I run the “repair” functions one by one. Now it seems that most of the posts are gone and the most recent are from 2004. (Most recent should be from 11/2012). Some posts are visible but there are no responses where they should be.

Am I missing a step, or is some setting not correct, or is the beta version of the plugin just not doing what I need it to do? I would love to have all of my site’s content in one interface but I can’t find where the content goes. What am I missing?

[Stephen Edgar]

Cool…. Lets start fresh and see what happens….

Firstly make sure phpBB is running smoothly…

You state that phpBB has ‘1400 users and at least 50K posts’ though the phpBB forum itself states ‘Total posts 26878, Total topics 3468 & Total members 1263’ so there could be some issues with your actual phpBB install and/or database.

Run these checks which should clear up any potential database issues with phpBB from the phpBB admin tools (ACP) /phpbbv3/adm/index.php

• Resynchronise statistics
• Resynchronise post counts
• Resynchronise dotted topics

Now resync all of the phpBB forums which again should clear up any phpBB issues and in particular any issues for when bbPress starts to ‘calculating board hierarchy’

• Click the ‘Forums’ tab in phpBB ACP
• You should now see a list of category/parent forums
• For each one of these over to the right is an option to ‘resynchronize’ the forum
• Do this for each of the category/parent forums then drill down through each of those to any child forums and do the same for each

That should take care of any phpBB database issues…

Now go and try the import again with the same settings you have used previously but also check the following two options.

• Start Over – Start a fresh conversion from the beginning
• Purge Previous Import – Purge all information from a previously attempted import

If you believe the importer starts to hang at any point and is not moving along simply click ‘stop’ to pause the import process and give your webhost a bit of time to catch up on any other tasks that might be running and click ‘start’ to resume the import.

Don’t run any of the repair tools whilst the importer is importing and wait until the import has completed before running each of the tools individually.

So how did it go?

[sdocpublishing]

Good idea re: resyncing the database. I don’t think this DB was maintained properly and upgrading from phpbb 2.x to 3.0.11 may have introduced some errors.

Unfortunately, the outcomes were all the same. the import seemed to hang, although it did get further this time (converting forums 100 – 199), I used the stop/start suggestion, let it run for an hour, and it seemed that everything was imported. Then when you go to view a forum or category or topic, and most of it is missing. Most recent posts were from 2004. This was before running any repairs. Running the repair functions depleted the counts further.

I’m not sure I can troubleshoot this in time for my client’s deadline but if further suggestions or upgrades come up, I’m game for trying them out in the future.

[Stephen Edgar]

Good idea re: resyncing the database. I don’t think this DB was maintained properly and upgrading from phpbb 2.x to 3.0.11 may have introduced some errors.

Does phpBB now have your “50K+” posts after the repair or still ~27K?

Unfortunately, the outcomes were all the same. the import seemed to hang, although it did get further this time (converting forums 100 – 199)

You only had ~10-15 phpBB forums when I looked earlier and it now looks like you have removed phpBB from your site so I can no longer confirm this.

The last message you should have seen during the import ‘forums’ step should have been `(converting forums 0 – 99)` before it moved onto the next step `Calculating forum hierarchy (0 – 99)`.

I used the stop/start suggestion, let it run for an hour, and it seemed that everything was imported.

Why do you presume that? Did you see your 50K+ posts somewhere?

Then when you go to view a forum or category or topic, and most of it is missing. Most recent posts were from 2004.

I stated in my previous post any counts you see before running the repair tools will be inaccurate.

This was before running any repairs. Running the repair functions depleted the counts further.

I’ll reiterate this again, any counts you see before running the repair tools after the import is finished are inaccurate and should not be relied upon.

The repair tools nor any part of bbPress will delete content from your WordPress database unless you explicitly do this via ‘forum reset’ or ‘purge previous import’.

I’m not sure I can troubleshoot this in time for my client’s deadline but if further suggestions or upgrades come up, I’m game for trying them out in the future.

At this stage yours is the ONLY phpBB forum that has NOT been successfully imported, there are virtually no future upgrades planned for the phpBB importer because they are not needed as it has thus far worked with every single phpBB import that I have been asked to test or help out with.

The output of the converter should be similar to this (They should match your phpBB stats to the nearest 100)
Did the import end up looking like this below? If not which step did it stop at?
`
Repair any missing information: Continue
Conversion Complete
Converting replies (2000 – 2099)
…
Converting replies (100 – 199)
Converting replies (0 – 99)
Converting topics (1200 – 1299)
…
Converting topics (200 – 299)
Converting topics (100 – 199)
Converting topics (0 – 99)
Calculating forum hierarchy (0 – 99)
Converting forums (0 – 99)
No data to clean
Starting Conversion
`

As I stated in the comments of the review you posted for the bbPress plugin at WordPress.org I am more than happy to take a look and help you get it imported.

If there is a bug with the importer and why your phpBB is not playing nice like all the other phpBB forum imports I can write a patch to fix it but I will need accurate detailed information from you to do this.

Cheers,

Stephen

[Bryan Eggers]

I just imported a phpBB forum of 2000 members and 100,000 posts.  I was getting an error that MySQL died so I changed a couple of settings in my.cnf to keep the connection alive.  You can also change the default wait time in the importer from 1 to 3 to give everything time to catch up.

Unfortunately, the forums get imported in seemingly random order so you’ll have to edit each forum and use the Order field to display them properly.

If you’ve already run the importer once, check to see if you’ve already imported the members.  Clearing the previous attempt doesn’t clear the members you imported, only the forum data.  So if you run the entire import again you’ll get duplicate usernames (like Joe and ImportedJoe) and all the forum links will be wrong.  I used a plugin in that lets you mass delete members, and then started over from scratch.  Honestly, it took me several times to get it right.

[Stephen Edgar]

Thanks for feedback @beggers, glad its worked for you 🙂

The forum order is already fixed for upcoming bbPress 2.3 with lots of extra tweaks also… storing old phpBB user profile fields.. ‘all the BBCodes’ are converted bbPress HTML…

[webp]

After removing almost half of the Drupal 6 spam and blocked uses, I still have over 250K users to import to WP/bbPress.

I run a prior test import for the topics, and it seemed to run fine. But importing users one at a time hangs after every ~one hundred users. I have to manually click stop and start at the Converter Admin page every few minutes. If I can’t find other solutions, I might want to install a Browser addon to click stop and start for me every 3 minutes when I’m away. Otherwise, I’m at 2500 users now and I still have 247K users to go! It won’t finish for weeks.

Suggestions? Thanks.

[webp]

Found the solution.
In /etc/mysql/my.cnf (/root/my2.cnf for me since it’s my second temp mysql instance for me), I changed wait_timeout from 600 to:

wait_timeout = 600000000000000

I found I read somewhere else in this forum that says the same, but I couldn’t find it again.
Please refer to: MySQL wait_timeout and WordPress

• wait_timeout is the time mysql will hold a non-interactive connection open for before closing it basically.
• It’s the number of seconds of inactivity on a single connection mysql will wait for before terminating a connection. For instance you may make a connection 1 second into script execution then go away and do 40 seconds of work in php and when you go back to use your connection again, mysql has killed it.

[webp]

Adding the wait_timeout above doubled the run time to maybe 6 minutes, but still hangs after ~ 6 minutes.

Per stackoverflow.com/questions/18253934/set-maximum-execution-time-in-mysql-php , adding the following to to the beginning of converter.php did not help:

ini_set('max_execution_time', 0);

Ah…

[webp]

I installed iMacros FireFox addon and did a simple loop to click stop and start every 5 minutes. I could have use Python to do this job but simple 3 line Macro is easiest. Now it’s been running over 12 hours with over 30K users imported.