[mmice]

Hello! I’ve been importing forum from phpBB *the v.1, so converters are not working for me and i’m working through the database.
Well, – forums are showing, so do topics, – when i click on “Replies” in admin – i get no response from server. Yep, there’re about 7,5M replies there, but i guess it has to work despite of that fact?
Those replies are shown in frontend and forums seem to work well.

Another issue is: as i imported forums and topics to db – i could see them both listed at backend and frontend. But if i tried to come into the forum or topic on frontend i’ve got “not found”.
Open forum in backend, then just “save” helped that.
Bulk select topics, edit, changing nothing, just “save” helped that.

What could be the reason? I couldn’t find any changes in _posts table @db. Maybe bbpress uses some other tables for it? Just couldn’t find it out.

[mmice]

Still no idea?

[mmice]

I just ran a test, – tried to import just 2 replies (by DB), – see white page (No data received ERR_EMPTY_RESPONSE) @ the “Replies” section in admin.
After i ran “Recalculate the parent topic for each post”, “Recalculate the parent forum for each post”, “Recalculate the position of each reply” – problem get fixed.

Now i import all the 3M+ replies. I’m able to run only “Recalculate the parent topic for each post”, “Recalculate the parent forum for each post”. “Recalculate the position of each reply” fails with white page. The “Replies” section in admin is white (((

[Robkk]

hmmm you might need @netweb’s help on this one.

You might need to update phpBB (but not to the latest 3.0) but I am not entirely sure.

[mmice]

But i can’t! And, well, what’s the matter, – from wich version of phpBB do i import my forum, as i do it manually?

[Stephen Edgar]

Well, – forums are showing, so do topics, – when i click on “Replies” in admin – i get no response from server. Yep, there’re about 7,5M replies there, but i guess it has to work despite of that fact?

Those replies are shown in frontend and forums seem to work well.

This is because there is a slight data mismatch I expect, most likely around either post_parent in wp_posts or some of the post meta relationships in wp_postmeta.

And yes, even though they show on the front end correctly.

Another issue is: as i imported forums and topics to db – i could see them both listed at backend and frontend. But if i tried to come into the forum or topic on frontend i’ve got “not found”.

This is going to similar I think to the above, one or two of the meta relationships is not quite correct.

Open forum in backend, then just “save” helped that.
Bulk select topics, edit, changing nothing, just “save” helped that.

The affect of doing this is that when saving/updating forums, topics, replies we have some error checking that will validate that the ID is indeed the correct type that is be saved, so be it a replies parent topic, or a topics parent forum etc these get validated upon saving.

Thus this is kind of working as a pseudo repair tool repairing that meta I mentioned above.

I’ll add another reply in a moment…

[Stephen Edgar]

So your importing from phpBB v1?

Indeed our importers only support phpBB 3.x for bbPress 2.5.x and phpBB v3.1 for bbPress 2.6

Did you write your own phpBB v1 importer or copy and modify the one that is included with bbPress?

There are also some fixes and improvements in our repair tools coming for bbPress 2.6 that hopefully will be released in the next couple of weeks.

I’m happy to take a look at the importer if you made one and/or some data examples, you can post code examples on GitHub here https://gist.github.com/ntwb

(If you added the raw MySQL data from a single forum, a topic of that forum, and a reply of that topic I could give you some tips on what the main issues are that your most likely having)

[mmice]

Raw SQL, yep.
Here it goes:

EDIT: SQL code is now here https://gist.github.com/anonymous/8d1cb71e69d52ac69f5d

That’s not all, – i still have to implement a karma system to bbpress and also import it, and still have to import private messages. If you will be so kind to point me, what i’ve missed (and that cause error i’ve been experiencing), i appreciate it so much )

[Robkk]

@mmice

you should have posted that on gist.github.com or at least pastebin so that users do not have to scroll through all that.

As for importing have you tried a creating a custom import using the example one in the bbPress plugin
https://codex.bbpress.org/import-forums/custom-import/

And for Karma system you might have to look into a WordPress plugin like MyCred.

And for importing private messages , I am not sure how that is going to work, but there is a lot of Private messaging plugins for WordPress like BuddyPress.

[Stephen Edgar]

So I edited the post and copied the SQL code to a gist on GitHub:

https://gist.github.com/anonymous/8d1cb71e69d52ac69f5d

First, it looks like you are importing the phpBB into bbPress 1.x, bbPress 1.x is not supported anymore.

Per @robkk post immediately above you should checkout that link to our docs on the codex on importing to bbPress 2.x, and in general look toward using the bbPress 2.x plugin for WordPress https://wordpress.org/plugins/bbpress/

I’d suggest taking a look at our phPBB v3.x importer and making any change to match the phpBB v1.x database schema changes you would need:
https://bbpress.trac.wordpress.org/browser/branches/2.5/includes/admin/converters/phpBB.php

[mmice]

No, i have bbPress 2.5.7
Actually, first i tried to use your phPBB v3.x importer and rebuild it for phpBB 1.
But i gave up (just couldn’t understand all the points).

Anyway, – for this moment most problems are already resolved. The only question is:
Isn’t it enough to import replies to bb_posts table?

[Robkk]

@mmice

bbPress v2 does not have a bb_posts database table , that is in bbPress v1 standalone. Which is what @netweb is referring that you are trying to import into the old v1 version even though your suppose to import to the latest version.

All of bbPress data is stored inside of the WordPress database tables.

In this guide I explain it.

Stored Database Data

[mmice]

@robkk oh, G-d! I know that ))) I have been using a bb_ prefix for my WordPress install )))
So yep, – i have all posts in bb_posts (if i used wp_ prefix, it would be wp_posts).

[mmice]

I guess, by trouble could be caused by:

• i import nothing to _post_meta – what should i import there?

[Robkk]

@mmice

oh haha well thanks for clarifying that , easy mistake because really v1 did have that same db table name.

http://bbshowcase.org/forums/view/bbpress-mysql-reference

[mmice]

Maybe, there’s some SQL code to check my DB for consistency and to fix some relationship problems if they’re present?
I guess, for such a big number of replies it could be faster, than any php tool, and also more clear, as i’m already working through DB manually.

[Robkk]

I’m not really sure you have to worry about the post meta table?? Nevermind it is most likely important for post IDs and such.

https://codex.wordpress.org/Database_Description#Table:_wp_postmeta

And as for sql code to check DB for consistency , I do not know what to tell ya there either.

[mmice]

@robkk, so didn’t I. But in fact, – all replies are present in there right places with right authors and right post dates, i can also make new replies or edit old ones, except 1 little, but weird thing: i can’t open “replies” tab in backend ((

Maybe, i even do not need to, but the time my customer see that, – he asks, – WTF?

[Robkk]

@mmice

I edited my last reply so refresh to see it.

For right now I guess you are going to have to wait til @netweb comes back into this discussion because I am not really helping haha.

But yeah your client would have that reaction.

[mmice]

@robkk thank’s.
yep, looks like i’m waiting for @netweb )

@netweb, – let me clarify:

• i know, that forums, topics, replies, posts, attachments are all in one DB table (_posts)
• i’m sure, i’m not mistaken in parent id’s and author id’s when importing all that
• but i do nothing to the _post_meta table, guess, – that’s my problem

So, is there any way to fix _post_meta? If not, then which thing’s should i insert there manually to get success? )

[Stephen Edgar]

Ha… Using bb_ as your table prefix really did throw me 😉

If you look at lines 59-60 for your topics your selecting the forum_id from the phpbb_topics table and inserting it into post_parent

https://gist.github.com/anonymous/8d1cb71e69d52ac69f5d#file-gistfile1-sql-L59-L60

The same needs to happen for replies, so you should be able to get the topic_id that the reply is for and then insert that into the reply post_parent

https://gist.github.com/anonymous/8d1cb71e69d52ac69f5d#file-gistfile1-sql-L59-L60

The following should do it, assuming that topic_id is actually in the phpbb_posts_text table.

INSERT INTO
	bb_posts(post_title,post_excerpt,post_name,post_content,post_type,post_parent)
	SELECT post_subject, post_id, post_subject, REPLACE (post_text, CONCAT (":", bbcode_uid), ""), 'reply', topic_id FROM phpbb_posts_text;

[mmice]

I already did that!

As i import topics and replies, i store their old id’s in post_excerpt.
And then updating parent_id’s, depending on their new ones.

Any way, – parentness is all right in frontend. Backend stucks (

[Stephen Edgar]

Ok, run ONLY the following 3 repair tools: https://codex.bbpress.org/features/repair-forums/

* Recalculate the parent topic for each post
* Recalculate the parent forum for each post
* Recalculate last activity in each topic and forum

Do the replies look OK in the backend now?

If not then run this repair tool:
* Recalculate the position of each reply

How about now?

(The above should repair all the post_meta for you)

[mmice]

Recalculating the topic for each post … Complete!
Recalculating the forum for each post … Complete!
Recomputing latest post in every topic and forum… Complete!

Trying “replies” tab:
No data received
ERR_EMPTY_RESPONSE

Recalculate the position of each reply loads for a while and than get’s white screen.

P.S. I already wrote about that. When i try to import not millions, but just 2 replies for the test, – have the same trouble (replies tab doesn’t open), but “Recalculate the position of each reply” fixes the problem.
But it just don’t work for millions (

[mmice]

Is there a kinda SQL-version for Recalculate the position of each reply?
It could workout, i guess.

[Stephen Edgar]

I’ll go grab that code for you now, I’ve also got a patch that I’m trying to find that fixes this per the details below.

The thing is, the tool works as you noted that it works for when you’ve only got 2 replies, it fails on millions because the PHP code times out after 30 seconds, theoretically if you keep running that tool, click back after each white screen and hit go again you will eventually achieve said goal, with millions I would not advise this though.