Export / Import through wp-cli for massive forum
-
Hi,
I’m trying to find a way to export and then import my forum
about 120k subjects and about 1 244K replies.We also need to be sure that accounts are imported and correctly linked to replies.
I saw a git project which is not anymore updated for 6 years, and linked to wp-cli/bbpress package that doesn’t exist anymore.
Do you have any solutions to solve this ?
-
how code savvy are you ?
About prompt cmd or PhP edition ?
php level.
Frankly it’s really not easy to move forums, topics and replies between sites if not moving the while site.
bbpress users custom post types, and manages the relationships between them via both post_parent and post meta. If you just export and import again, if the post ID frtom the source site is already being used by the destinatiion site, the the wp import process just allocates a free number, so the relationships gets all messed up.
ditto if the user ID’s do not exactly match, then again these get to be wrong.
I have done transfers as paid jobs for clients in the past, and have some code that does a few bits that get over this, but it is not really generic, so would need someone who knows WordPress database well and is happy with php. I am short on time at the moment to do the job.
I don’t have time to do a detailed answer right now, but what’s wrong mysqldump:
mysqldump -u db_username -p db_name dbprefix_posts --where="post_type='forum' OR post_type='topic' OR post_type='reply'" > bbp_content.sql
That will give you a db dump of all forum content.
Then within phpmyadmin or whatever, doing a select query to get all of the post IDs for that forum content (forums/topics/replies):
`
SELECTID
FROMdbprefix_posts
WHEREpost_type
=’forum’ ORpost_type
=’topic’ ORpost_type
=’reply’;
`
Export that query result from phpmyadmin which should give you a new query similar to this:
`
SELECTID
FROMdbprefix_posts
WHERE (dbprefix_posts
.ID
= 5) OR (dbprefix_posts
.ID
= 8) OR (dbprefix_posts
.ID
= 11) OR (dbprefix_posts
.ID
= 43) OR (dbprefix_posts
.ID
= 86) OR (dbprefix_posts
.ID
= 87) OR (dbprefix_posts
.ID
= 88) OR (dbprefix_posts
.ID
= 89) OR (dbprefix_posts
.ID
= 91) OR (dbprefix_posts
.ID
= 92) OR (dbprefix_posts
.ID
= 93) OR (dbprefix_posts
.ID
= 94) OR (dbprefix_posts
.ID
= 95) OR (dbprefix_posts
.ID
= 96) OR (dbprefix_posts
.ID
= 97) OR (dbprefix_posts
.ID
= 98) OR (dbprefix_posts
.ID
= 99) OR (dbprefix_posts
.ID
= 100) OR (dbprefix_posts
.ID
= 110) OR (dbprefix_posts
.ID
= 111) OR (dbprefix_posts
.ID
= 112) OR (dbprefix_posts
.ID
= 114) OR (dbprefix_posts
.ID
= 115) OR (dbprefix_posts
.ID
= 116) OR (dbprefix_posts
.ID
= 117);
`
Then you just need to do a little find/replace to re-do the query so that it grabs all of the post_meta for those forum posts (forums/topics/replies).
find all:
dbprefix_posts
replace with:dbprefix_postmeta
change the beginning: ‘SELECT
ID
FROM’
to: ‘SELECT * FROM’then find all remaining:
ID
replace with:post_id
That will give you a new query ready to dump the post_meta like this:
`
SELECT * FROMdbprefix_postmeta
WHERE (dbprefix_postmeta
.post_id
= 5) OR (dbprefix_postmeta
.post_id
= 8) OR (dbprefix_postmeta
.post_id
= 11) OR (dbprefix_postmeta
.post_id
= 43) OR (dbprefix_postmeta
.post_id
= 86) OR (dbprefix_postmeta
.post_id
= 87) OR (dbprefix_postmeta
.post_id
= 88) OR (dbprefix_postmeta
.post_id
= 89) OR (dbprefix_postmeta
.post_id
= 91) OR (dbprefix_postmeta
.post_id
= 92) OR (dbprefix_postmeta
.post_id
= 93) OR (dbprefix_postmeta
.post_id
= 94) OR (dbprefix_postmeta
.post_id
= 95) OR (dbprefix_postmeta
.post_id
= 96) OR (dbprefix_postmeta
.post_id
= 97) OR (dbprefix_postmeta
.post_id
= 98) OR (dbprefix_postmeta
.post_id
= 99) OR (dbprefix_postmeta
.post_id
= 100) OR (dbprefix_postmeta
.post_id
= 110) OR (dbprefix_postmeta
.post_id
= 111) OR (dbprefix_postmeta
.post_id
= 112) OR (dbprefix_postmeta
.post_id
= 114) OR (dbprefix_postmeta
.post_id
= 115) OR (dbprefix_postmeta
.post_id
= 116) OR (dbprefix_postmeta
.post_id
= 117);
`
You can then export all of the post_meta as sql dump file(s)
Then you just have to import those dump files into the new DB, and run the bbPress fixes to re-do counts and stuff like that.
NOTE: this approach will only work if you’re importing into a fresh DB where there are no posts with the same ID as the old DB. To import into a DB where there are existing/conflicting post IDs, a little extra work would have to be done.
Like I said, don’t have time to give you a “polished” script to run or anything, but this should point you in the right direction.
I will again reiterate, that any conflicting IDs will cause problems and extra steps are required, and like @robin-w said, that also includes user_ids
my recommendation would be to
on the destination site export any posts and pages (there are not usually that many) using dashboard>ttols>export. Then use phpmyadmin and move the whole source site to the destination site.
Then you can import the posts pages and add back any users unique to the destination site.
that way the linkage between users, topics and replies will remain in tact.
the fact that any old destination site page or post ID’s and user ID’s change should not matter
@tealcfr – what Robin W suggested is probably your best bet if you need it done immediately.
Considering that there are plenty of other topics on this very same thing and there’s been a 9 year old ticket for this very thing (https://bbpress.trac.wordpress.org/ticket/2605), your options are limited. A simple and logical approach without any code like Robin suggested above, or custom queries and perhaps some custom code like I suggested before that.
I’ve taken a look at /includes/converters/bbPress1.php and /includes/classes/class-bbp-converter-base and I think it’s realistic to make a bbPress2.php converter to use within the default bbPress importer (and would also solve the outstanding ticket #2605). If you don’t have the skills to do that, I’d consider doing it as an addition to the bbPress community, but I’d want you (and others) to test it before it gets submitted as a patch. I could have it ready for testing within a week or two. Not a good option if you’re in a rush.
@codejp3 the issue with the convertors is 2 fold.
1. it needs the users on both sites to be exactly the same user_ID’s
2. With a million replies it’ll fall over and need repeating/restarting – going the database route is the best option
- You must be logged in to reply to this topic.