Skip to:
Content
Pages
Categories
Search
Top
Bottom

Export / Import through wp-cli for massive forum

  • @tealcfr

    Participant

    Hi,

    I’m trying to find a way to export and then import my forum
    about 120k subjects and about 1 244K replies.

    We also need to be sure that accounts are imported and correctly linked to replies.

    I saw a git project which is not anymore updated for 6 years, and linked to wp-cli/bbpress package that doesn’t exist anymore.

    Do you have any solutions to solve this ?

Viewing 8 replies - 1 through 8 (of 8 total)
  • @robin-w

    Moderator

    how code savvy are you ?

    @tealcfr

    Participant

    About prompt cmd or PhP edition ?

    @robin-w

    Moderator

    php level.

    Frankly it’s really not easy to move forums, topics and replies between sites if not moving the while site.

    bbpress users custom post types, and manages the relationships between them via both post_parent and post meta. If you just export and import again, if the post ID frtom the source site is already being used by the destinatiion site, the the wp import process just allocates a free number, so the relationships gets all messed up.

    ditto if the user ID’s do not exactly match, then again these get to be wrong.

    I have done transfers as paid jobs for clients in the past, and have some code that does a few bits that get over this, but it is not really generic, so would need someone who knows WordPress database well and is happy with php. I am short on time at the moment to do the job.

    @codejp3

    Participant

    I don’t have time to do a detailed answer right now, but what’s wrong mysqldump:
    mysqldump -u db_username -p db_name dbprefix_posts --where="post_type='forum' OR post_type='topic' OR post_type='reply'" > bbp_content.sql

    That will give you a db dump of all forum content.

    Then within phpmyadmin or whatever, doing a select query to get all of the post IDs for that forum content (forums/topics/replies):
    `
    SELECT ID FROM dbprefix_posts WHERE post_type=’forum’ OR post_type=’topic’ OR post_type=’reply’;
    `

    Export that query result from phpmyadmin which should give you a new query similar to this:
    `
    SELECT ID FROM dbprefix_posts WHERE (dbprefix_posts.ID = 5) OR (dbprefix_posts.ID = 8) OR (dbprefix_posts.ID = 11) OR (dbprefix_posts.ID = 43) OR (dbprefix_posts.ID = 86) OR (dbprefix_posts.ID = 87) OR (dbprefix_posts.ID = 88) OR (dbprefix_posts.ID = 89) OR (dbprefix_posts.ID = 91) OR (dbprefix_posts.ID = 92) OR (dbprefix_posts.ID = 93) OR (dbprefix_posts.ID = 94) OR (dbprefix_posts.ID = 95) OR (dbprefix_posts.ID = 96) OR (dbprefix_posts.ID = 97) OR (dbprefix_posts.ID = 98) OR (dbprefix_posts.ID = 99) OR (dbprefix_posts.ID = 100) OR (dbprefix_posts.ID = 110) OR (dbprefix_posts.ID = 111) OR (dbprefix_posts.ID = 112) OR (dbprefix_posts.ID = 114) OR (dbprefix_posts.ID = 115) OR (dbprefix_posts.ID = 116) OR (dbprefix_posts.ID = 117);
    `

    Then you just need to do a little find/replace to re-do the query so that it grabs all of the post_meta for those forum posts (forums/topics/replies).

    find all: dbprefix_posts
    replace with: dbprefix_postmeta

    change the beginning: ‘SELECT ID FROM’
    to: ‘SELECT * FROM’

    then find all remaining: ID
    replace with: post_id

    That will give you a new query ready to dump the post_meta like this:
    `
    SELECT * FROM dbprefix_postmeta WHERE (dbprefix_postmeta.post_id = 5) OR (dbprefix_postmeta.post_id = 8) OR (dbprefix_postmeta.post_id = 11) OR (dbprefix_postmeta.post_id = 43) OR (dbprefix_postmeta.post_id = 86) OR (dbprefix_postmeta.post_id = 87) OR (dbprefix_postmeta.post_id = 88) OR (dbprefix_postmeta.post_id = 89) OR (dbprefix_postmeta.post_id = 91) OR (dbprefix_postmeta.post_id = 92) OR (dbprefix_postmeta.post_id = 93) OR (dbprefix_postmeta.post_id = 94) OR (dbprefix_postmeta.post_id = 95) OR (dbprefix_postmeta.post_id = 96) OR (dbprefix_postmeta.post_id = 97) OR (dbprefix_postmeta.post_id = 98) OR (dbprefix_postmeta.post_id = 99) OR (dbprefix_postmeta.post_id = 100) OR (dbprefix_postmeta.post_id = 110) OR (dbprefix_postmeta.post_id = 111) OR (dbprefix_postmeta.post_id = 112) OR (dbprefix_postmeta.post_id = 114) OR (dbprefix_postmeta.post_id = 115) OR (dbprefix_postmeta.post_id = 116) OR (dbprefix_postmeta.post_id = 117);
    `

    You can then export all of the post_meta as sql dump file(s)

    Then you just have to import those dump files into the new DB, and run the bbPress fixes to re-do counts and stuff like that.

    NOTE: this approach will only work if you’re importing into a fresh DB where there are no posts with the same ID as the old DB. To import into a DB where there are existing/conflicting post IDs, a little extra work would have to be done.

    Like I said, don’t have time to give you a “polished” script to run or anything, but this should point you in the right direction.

    @codejp3

    Participant

    I will again reiterate, that any conflicting IDs will cause problems and extra steps are required, and like @robin-w said, that also includes user_ids

    @robin-w

    Moderator

    my recommendation would be to

    on the destination site export any posts and pages (there are not usually that many) using dashboard>ttols>export. Then use phpmyadmin and move the whole source site to the destination site.

    Then you can import the posts pages and add back any users unique to the destination site.

    that way the linkage between users, topics and replies will remain in tact.

    the fact that any old destination site page or post ID’s and user ID’s change should not matter

    @codejp3

    Participant

    @tealcfr – what Robin W suggested is probably your best bet if you need it done immediately.

    Considering that there are plenty of other topics on this very same thing and there’s been a 9 year old ticket for this very thing (https://bbpress.trac.wordpress.org/ticket/2605), your options are limited. A simple and logical approach without any code like Robin suggested above, or custom queries and perhaps some custom code like I suggested before that.

    I’ve taken a look at /includes/converters/bbPress1.php and /includes/classes/class-bbp-converter-base and I think it’s realistic to make a bbPress2.php converter to use within the default bbPress importer (and would also solve the outstanding ticket #2605). If you don’t have the skills to do that, I’d consider doing it as an addition to the bbPress community, but I’d want you (and others) to test it before it gets submitted as a patch. I could have it ready for testing within a week or two. Not a good option if you’re in a rush.

    @robin-w

    Moderator

    @codejp3 the issue with the convertors is 2 fold.

    1. it needs the users on both sites to be exactly the same user_ID’s
    2. With a million replies it’ll fall over and need repeating/restarting – going the database route is the best option

Viewing 8 replies - 1 through 8 (of 8 total)
  • You must be logged in to reply to this topic.
Skip to toolbar