Skip to:
Content
Pages
Categories
Search
Top
Bottom

Help with importing from WBB3.1

  • @inspirationally

    Participant

    Hello,

    after exactly 10 years and over 500.000 posts with our board deppheads.com running with Woltlab Burning Board (right now version 3.1.8), social networks have sort of won – instead of 150 new posts the day we have now maximum 5 posts. WBB4 will be coming out soon, so this is the right time to sell my WBB license and move it all to my site johnny-depp.org, powered by WordPress. This is where I seek help.

    I already deleted inactive members, so that we now have “just” over 300, but I don’t want to delete all the written things like movie reviews and serious discussion, as this is still the home of many. So there are 500.000 posts and nearly 10.000 topics to be imported.

    I edited the example.php and so far it worked with importing members and forums (just on a test WordPress installation for now).

    However, there are some problems/questions I’ve come up with and I’m almost sure, there will be more following.

    1. there is, equally to phpBB, the problem, that there are topics and posts, and the first post of the topic has to become “topic content” of WBB.
    Here I used

    // Topic content.
    // Note: We join the posts table because topics do not have content.
    $this->field_map[] = array(
    'from_tablename' => 'wbb1_1_post',
    'from_fieldname' => 'message',
    'join_tablename' => 'topics',
    'join_type' => 'INNER',
    'join_expression' => 'USING (threadID) WHERE wbb1_1_post.postID = wbb1_1_thread.firstPostID',
    'to_type' => 'topic',
    'to_fieldname' => 'post_content',
    'callback_method' => 'callback_html'
    );

    Unfortunately, I get the error, that wbb1_1_post.message does not exist:

    WordPress database error: [Unknown column 'wbb1_1_post.message' in 'field list']
SELECT convert(wbb1_1_thread.threadID USING "utf8") AS threadID,convert(wbb1_1_thread.replies USING "utf8") AS replies,convert(wbb1_1_thread.boardID USING "utf8") AS boardID,convert(wbb1_1_thread.userID USING "utf8") AS userID,convert(wbb1_1_post.message USING "utf8") AS message,convert(wbb1_1_thread.topic USING "utf8") AS topic,convert(wbb1_1_thread.time USING "utf8") AS time,convert(wbb1_1_thread.lastPostTime USING "utf8") AS lastPostTime FROM wbb1_1_thread AS wbb1_1_thread LIMIT 0, 100

    But it does!
    I attached both, my wbb3.php import script and the database structure of the most important tables, here you go:
    http://deppheads.com/uploads/wbb.txt
    http://deppheads.com/uploads/database_structure.txt

    2. I hope/guess, this is also the reason that all topics (once I remove that passage that produces an error) are started either by Anonymous or by ME (although I haven’t done it)?

    3. some users are twice now (like myself..who have been admins of wordpress and users of the forum). Can I simply merge them with
    UPDATE wp_posts SET post_author = ‘1’ WHERE post_author = ‘2’; and it will count for the forum, too?

    4. the passwords – in WBB they are hashed by this:
    $passwort = sha1($row->salt.sha1($row->salt.sha1($_POST[‘passwort’])));
    so what do I write at the end of wbb3.php
    public function authenticate_pass( $password, $serialized_pass ) {
    $pass_array = unserialize( $serialized_pass );
    return ( $pass_array[‘hash’] == md5( md5( $password ). $pass_array[‘salt’] ) );
    }
    or is this impossible and the users have to get the new password by mail?

    5. the imported users have no “Forum Role”. Is is “normal”? Is there any chance to automatically give them the forum role “Participant”?

    6. do I have to keep the wbb3.php “forever” because of the passwords or can I delete it one day, even if not all members have com back yet?

    7. until now we have everything like private messages, profiles, friends and especially user groups (Moderators, crew members) with hidden forums. Do I see correct, that I can install both, import bbpress and afterwards attach for example our “Mods Only” to the group “Moderators” as group forum?

    8. I see that every user gets a new User ID – old IDs are saved in table _bbp_converter_translator – will that table be kept? Maybe by this I have a chance to later import – somehow – private messages for buddy press!?

    Alone if my first point is solved, I’d be thankful.

    Kind regards, inspirationally.

Viewing 16 replies - 1 through 16 (of 16 total)
  • @inspirationally

    Participant

    Allright, I could at least solve #1 and thus #2 on my own. I just had that one line wrong = topics. I also had to add wbb1_1_thread there.

    @netweb

    Keymaster

    3. some users are twice now (like myself..who have been admins of wordpress and users of the forum). Can I simply merge them with
    UPDATE wp_posts SET post_author = ’1′ WHERE post_author = ’2′; and it will count for the forum, too?

    If a user already exists in WordPress during the import a new user will be created called ‘Imported_Username’, you can go to wp-admins user panel and when you select ‘Delete User’ you will be prompted to delete all posts or attribute them to another user, select the later and all the topics/replies etc will be attributed to the user you selected.

    4. the passwords – in WBB they are hashed by this:
    $passwort = sha1($row->salt.sha1($row->salt.sha1($_POST[‘passwort’])));
    so what do I write at the end of wbb3.php
    public function authenticate_pass( $password, $serialized_pass ) {
    $pass_array = unserialize( $serialized_pass );
    return ( $pass_array[‘hash’] == md5( md5( $password ). $pass_array[‘salt’] ) );
    }
    or is this impossible and the users have to get the new password by mail?

    I am going to look at passwords next week for all the importers as my skills with PHP are only just beginning so this is all a bit tricky for me at the moment. 😉

    So the best workaround is to have the users reset their password by email for now.

    5. the imported users have no “Forum Role”. Is is “normal”? Is there any chance to automatically give them the forum role “Participant”?

    Go to bbPress tools and run the repair tool ‘Remap existing users to default forum roles’ and that will do the trick. https://codex.bbpress.org/repair-forums/

    6. do I have to keep the wbb3.php “forever” because of the passwords or can I delete it one day, even if not all members have com back yet?

    If the passwords are not being converted you do not need to keep it, though I don’t see any issues why you would need to delete it.

    Speaking of keeping it, can we include your WBB v3 Importer and distribute it with bbPress?

    7. until now we have everything like private messages, profiles, friends and especially user groups (Moderators, crew members) with hidden forums. Do I see correct, that I can install both, import bbpress and afterwards attach for example our “Mods Only” to the group “Moderators” as group forum?

    Currently bbPress only has: https://codex.bbpress.org/getting-started-with-bbpress/#creating-your-first-forum

    Visibility: decide whether your forum is public, private or hidden.
    Public – Anyone can see these forums
    Private – Only logged in registered users can see these forums
    Hidden: Only Moderators/Admins can see these forums

    I see that every user gets a new User ID – old IDs are saved in table _bbp_converter_translator – will that table be kept? Maybe by this I have a chance to later import – somehow – private messages for buddy press!?

    The translator table is not kept and the old user ID is stored in wp_usermeta as per this item in the ‘Example.php’ importer.

    https://bbpress.trac.wordpress.org/browser/trunk/includes/admin/converters/Example.php#L458

                    // Store old User id (Stored in usermeta)
    459	                $this->field_map[] = array(
    460	                        'from_tablename'  => 'users_table',
    461	                        'from_fieldname'  => 'the_users_id',
    462	                        'to_type'         => 'user',
    463	                        'to_fieldname'    => '_bbp_user_id'
    464	                );

    I’ll take closer look at the importer and structure you linked to over the weekend and post back with anything else I can think of.

    @inspirationally

    Participant

    Thank you, Stephen. Sure it can be used for everyone later, but please don’t download/upload yet, as not everything works as it should. Now that I finished import and ran the repair tool, the posts are not part of the topics. Still, much to do!

    @inspirationally

    Participant

    Allright, another thing I need help with:

    In WBB, there's the field isClosed to create a WordPress status from it with this:
    // Topic status (Open or Closed)
    		$this->field_map[] = array(
    			'from_tablename'  => 'wbb1_1_thread',
    			'from_fieldname'  => 'isClosed',
    			'to_type'         => 'topic',
    			'to_fieldname'    => 'post_status',
    			'callback_method' => 'callback_topic_status'
    		);

    and

    public function callback_topic_status( $status = 0 ) {
    		switch ( $status ) {
    			case 1 :
    				$status = 'closed';
    				break;
    
    			case 0  :
    			default :
    				$status = 'publish';
    				break;
    		}
    		return $status;
    	}

    But there’s also a field called “isDeleted”, so I could also use

    `// Topic status (Deleted)
    $this->field_map[] = array(
    ‘from_tablename’ => ‘wbb1_1_thread’,
    ‘from_fieldname’ => ‘isDeleted’,
    ‘to_type’ => ‘topic’,
    ‘to_fieldname’ => ‘post_status’,
    ‘callback_method’ => ‘callback_trash_status’
    );`

    and

    public function callback_trash_status( $status = 1 ) {
    		switch ( $status ) {
    			case 1 :
    				$status = 'trash';
    				break;
    
    			case 0  :
    			default :
    				$status = 'publish';
    				break;
    		}
    		return $status;
    	}

    The second, though, would overwrite the first, so that even closed topics that are not trash have the status publish again. Is there any chance to combine these two?

    @inspirationally

    Participant

    Hmm…unfortunately, all the posts written by unregistered users are assigned to ME (UserID = 1) automatically. Is there any possibility to give them the user ID 0 (=Anonymous) and in that case also save the username in postmeta?

    // Reply unregistered author name (stored in postmeta)
    		$this->field_map[] = array(
    			'from_tablename'  => 'wbb1_1_post',
    			'from_fieldname'  => 'username',
    			'to_type'         => 'reply',
    			'to_fieldname'    => '_bbp_anonymous_name',
    		);
    		

    @inspirationally

    Participant

    I’m too silly, damn. This is over my intelligence.
    I have the problem, that the posts have the wrong post_parent, so that two topics are merged curiously.

    For example Post ID 9257 from the WBB board is attached to Thread ID 9257 from the old board, and I cannot see what I have to change. Please please help!
    This took me now over 2 hours changing things and I haven’t found it. I guess it is this part, but I have no idea what to change:

    // Parent Post ID.
    		// Note: We join the posts table because this needs to be the fist post of a topic.
    		$this->field_map[] = array(
    			'from_tablename'  => 'wbb1_1_thread',
    			'from_fieldname'  => 'firstPostID',
    			'join_tablename'  => 'wbb1_1_post',
    			'join_type'       => 'INNER',
    			'join_expression' => 'USING (threadID) WHERE wbb1_1_thread.threadID = wbb1_1_post.threadID',
    			'to_type'         => 'reply',
    			'to_fieldname'    => 'post_parent',
    		);
    		

    I updated the wbb.txt above (where I also linked the WBB database structure).

    @netweb

    Keymaster

    Monday morning 🙂 I’ll download your updated file and take a look now.

    @netweb

    Keymaster

    Forum Section

    • Added Forum Display Order using wbb1_1_board.sortOrder
    • Added wbb1_1_board.boardType and callback method callback_forum_type where category =1 & forum = 0
    • wbb1_1_board.isClosed & callback method callback_forum_status where closed =1 & open = 0
    • (Are the above values correct for these DB fields? I have guessed these ;))

    Topic Section

    • Changed callback method for Topic Language from callback_topic_reply_count to callback_topic_language
    • Added topic author ip
    • Your SQL join for the topic content looks good

    Reply Section

    • Fixed ‘Reply parent forum id’ (we need the ‘parent forum’ here not the first ‘post id’, the inline docs were not very clear here)
    • Removed ‘Reply Unregistered Author name’ (See #2347for more info)

    User Section

    • Added ‘User Signature’ wcf1_user.signature

    Generally cleaned up the code and inline docs to match the other importers.

    The updated version here: https://gist.github.com/ntwb/41b8bb57df78d1a1bd01
    The differences between versions here: https://gist.github.com/ntwb/41b8bb57df78d1a1bd01/revisions

    @inspirationally

    Participant

    Wow, great work!! Thank you so much!
    Baby sleeping, big one sent to kindergarden, so I have a bit of time, too.

    Little changes in your version:

    replaced

    			'from_tablename'  => 'forum',
    

    with

    			'from_tablename'  => 'wbb1_1_board',
    

    at forum status and forum board

    (Btw. everyone may use another board ID!!!) I have 1, some may also have wbb2_1_board and wcf2_ or even wbb1_2_board (if they have a second board – you can install two baords in one WCF – Community Framework – this is to change individually. The same goes for the language IDs. There may be more/different languages.

    Signature import works (will have to see how to implement that later for buddypress maybe), just special characters don’t convert: “there’s Johnny.— – but that’s just a mini error,
    languages work, too (I just have to see how to implement it),
    I have whole topics with all their answers (what I did not have yesterday – instead a headache).
    Categories work quite perfect, too!
    Closed forums -> no idea, because I don’t have any. But it sounds correct.
    BBCodes have been converted, too, besides a few like [align=center]

    The sortOrder does NOT work. Simply, because that field exists in wbb1_1_board, but it is empty.
    This is saved in wbb1_1_board_structure with the fields parentID boardID and position.
    I tried a table join here again and probably destroyed your clean coding once again. But it works, I just tested!!
    See the updated wbb.txt from the top!!

    The author thing is bad,
    It’d be great if they even were anonymous and not ME. Actually there are some pretty silly posts there, and I keep replying to myself this way. Hmmm…
    But I can solve it with manually replacing User ID 1 with 0 where it is a topic or reply for now to MAKE it anonymous.

    I tried – because of the smilies not working and the special chars in the signature – an import with uncommenting the last lines (see wbb.txt)
    Some Special Characters still wrong, but who cares for the few? as long as ä ö ü ß from German language work, everything is fine.
    Smilies started to work. Great!!

    So the only thing left is the author and the passwords.
    I tried to implement parts of a paid plugin WBB Bridge here, but even that didn’t work (plus we cannot simply use it because of the license)

    @inspirationally

    Participant

    I think I can do a live import now. With keeping my WBB Bridge plugin and the old user table, the can still log in.

    I did these changes/additions:

    `
    // User nice name.
    $this->field_map[] = array(
    ‘from_tablename’ => ‘wcf1_user’,
    ‘from_fieldname’ => ‘username’,
    ‘to_type’ => ‘user’,
    ‘to_fieldname’ => ‘user_nicename’,
    ‘callback_method’ => ‘callback_nicename’
    );

    /**
    * This makes the nicename nice to make profile urls work
    */
    public function callback_nicename( $nicename ) {
    //lower case everything
    $nicename = strtolower($nicename);
    //make alphaunermic
    $nicename = preg_replace(“/[^a-z0-9_\s-]/”, “”, $nicename);
    //Clean multiple dashes or whitespaces
    $nicename = preg_replace(“/[\s-]+/”, ” “, $nicename);
    //Convert whitespaces and underscore to dash
    $nicename = preg_replace(“/[\s_]/”, “-“, $nicename);
    return $nicename;
    }
    `

    @pegelf

    Participant

    Hello,

    Is there a final version of the script? I tried the one linked by @netweb on github, but it freezes after 2100 answers.

    And if I try to import the users also, I get the following error:

    WordPress database error: [Unknown column ‘forum.boardType’ in ‘field list’]
    SELECT convert(wbb1_1_board.boardID USING “utf8mb4”) AS boardID,convert(wbb1_1_board.parentID USING “utf8mb4”) AS parentID,convert(wbb1_1_board.threads USING “utf8mb4”) AS threads,convert(wbb1_1_board.posts USING “utf8mb4”) AS posts,convert(wbb1_1_board.title USING “utf8mb4”) AS title,convert(wbb1_1_board.description USING “utf8mb4”) AS description,convert(wbb1_1_board.sortOrder USING “utf8mb4”) AS sortOrder,convert(forum.boardType USING “utf8mb4”) AS boardType,convert(forum.isClosed USING “utf8mb4”) AS isClosed FROM wbb1_1_board AS wbb1_1_board LIMIT 0, 100</blockquote

    @inspirationally

    Participant

    Oh this is long ago.
    I just searched through my harddrive and could find a copy of the wbb3.php, my old import file, but I don’t think that’s the most recent version – I could not find anything with wbb in it. Unfortunately, the wbb3.php in the bbpress plugin folder was overwritten with the bbpress updates.

    @robkk

    Moderator

    @pegelf

    is the exact version of WBB that you are trying to import version 3.1?

    @pegelf

    Participant

    @Robk Yes it is Version 3.1.

    Anyhow I was able to fix the issue by changing the var of the value of “from_tablename” from “forum” to “wbb1_1_board”. I also commented out the lines which doesn’t exist in WBB and also have “forum” as value.

    Here’s my fork:
    https://gist.github.com/pegelf/faa9589f58d2ad7bfb5e

    Now I only have the problem with the freezing. I already tried to use steps of 10 instead from 100, but that didn’t work.

    @robkk

    Moderator

    Thanks you for sharing your fork.

    Have you read this for that freezing issue??

    Import Troubleshooting

    @netweb

    Keymaster

    Thanks for this @@pegelf, bonus points for forking my gist, makes the changes you made easy to see, much appreciated 🙂

    It sounds like you are hitting this crazy freezing issue, it’s not easy to track down, but once you’ve found the offending database row you should be good to go.

    Following the instructions @robkk linked to above should help track it down, when I hit this issue if I’ve got 100,000 replies to import and it fails at 2,100, I’ll delete those first 2,199 reply rows from a working copy/backup of the database as a quick indicator of if/when the issue will crop again, helps to “scope” out whats going on.

Viewing 16 replies - 1 through 16 (of 16 total)
  • You must be logged in to reply to this topic.
Skip to toolbar