Vanilla 2 Converter
-
I’m working on adding a Vanilla Forums 2 converter to the bbPress Import package. Are there any developers out there that can answer a couple questions for me about how the converter engine works? I can reverse engineer it, but takes a lot more time than chatting with the guy/gal that wrote it. Thanks!
-
What do you want to know?
There is no need to reverse engineer anything, just grab the latest copy of Example.php either from /bbpress/includes/admin/converters/ or the latest updated version I am working on here and start modifying the table mappings as per the docs in the codex.
https://codex.bbpress.org/import-forums/ & https://codex.bbpress.org/import-forums/custom-import/
I’m well beyond the docs you referenced. I have the Vanilla 2 importer working, but there is a couple things I’m trying to correct. 1) I would like to filter the users to confirmed only, but the filter expression doesn’t apply. I’ve tried including the where conditions as a from_expression as well as putting them both in the join_expression. It just doesn’t look like the converter is referencing my filter.
` $this->field_map[] = array(
‘from_tablename’ => ‘User’,
‘from_fieldname’ => ‘UserID’,
‘from_expression’ => ‘WHERE User.Deleted = 0’,
‘join_tablename’ => ‘UserRole’,
‘join_type’ => ‘INNER’,
‘join_expression’ => ‘USING (UserID) WHERE UserRole.RoleID != 3’,
‘to_type’ => ‘user’,
‘to_fieldname’ => ‘_bbp_user_id’
);`2) I would like to import all users as “Participants”, but all users come into the system with no bbPress role selected.
1) In the yellow status window of the importer if you right click in Firefox/Chrome ‘inspect element’ you can check the SQL queries that the importer ran, have a look at the SQL query for the user import and check if that is what the SQL query should be.
The from/join logic is in converter.phpL#777 and you can use that as a guide to how to translate your query to the fields you are mapping.
2) I would not worry about this option as once you have finished the import and start to run the ‘repair forums’ tools the last option is “Remap existing users to default forum roles” and that will set all users without a bbPress role to ‘participant’
Feel free to throw any questions you have here as I am trying to fix a join for the Invision.php importer as we speak and its being a pain also 😉
From Firebug inspection, the converter is ignoring the join altogether for the user import.
Try this…
`
$this->field_map[] = array(
‘from_tablename’ => ‘UserRole’,
‘from_fieldname’ => ‘UserID’,
‘from_expression’ => ‘WHERE User.Deleted = 0′,
‘join_tablename’ => ‘User’,
‘join_type’ => ‘INNER’,
‘join_expression’ => ‘USING (UserID) WHERE UserRole.RoleID != 3′,
‘to_type’ => ‘user’,
‘to_fieldname’ => ‘_bbp_user_id’
);
`Unless the ‘from_tablename’ is different from the other ‘from_tablename’ fields in each of the query sections it will not initiate the SQL ‘join’…
Its just one of those things going on in the whole ‘lets build some awseome SQL queries’
I tried reversing the join alreay, but I’ll try again. This doesn’t work either:
` $this->field_map[] = array(
‘from_tablename’ => ‘User’,
‘from_fieldname’ => ‘UserID’,
‘join_tablename’ => ‘UserRole’,
‘join_type’ => ‘INNER’,
‘join_expression’ => ‘USING (UserID) WHERE UserRole.RoleID != 3 AND User.Deleted = 0’,
‘to_type’ => ‘user’,
‘to_fieldname’ => ‘_bbp_user_id’
);`The reversed join you provided above didn’t do it either. Firebug says:
`
SELECT convert(User.Password USING “utf8”) AS Password,convert(User.Name USING “utf8”) AS Name,convert(User.Email USING “utf8”) AS Email,convert(User.DateInserted USING “utf8”) AS DateInserted FROM GDN_User AS User LIMIT 0, 100
`Do you have any other ‘joins’ you are trying to use with the ‘user’ section?
All your mappings including all ‘join’ and ‘where’ statements etc in each field mapping in each of the sections forums, topics, topic-tags, replies & users must all be able to be piled to together to form a valid SQL query for that section.
I’m not using any joins on the other user blocks. Doing that causes other problems. After some work, I managed to get your example working on UserID, but then it can’t find the password and other fields from the UserRole table; even though I left the other user sections referencing the User table. I’m assuming, like you said, all the user stuff is getting combined into one huge query, I’m just not sure how to work withing that process. I’ll keep digging tomorrow. Thanks for the help.
I just added a UserRole table to the DB I am working with and added a field RoleID and set some users to RoleID = 3 and the resulting query should not import users with RoleID = 3 though it is.
I’ll take another look at what might be going on here as it doesn’t seem to be picking up the ‘join’ when processing the user section.
Can you upload a copy of your Vanilla converter to http://gist.github.com and i’ll take a look at what you have in there and will add some of the other table mappings that I have been adding to all the other converters over the past week (I haven’t added these to the Example.php converter yet, they are just extra mappings to help bbPress during the recounts after importing etc)
I created a public shared GitHub doc of my current Vanilla2.php. You can find it here: https://gist.github.com/4591432
Thanks for the help. I’m going to continue messing with it.
Of course, this is probably the easiest way to clean out deleted and un-confirmed users, but it would be cool to build it into the converter for others to use. Vanilla has no way of cleaning these by default.
`
DELETE GDN_User
FROM GDN_User
LEFT OUTER JOIN GDN_UserRole ON GDN_User.UserID = GDN_UserRole.UserID
WHERE GDN_UserRole.RoleID = 3 OR GDN_User.Deleted = 1
`Did you use a custom table prefix when importing Vanilla?
When you setup Vanilla does it let you set a table prefix or does it always use its own choice?
eg from_tablename’ => ‘User’,
If ALL Vanilla forums use this structure and you DO NOT enter a table prefix for conversion I have a fix that will the filter users during import.
Yeah, all the Vanilla tables are prefixed with “GDN_”. I was mostly trying to work it out in the converter file so I could share it with others that might be trying to convert. Vanilla is pretty popular, so there’s gotta be a few out there tired of Vanilla’s poor integration with WordPress. Unless there is an easy way to filter them, I’ll just run the SQL statement above, that clears them out nicely. Thanks alot for your help.
We will have to go down that route for now as the phpBB import has a similar issue and at the moment the docs state to manually delete these after import.
We can look into why the converter isn’t doing joins correctly on the user table for future versions as if we changed to much there now I am quite sure we would break other stuff and would have very little time to test this at this stage.
One more thing Stephen, is the converter supposed to convert Topic Subscription? If not, do you have any idea where that’s stored in WordPress/bbPress?
Currently there is no support for topic subscriptions or favorites…
It is all stored in wp_usermeta
eg. User ID = 1, meta_key = __bbp_subscriptions, topic_id = 181889
user_id | meta_key | meta_value
1 | wp__bbp_subscriptions | 181889
1 | wp__bbp_favorites | 181889
(Note the double underscores __bbp)Your old database will have the old topicid you’ll want to look in wp_postmeta to get some values.
eg. New TopicID = 181889, meta_key = _bbp_old_topic_id, Old TopicID
post_id | meta_key | meta_value
181889 | _bbp_old_topic_id | 4631Could one of you guys post a gist for just the regular importing of Vanilla2 into bbPress? I’m awful with SQL. I’m trying to get the previously posted gist to work by modifying little bits here and there but it still keeps telling me there’s nothing to convert…
Just finishing this off today and I will post back with a link to the docs and download
Thank you so much! This saves me days of effort!
Stephen,
I think I made a couple tweaks to the Vanilla2 during final testing. Have you tested the version you have, or should I post my latest version?I have made quite a few tweaks 😉
You can see the revisions I made up until last night here
https://gist.github.com/4633175/revisions
(It is a fork or your gist with my changes)I have just added ‘topic tags’ and just cleaning up the code ready to submit for bbPress core 😉
What else have you changed? If you update your original gist, I can look at any differences and merge those also.
Jack,
You can try this one, it’s works for me. A couple notes, I didn’t finish the “Tags” section because I don’t have any tags in my Vanilla Forum. Also, I didn’t have any child Categories, but the converter “should” work for parent/child Categories. I also removed the code that excludes Deleted and Comment Email statuses, so those will get imported.https://gist.github.com/4591432
I have a full test server setup however, and would happy to help tweak this converter so it works for you.
That’s great your including it in the core. I put the filter back in the Users section that excludes the Deleted users and updated the Gist again. That filter worked, the UserRole join was what we were jacking with up above; never got that to work.
One thing left to do:
1) Passwords with a type of Vanilla are the same hash that WP uses, so they convert over nicely. Vanilla allows you to integrate a user with Facebook, and some other systems. I didn’t have any of those users, but some kind of call-back would need to be built for it. Also, this converter includes some password functions at the bottom, but I don’t think it’s used.
Outside of that, I got a perfect conversion from this one. I would also be happy to post the SQL code I used to clean up the User table, and also SQL code I used to move posts from Vanilla Users that already existed in WP (Admins and authors).
Stephen,
Looks like you pulled out the custom callback I created to cleanup the parent ID on Categories. In Vanilla, the parent is -1 instead of 0. Unless it’s in there and I missed it, my callback fixed that so it came back as a 0. (callback_forum_parent)
- You must be logged in to reply to this topic.
