SMF Import to bbPress
-
Introduce Forum Importer template for use with importing from SMF v2.0.x(Tested on v2.0.4)
http://codex.bbpress.org/import-forums/ & http://codex.bbpress.org/import-forums/smf
Working: Forums, Topics, Replies, Users & User Passwords
Partial: Custom BBCode for Topic & Reply Content
Not Working Yet: CategoriesAttached to bbPress Trac Ticket #2380 is the 1st pass patch for SMF importing into bbPress.
To test the importer download the latest patch from the link above and copy/ftp the file into the /includes/admin/converters/ folder of your bbPress plugin.
I hope to get the remaining outstanding items done ASAP.
(User passwords will be a little trickier and will take a bit more time)Any feedback and/or testing that you can provide will help me out tremendously.
Cheers,
Stephen 🙂
-
Mike, thanks, much appreciated.
I’ve copied and pasted it into a GitHub Gist here:
https://gist.github.com/ntwb/9ff3f1ed93ae9671a93bIt makes it much clearer to read and update, if anyone wants to make improvements or suggestions drop a comment on the Gist 🙂
This is a great start and I hope to one of these days get attachment importing into bbPress importer 😉
Beautiful – thanks. Hopefully I can contribute more in the future.
Hi again,
Here is an improved version of the SMF to bbPress script. This version supports additional, non-image filetypes, and has more safeguards against inserting attachments multiple times if the script is interrupted and restarted. Since I am not yet savvy in the ways of submitting source code to the forum, I am cutting and pasting, which seems to butcher the syntax horribly…
Many thanks, again, to Stephen.
================================================================
<?php
//Standalone script to import smf attachments for the GD bbPress Attachments plugin
//Mike Russell, April 4, 2015
//Execute after you have imported smf into bbPress.
//Run this from the command line, not the web (it will timeout)require( ‘wp-load.php’ );
require( ‘wp-admin/includes/image.php’ );
global $wpdb;$verbose = false;
$force_attachment_update = false; //normally false. set true to re-write existing attachments
//$limit = ” LIMIT 0,1″;
$limit = ” LIMIT 0,99999″;// source database connection
$host=”localhost”;
$uname=”xxx”;
$pass=”xxx”;
$database = “xxx”;
$site_url = ‘http://www.xxx.com/forum’;
$forum_path = ‘forum/attachments/’;echo “start smf_attachments-to-bbpress v1.2\n”; flush();
//get the attachment rows from SMF
$smf_db = new wpdb($uname, $pass, $database, $host);
$smf_rows = $smf_db->get_results(”
SELECT * FROM smf_attachments
WHERE file_hash != ”
AND filename NOT LIKE ‘%thumb’
ORDER BY id_attach”
.$limit);echo “processing “.$smf_db->num_rows.” rows total\n”;
// process each row
$count = 0;
foreach ($smf_rows as $smf_attachment_row) {
if($verbose) { echo ‘next row, id_attach = ‘.$smf_attachment_row->id_attach.”\n”; flush(); }//look for both a new and old style filename to copy. If neither exists, skip this attachment
$smf_filename = $forum_path.$smf_attachment_row->id_attach.’_’.$smf_attachment_row->file_hash;
if(!file_exists($smf_filename))
$smf_filename = $forum_path.$smf_attachment_row->id_attach.’_’.$smf_attachment_row->filename.$smf_attachment_row->file_hash;if(!file_exists($smf_filename))
{
echo “no file, skipping attachment for SMF attachment “.$smf_attachment_row->id_attach.”, missing SMF file: “.$smf_filename.”\n”; flush();
continue;
}$uploads = wp_upload_dir(‘SMF’);
$new_upload_dir = $uploads[‘path’];
$new_full_filename = $new_upload_dir.’/’.$smf_attachment_row->filename;
if($verbose) { echo(‘old->new = ‘.$smf_filename.’ -> ‘.$new_full_filename.”\n”); flush(); }//copy the enclosed file from SMF to the upload dir if necessary
if(!file_exists($new_full_filename) && !copy($smf_filename, $new_full_filename) ) {
echo “cannot copy: “.$smf_filename.”->”.$new_full_filename.”\n”;
} else {
//look for a reference to the bbPress-imported parent post for the current attachment
$parent_args = array(
‘post_type’ => array(‘topic’, ‘reply’),
‘meta_key’ => ‘_bbp_post_id’,
‘meta_value’ => $smf_attachment_row->id_msg
);
//echo “$parent_args = “.print_r($parent_args).”\n”;$parent_query = new WP_Query($parent_args);
$parent_query->get_posts();
if($verbose || $parent_query->post_count > 1) {
echo $parent_query->post_count.” posts found for smf_post id “.$smf_attachment_row->id_msg.”\n”; flush();
}//add the attachment if there is exactly one parent referencing it
if($parent_query->have_posts() && $parent_query->post_count == 1) {
$guid = $uploads[‘url’] . ‘/’ . basename( $new_full_filename );
$parent_query->the_post();
$post_id = get_the_ID();//if force option is not specified, skip updating an already existing attachment
if(!$force_attachment_update) {
$q = $wpdb->prepare(
”
SELECT COUNT(*)
FROM $wpdb->posts
WHERE guid = %s AND post_parent = %d
“,
$guid,
$post_id);$nrefs = $wpdb->get_var($q);
if($nrefs > 0) {
if($verbose) { echo “skip existing attachment: “.$guid.”\n”; flush(); }
wp_reset_postdata(); //reset post info
goto done;
}
}$attachment_data = array(
‘guid’ => $guid,
‘post_mime_type’ => ‘image/’.$smf_attachment_row->fileext,
‘post_title’ => $smf_attachment_row->filename,
‘post_status’ => null,
‘post_content’ => ”,
);
if($verbose) { echo “attachment_data = “.print_r($attachment_data).”\n”; flush(); }$attach_id = wp_insert_attachment($attachment_data, $new_full_filename, $post_id);
//echo “attach_id = “.$attach_id.”\n”; flush();
if($attach_id) {
//update_post_meta($attach_id, ‘_bbp_attachment’, 1);
if($attach_metadata = wp_generate_attachment_metadata($attach_id, $new_full_filename)) {
//echo ‘attach_metadata = ‘.print_r($attach_metadata).”\n”; flush();
wp_update_attachment_metadata( $attach_id, $attach_metadata );
set_post_thumbnail( $post_id, $attach_id );
} else {
echo ‘wp_generate_attachment_metadata failed, fname = ‘.$new_full_filename.”\n”; flush();
}
}
wp_reset_postdata();
}
}
done:
$count++;
if($count%100 == 0)
{
echo $count.” attachments processed\r”;
flush();
}
}
echo “Done, processed “.$count.” records\n\n”;//clean up message body text
//convert <tt> -> <br />
mysql_query(“UPDATE wp_posts SET post_content = REPLACE (post_content, \'<tt>\’, \'<br />\’) WHERE post_content LIKE \’%<tt>%\'”);mysql_close($connection);
exit;
?>Another script – this one cleans up the messy duplicate attachment references to the same media entry that were left behind, should you happen to re-run the older version of the script. Hopefully
you will not need this, but I certainly did after developing the script and running it dozens of times! – Mike
===================================================================================
//php_removedupes.php – clean up after running older version of script multiple times.
//Mike Russell
//
<?php
require( ‘wp-load.php’ );
require( ‘wp-admin/includes/image.php’ );
global $wpdb;echo “version 1.0\n”;
$chunk = 100; //posts at a timea
//utility functions
function dumpguids($children)
{
foreach($children as $child)
echo “$child->ID $child->guid\n”; flush();
}//remove the %20’s from the supplied guid
function fixguid($guid)
{
return str_replace(“%20”, “”, $guid);
}for($kerchunk = 0; true; $kerchunk++) {
//Get the next group of posts.
$args = array( ‘offset’=> $kerchunk * $chunk,
‘numberposts’ => $chunk,
‘post_type’ => array(‘topic’, ‘reply’)
);
$posts = get_posts($args);
if(empty($posts))
break;foreach($posts as $post) {
//echo print_r($post); flush();//get all attachments for this post.
$children = get_children( array (
‘post_parent’ =>$post->ID,
‘post_type’ => ‘attachment’,
‘numberposts’ => -1
) );
if(count($children) < 2)
continue;
//echo count($children).” attachments for $post->post_title\n”; flush();//sort the guid’s, ignoring %20 added by the SMF attachment script
usort($children, function($a, $b) {
return strcasecmp(str_replace(“%20”, “”, $a->guid), str_replace(“%20”, “”, $b->guid));
});
//reset the parent field of all the extra attachments with matching guid’s, keeping the last one.
//this leaves orphaned meta rows, which can be cleaned up later by standard WordPress
$deleted = 0;
$unique = 0;
$n = count($children);
for($i=0; $i<($n-1); $i++) {
if(fixguid($children[$i]->guid) == fixguid($children[$i+1]->guid)) {
//echo “delete dupe “.$children[$i]->post_title.” from parent “.$children[$i]->post_parent.”\n”; flush();
$wpdb->delete($wpdb->posts, array(‘ID’ => $children[$i]->ID));
$deleted++;
} else {
$unique++;
}
}
}
echo $kerchunk * $chunk.” posts scanned\r”; flush();
}
echo “done\n”;
?>Is there a way to use this importer for the latest version SMF 2.1 Beta 1? Please help me, because I want to use bbPress. I have over 20,000 registered members and this is a big problem. SMF has become a very bad solution.
@kweh, I’ll update the SMF importer once the full v2.1 is released, we won’t be supporting the 2.1 beta, if you’ve upgraded from 2.0.x to 2.1 beta I’d look at seeing if you can downgrade back to 2.0.x and import from there.
Hi Stephen,
I’ve made some slight improvements to the attachment importer script – pls check with me when the time comes, and I’ll send you the latest version. I have looked around for how to get write access to the source repository, and still don’t have a clue.
Looking over the SMF->bbPress import template, I noticed that avatars are being imported from the wrong place – they actually live in the attachments table, not the avatar table, with the associated ID of the user.
All the best,
Mike
When I attempt to access the URL https://codex.bbpress.org/import-forums/smf the page is blank. Am I clicking wrong here, or is something wrong with the page?
Nevermind I figured out where I was going wrong. It helps to read the documentation!
Hi Stephen! Thanks for all of your volunteer work on this, I greatly appreciate it. 🙂
Just a few questions, and I did do a few days of research (with no luck) before asking.
1.) Any plans to support importing of private messages? I plan to use BuddpyPress + bbPress, so the feature will be covered. What are the chances of adding another checkbox to “optional settings” that says “Attempt to import private messages to BuddyPress”
2.) I have noticed that my existing SMF database had a handful of deleted accounts, so my settings left those posts behind as “Guest” posts (and it displayed the deleted user’s old username). In WordPress + bbPress, I created the original admin/keymaster user in my fresh install. After doing the SMF to bbPress import, the “Guest” posts from the deleted account are automatically assigned to the existing admin/keymaster user that I created in WordPress. Is there a cleaner way to handle this default behavior? Do these posts from deleted accounts require being assigned to an account? If not, could I somehow mass re-attribute these posts to a different account?
Hi timsilva_
We have the same issue.
SMF = we have created a user : “Guest” All the posts were attribute to “Guest”.
Import = a new member “Guest”.The other solution:
If “keymaster” has no post
Create an account “guest”
Create a new “keymaster”
Delete the previous “keymaster” attribute post to guest.Any plans to support importing of private messages?
Yes, I want to, I just haven’t worked out the best way to do this 🙁
2.) I have noticed that my existing SMF database had a handful of deleted accounts…
All of your questions in regard to this point ship in bbPress 2.6, with a few more not yet mentioned. 😉
I got the problem, that the importer can´t handle “ä”, “ö” and “ü” from my german board.
How can I change this? Is there a fix available?They are displayed like this “Ankündigungen” instead of “Ankündigungen”.
No idea? 🙁
sorry for the late reply, this is a good question. I might need Stephens help on this one.
I had fundamentally wrapped up a converter format for smf 1.1.8 when I went over your post and your script. Uplifting news is it would appear that I got really close.
I might want to see better what the authenticate_pass capacity should do. On the off chance that it is simply expected to return genuine if the present client’s watchword coordinates the old smf hashed secret word then it appears that I ought to have the capacity to utilize a capacity like ssi_checkPassword that was incorporated in the SMF ssi.php script, accepting that the sha1 capacity is accessible.
@donchulio Adding a topic titled
Ankündigungen
with the topic contentAnkündigungen
and a reply with contentAnkündigungen
works as expected for me with bbPress 2.5.8Can you check what character set and collation both your source SMF database is and also the same for your WordPress database?
Hi Stephen,
sorry if I am a newbie in this but where can I check this exactly?
If you open up phpMyAdmin which I presume you’ve most likely got installed on your webhost and then open the SMF database and then open the
smf_messages
table then click ‘sql’, copy and paste this into the query box and click “Go” (The Go button is at the bottom right)show variables like 'character_set_database'; show variables like 'collation_database';
You should see something like this https://cloudup.com/cluBnqdtz4x
In my case the character set is
utf8
and collation isutf8_general_ci
I don’t know why, but the value is “latin1”. How can I change this?
Which one is
latin1
? Having both these values would be helpful so I can test it please 🙂It is in: “character_set_database”.
What about collation?
I think a Screenshot is easier: http://abload.de/img/imageyyq0e.png
@donchulio Thanks, strange the collation isn’t there, but anyway, I’ll create an SMF test site using
latin1
and see what results I get 🙂
- The topic ‘SMF Import to bbPress’ is closed to new replies.