Skip to:
Content
Pages
Categories
Search
Top
Bottom

SMF Import to bbPress

  • Introduce Forum Importer template for use with importing from SMF v2.0.x(Tested on v2.0.4)

    http://codex.bbpress.org/import-forums/ & ​​http://codex.bbpress.org/import-forums/smf

    Working: Forums, Topics, Replies, Users & User Passwords
    Partial: Custom BBCode for Topic & Reply Content
    Not Working Yet: Categories

    Attached to bbPress Trac Ticket #2380 is the 1st pass patch for SMF importing into bbPress.

    To test the importer download the latest patch from the link above and copy/ftp the file into the /includes/admin/converters/ folder of your bbPress plugin.

    I hope to get the remaining outstanding items done ASAP.
    (User passwords will be a little trickier and will take a bit more time)

    Any feedback and/or testing that you can provide will help me out tremendously.

    Cheers,

    Stephen 🙂

Viewing 25 replies - 126 through 150 (of 155 total)

  • Stephen Edgar
    Keymaster

    @netweb

    Mike, thanks, much appreciated.

    I’ve copied and pasted it into a GitHub Gist here:
    https://gist.github.com/ntwb/9ff3f1ed93ae9671a93b

    It makes it much clearer to read and update, if anyone wants to make improvements or suggestions drop a comment on the Gist 🙂

    This is a great start and I hope to one of these days get attachment importing into bbPress importer 😉


    curvemeister
    Participant

    @curvemeister

    Beautiful – thanks. Hopefully I can contribute more in the future.


    curvemeister
    Participant

    @curvemeister

    Hi again,

    Here is an improved version of the SMF to bbPress script. This version supports additional, non-image filetypes, and has more safeguards against inserting attachments multiple times if the script is interrupted and restarted. Since I am not yet savvy in the ways of submitting source code to the forum, I am cutting and pasting, which seems to butcher the syntax horribly…

    Many thanks, again, to Stephen.

    ================================================================
    <?php
    //Standalone script to import smf attachments for the GD bbPress Attachments plugin
    //Mike Russell, April 4, 2015
    //Execute after you have imported smf into bbPress.
    //Run this from the command line, not the web (it will timeout)

    require( ‘wp-load.php’ );
    require( ‘wp-admin/includes/image.php’ );
    global $wpdb;

    $verbose = false;
    $force_attachment_update = false; //normally false. set true to re-write existing attachments
    //$limit = ” LIMIT 0,1″;
    $limit = ” LIMIT 0,99999″;

    // source database connection
    $host=”localhost”;
    $uname=”xxx”;
    $pass=”xxx”;
    $database = “xxx”;
    $site_url = ‘http://www.xxx.com/forum&#8217;;
    $forum_path = ‘forum/attachments/’;

    echo “start smf_attachments-to-bbpress v1.2\n”; flush();

    //get the attachment rows from SMF
    $smf_db = new wpdb($uname, $pass, $database, $host);
    $smf_rows = $smf_db->get_results(”
    SELECT * FROM smf_attachments
    WHERE file_hash != ”
    AND filename NOT LIKE ‘%thumb’
    ORDER BY id_attach”
    .$limit);

    echo “processing “.$smf_db->num_rows.” rows total\n”;

    // process each row
    $count = 0;
    foreach ($smf_rows as $smf_attachment_row) {
    if($verbose) { echo ‘next row, id_attach = ‘.$smf_attachment_row->id_attach.”\n”; flush(); }

    //look for both a new and old style filename to copy. If neither exists, skip this attachment
    $smf_filename = $forum_path.$smf_attachment_row->id_attach.’_’.$smf_attachment_row->file_hash;
    if(!file_exists($smf_filename))
    $smf_filename = $forum_path.$smf_attachment_row->id_attach.’_’.$smf_attachment_row->filename.$smf_attachment_row->file_hash;

    if(!file_exists($smf_filename))
    {
    echo “no file, skipping attachment for SMF attachment “.$smf_attachment_row->id_attach.”, missing SMF file: “.$smf_filename.”\n”; flush();
    continue;
    }

    $uploads = wp_upload_dir(‘SMF’);
    $new_upload_dir = $uploads[‘path’];
    $new_full_filename = $new_upload_dir.’/’.$smf_attachment_row->filename;
    if($verbose) { echo(‘old->new = ‘.$smf_filename.’ -> ‘.$new_full_filename.”\n”); flush(); }

    //copy the enclosed file from SMF to the upload dir if necessary
    if(!file_exists($new_full_filename) && !copy($smf_filename, $new_full_filename) ) {
    echo “cannot copy: “.$smf_filename.”->”.$new_full_filename.”\n”;
    } else {
    //look for a reference to the bbPress-imported parent post for the current attachment
    $parent_args = array(
    ‘post_type’ => array(‘topic’, ‘reply’),
    ‘meta_key’ => ‘_bbp_post_id’,
    ‘meta_value’ => $smf_attachment_row->id_msg
    );
    //echo “$parent_args = “.print_r($parent_args).”\n”;

    $parent_query = new WP_Query($parent_args);
    $parent_query->get_posts();
    if($verbose || $parent_query->post_count > 1) {
    echo $parent_query->post_count.” posts found for smf_post id “.$smf_attachment_row->id_msg.”\n”; flush();
    }

    //add the attachment if there is exactly one parent referencing it
    if($parent_query->have_posts() && $parent_query->post_count == 1) {
    $guid = $uploads[‘url’] . ‘/’ . basename( $new_full_filename );
    $parent_query->the_post();
    $post_id = get_the_ID();

    //if force option is not specified, skip updating an already existing attachment
    if(!$force_attachment_update) {
    $q = $wpdb->prepare(

    SELECT COUNT(*)
    FROM $wpdb->posts
    WHERE guid = %s AND post_parent = %d
    “,
    $guid,
    $post_id);

    $nrefs = $wpdb->get_var($q);
    if($nrefs > 0) {
    if($verbose) { echo “skip existing attachment: “.$guid.”\n”; flush(); }
    wp_reset_postdata(); //reset post info
    goto done;
    }
    }

    $attachment_data = array(
    ‘guid’ => $guid,
    ‘post_mime_type’ => ‘image/’.$smf_attachment_row->fileext,
    ‘post_title’ => $smf_attachment_row->filename,
    ‘post_status’ => null,
    ‘post_content’ => ”,
    );
    if($verbose) { echo “attachment_data = “.print_r($attachment_data).”\n”; flush(); }

    $attach_id = wp_insert_attachment($attachment_data, $new_full_filename, $post_id);

    //echo “attach_id = “.$attach_id.”\n”; flush();
    if($attach_id) {
    //update_post_meta($attach_id, ‘_bbp_attachment’, 1);
    if($attach_metadata = wp_generate_attachment_metadata($attach_id, $new_full_filename)) {
    //echo ‘attach_metadata = ‘.print_r($attach_metadata).”\n”; flush();
    wp_update_attachment_metadata( $attach_id, $attach_metadata );
    set_post_thumbnail( $post_id, $attach_id );
    } else {
    echo ‘wp_generate_attachment_metadata failed, fname = ‘.$new_full_filename.”\n”; flush();
    }
    }
    wp_reset_postdata();
    }
    }
    done:
    $count++;
    if($count%100 == 0)
    {
    echo $count.” attachments processed\r”;
    flush();
    }
    }
    echo “Done, processed “.$count.” records\n\n”;

    //clean up message body text
    //convert <tt> -> <br />
    mysql_query(“UPDATE wp_posts SET post_content = REPLACE (post_content, \'<tt>\’, \'<br />\’) WHERE post_content LIKE \’%<tt>%\'”);

    mysql_close($connection);
    exit;
    ?>


    curvemeister
    Participant

    @curvemeister

    Another script – this one cleans up the messy duplicate attachment references to the same media entry that were left behind, should you happen to re-run the older version of the script. Hopefully
    you will not need this, but I certainly did after developing the script and running it dozens of times! – Mike
    ===================================================================================
    //php_removedupes.php – clean up after running older version of script multiple times.
    //Mike Russell
    //
    <?php
    require( ‘wp-load.php’ );
    require( ‘wp-admin/includes/image.php’ );
    global $wpdb;

    echo “version 1.0\n”;

    $chunk = 100; //posts at a timea

    //utility functions

    function dumpguids($children)
    {
    foreach($children as $child)
    echo “$child->ID $child->guid\n”; flush();
    }

    //remove the %20’s from the supplied guid
    function fixguid($guid)
    {
    return str_replace(“%20”, “”, $guid);
    }

    for($kerchunk = 0; true; $kerchunk++) {

    //Get the next group of posts.
    $args = array( ‘offset’=> $kerchunk * $chunk,
    ‘numberposts’ => $chunk,
    ‘post_type’ => array(‘topic’, ‘reply’)
    );
    $posts = get_posts($args);
    if(empty($posts))
    break;

    foreach($posts as $post) {
    //echo print_r($post); flush();

    //get all attachments for this post.
    $children = get_children( array (
    ‘post_parent’ =>$post->ID,
    ‘post_type’ => ‘attachment’,
    ‘numberposts’ => -1
    ) );
    if(count($children) < 2)
    continue;
    //echo count($children).” attachments for $post->post_title\n”; flush();

    //sort the guid’s, ignoring %20 added by the SMF attachment script
    usort($children, function($a, $b) {
    return strcasecmp(str_replace(“%20”, “”, $a->guid), str_replace(“%20”, “”, $b->guid));
    });
    //reset the parent field of all the extra attachments with matching guid’s, keeping the last one.
    //this leaves orphaned meta rows, which can be cleaned up later by standard WordPress
    $deleted = 0;
    $unique = 0;
    $n = count($children);
    for($i=0; $i<($n-1); $i++) {
    if(fixguid($children[$i]->guid) == fixguid($children[$i+1]->guid)) {
    //echo “delete dupe “.$children[$i]->post_title.” from parent “.$children[$i]->post_parent.”\n”; flush();
    $wpdb->delete($wpdb->posts, array(‘ID’ => $children[$i]->ID));
    $deleted++;
    } else {
    $unique++;
    }
    }
    }
    echo $kerchunk * $chunk.” posts scanned\r”; flush();
    }
    echo “done\n”;
    ?>


    kweh
    Participant

    @kweh

    Is there a way to use this importer for the latest version SMF 2.1 Beta 1? Please help me, because I want to use bbPress. I have over 20,000 registered members and this is a big problem. SMF has become a very bad solution.

    @kweh, I’ll update the SMF importer once the full v2.1 is released, we won’t be supporting the 2.1 beta, if you’ve upgraded from 2.0.x to 2.1 beta I’d look at seeing if you can downgrade back to 2.0.x and import from there.


    curvemeister
    Participant

    @curvemeister

    Hi Stephen,

    I’ve made some slight improvements to the attachment importer script – pls check with me when the time comes, and I’ll send you the latest version. I have looked around for how to get write access to the source repository, and still don’t have a clue.

    Looking over the SMF->bbPress import template, I noticed that avatars are being imported from the wrong place – they actually live in the attachments table, not the avatar table, with the associated ID of the user.

    All the best,

    Mike


    l5z5heli
    Participant

    @l5z5heli

    When I attempt to access the URL ​​https://codex.bbpress.org/import-forums/smf the page is blank. Am I clicking wrong here, or is something wrong with the page?


    l5z5heli
    Participant

    @l5z5heli

    Nevermind I figured out where I was going wrong. It helps to read the documentation!


    timsilva_
    Participant

    @timsilva_

    Hi Stephen! Thanks for all of your volunteer work on this, I greatly appreciate it. 🙂

    Just a few questions, and I did do a few days of research (with no luck) before asking.

    1.) Any plans to support importing of private messages? I plan to use BuddpyPress + bbPress, so the feature will be covered. What are the chances of adding another checkbox to “optional settings” that says “Attempt to import private messages to BuddyPress”

    2.) I have noticed that my existing SMF database had a handful of deleted accounts, so my settings left those posts behind as “Guest” posts (and it displayed the deleted user’s old username). In WordPress + bbPress, I created the original admin/keymaster user in my fresh install. After doing the SMF to bbPress import, the “Guest” posts from the deleted account are automatically assigned to the existing admin/keymaster user that I created in WordPress. Is there a cleaner way to handle this default behavior? Do these posts from deleted accounts require being assigned to an account? If not, could I somehow mass re-attribute these posts to a different account?


    Mei Ling
    Participant

    @mei-ling

    Hi timsilva_

    We have the same issue.
    SMF = we have created a user : “Guest” All the posts were attribute to “Guest”.
    Import = a new member “Guest”.

    The other solution:

    If “keymaster” has no post
    Create an account “guest”
    Create a new “keymaster”
    Delete the previous “keymaster” attribute post to guest.

    Any plans to support importing of private messages?

    Yes, I want to, I just haven’t worked out the best way to do this 🙁

    2.) I have noticed that my existing SMF database had a handful of deleted accounts…

    All of your questions in regard to this point ship in bbPress 2.6, with a few more not yet mentioned. 😉


    donchulio
    Participant

    @donchulio

    I got the problem, that the importer can´t handle “ä”, “ö” and “ü” from my german board.
    How can I change this? Is there a fix available?

    They are displayed like this “Ankündigungen” instead of “Ankündigungen”.


    donchulio
    Participant

    @donchulio

    No idea? 🙁


    Robkk
    Moderator

    @robkk

    @donchulio

    sorry for the late reply, this is a good question. I might need Stephens help on this one.


    bulkseocontents
    Participant

    @bulkseocontents

    I had fundamentally wrapped up a converter format for smf 1.1.8 when I went over your post and your script. Uplifting news is it would appear that I got really close.

    I might want to see better what the authenticate_pass capacity should do. On the off chance that it is simply expected to return genuine if the present client’s watchword coordinates the old smf hashed secret word then it appears that I ought to have the capacity to utilize a capacity like ssi_checkPassword that was incorporated in the SMF ssi.php script, accepting that the sha1 capacity is accessible.

    @donchulio Adding a topic titled Ankündigungen with the topic content Ankündigungen and a reply with content Ankündigungen works as expected for me with bbPress 2.5.8

    Screen Shot

    Can you check what character set and collation both your source SMF database is and also the same for your WordPress database?


    donchulio
    Participant

    @donchulio

    Hi Stephen,

    sorry if I am a newbie in this but where can I check this exactly?

    If you open up phpMyAdmin which I presume you’ve most likely got installed on your webhost and then open the SMF database and then open the smf_messages table then click ‘sql’, copy and paste this into the query box and click “Go” (The Go button is at the bottom right)

    
    show variables like 'character_set_database';
    show variables like 'collation_database';
    

    You should see something like this https://cloudup.com/cluBnqdtz4x

    In my case the character set is utf8 and collation is utf8_general_ci


    donchulio
    Participant

    @donchulio

    I don’t know why, but the value is “latin1”. How can I change this?

    Which one is latin1? Having both these values would be helpful so I can test it please 🙂


    donchulio
    Participant

    @donchulio

    It is in: “character_set_database”.

    What about collation?


    donchulio
    Participant

    @donchulio

    I think a Screenshot is easier: http://abload.de/img/imageyyq0e.png

    @donchulio Thanks, strange the collation isn’t there, but anyway, I’ll create an SMF test site using latin1 and see what results I get 🙂

Viewing 25 replies - 126 through 150 (of 155 total)
  • The topic ‘SMF Import to bbPress’ is closed to new replies.
Skip to toolbar