Skip to:
Content
Pages
Categories
Search
Top
Bottom

Conversion MyBB to bbPress


  • EzYRiDaH
    Participant

    @ezyridah

    Hi there,
    First of all I am not a hardcore coder. Used to code 20 years ago, so I can read a lot of the syntax, but am quite new to the javascript, php etc. Can handle myself a bit in sql though.

    I am trying to migrate a very big MyBB forum to bbPres (1,7 mlj posts and 63k discussions). Tried to do it directly on my VPS, but it took way too long. Ages…

    After some trial and error I was able to make a complete dump of the MyBB as SQL and imported it on my Mac’s MAMP. I was one happy muppet and kicked of the conversion.

    Whatever I do I keep getting the same messages. No error messages, just that there is nothing to convert. I have tried every scenario I could think of that could go wrong. Localhost port, userid/pass on database, prefix etc etc. Whatever I do, even leave blank fields to get some kind of error message, I still get the same result. Absolutely nothing to convert.

    I’ve checked the code but could not find any error handling. Could be there, but I am not too familiar on linking php code files.

    I get the feeling this is some kind of MAMP issue because I got it to run on my VPN.

    Any help / suggestions are more than welcome.

    Thanks in advance!

Viewing 10 replies - 1 through 10 (of 10 total)

  • Stephen Edgar
    Moderator

    @netweb

    As I don’t own a Mac, yet, I think this is the issue in that your MAMP setup is using port 8888:

    http://bbpress.org/forums/topic/bbpress-2-3-now-available/page/3/#post-133305

    The only difference between XAMPP and MAMP (free) is that XAMPP uses “localhost” host name, while MAMP (free) uses “localhost:8888″ when using MAMP ports by default (MAMP pro uses “localhost” that tell why it works OK with you). So I went to MAMP preferences and changed the ports settings to use default Apache and MySQL ports to use “localhost” instead of “localhost:8888″, and finally posting topics and replies is available again :)

    Even better I setup a virtual hostname manually in MAMP (free) and the everything works fine.

    I hope this will help others with the same issue with MAMP (free). Thanks for your time


    EzYRiDaH
    Participant

    @ezyridah

    Hi Stephen,
    Thank you for your tip! That was the problem. Turned on the conversion. Very curious on what the results will be :) Thanks again!

    Cheers


    Stephen Edgar
    Moderator

    @netweb

    Excellent, let me know how it goes, haven’t seen a MyBB that big be imported, yet :)


    EzYRiDaH
    Participant

    @ezyridah

    A lot of trial and error. Have restarted it a couple of times. Running it on my Macbook Air with SSD and i7 and MAMP but it is taking days.

    #3 test conversion is progressing slowly. Already running for almost 2 days. Now done more than 50%. Doing 10.000 records at a time, with 0.1 delay time. No issues so far. Not sure if I can boost it more. The restarts seem to work ok too.

    Will keep you posted ;) Will be really glad when we get rid of MyBB! Fingers crossed ;)


    Stephen Edgar
    Moderator

    @netweb

    Cool, it is quite a few and the only real way to boost the performance is more resources, back with my IT hat on, separate hard drive spindles, preferably SSD (or RAM drives) for OS, source DB, destination DB and lots of RAM…


    EzYRiDaH
    Participant

    @ezyridah

    Hi Stephen / @netweb

    I was finally able to run a complete conversion without any restarts. The MyBB SQL of the database was 435,9 Mb big. We cleaned a lot of the old forum. It contained 1 million+ messages, close to 40k topics and a 400+ members.

    The conversion ran for roughly 48 hours on a MacBook Air with i7, 8Gb memory and a 500Gb Solid State drive under MAMP.

    For people who want to do such a big conversion, I also had to tweak a lot of the PHP ini files to import such a big query and get maximum throughput. Below a summary. It took me a while to find all the necessary settings.

    Copy /applications/mamp/library/support-files/my-huge.cnf and rename to /Applications/MAMP/conf/my.cnf
    Change in my.cnf setting [mysqld] max_allowed_packet = 1000M
    Change in /Applications/MAMP/bin/php/php5.5.10/conf/php.ini settings
    max_execution_time = 259200,
    max_input_time = 259200,
    memory_limit = 6144M,
    upload_max_filesize = 1000M,
    post_max_size = 1000M,
    mysql.connect_timeout = 1000.
    Add or change in Applications/MAMP/bin/phpMyAdmin/config.inc.php setting $cfg['ExecTimeLimit'] = 0;
    Don’t forget to restart MAMP.

    I ran the conversion with the following settings: Rows Limit 10,000 and Delay Time 0. During the conversion my memory was around 50% and CPU was very low. Most likely it is possible to squeeze out more performance.

    I did a lot of random checks and the outcome seems ok-ish. Not quite sure about some things and have some questions. Please bear with me ;)

    Some topics are not combined correctly. All replies are separate topics. The ‘Recalculate the parent topic for each post’ does not fix this. I ran all repair queries btw. An option in the Topics interface to combine these, would be ace ;) Is it possible to correct this with a query? My SQL skills are too limited unfortunately.

    The Repair Forums option ‘Recalculate the position of each reply’ does not work. It creates a white screen. Load problem?

    myBB accepts spaces in the login name. Unfortunately WordPress does not. The conversion leaves the spaces in the login name. Is it possible to correct this with an SQL? Is there another workaround? Correcting them one by one is not my preference ;)

    Two forums are created double. The copies do not contain any topics or replies, so it is easy to remove them. Just thought you should know ;)

    The original myBB forum contained 1mlj+ messages and 36k+ discussions. The converted forum contains 109k subjects. That can be explained because of the first issue. But it only contains 830k replies. It could be that myBB adds subjects and replies to messages, but the difference is a bit too big. Is it possible to do an SQL checksum to check the expected output?

    Thanks in advance for your time!


    Stephen Edgar
    Moderator

    @netweb

    Sorry for the delay, life has been getting in the way.

    Thanks for listing those tweaks you made, they make sense, we already bump the PHP timeout and up the memory limit to 256M but that is all we can really do safely before we cause grief in shared hosting environments.

    I linked this post from our Import Troubleshooting docs so when I get the time we can expand that section (or a dedicated section) on tweaking ‘all the things’.

    It doesn’t look I have asked this previously, what version of MyBB are you importing from?

    All my tests have been with MyBB v1.6.10

    Some topics are not combined correctly. All replies are separate topics. The ‘Recalculate the parent topic for each post’ does not fix this. I ran all repair queries btw. An option in the Topics interface to combine these, would be ace ;) Is it possible to correct this with a query? My SQL skills are too limited unfortunately.

    I just ran an MyBB import of my ‘standard’ MyBB database and without running any repair tools replies are correctly associated with their parent topics.

    https://cloudup.com/cyPQPLOe7iF

    As to the possibility of correcting this with a MySQL query, it could be, it’s hard to say without knowing what data is mismatched.

    Run the following query in phpMyAdmin:
    83 is my topic ID
    97,99,100 and 102 are the reply ID’s I know should be part of the topic ID 83
    – A single topic and a couple of replies should be all I need to see what’s going on

    SELECT * FROM wp_posts WHERE ID IN (83,97,99,100,102)

    Once you’ve executed the query, click ‘Print View’ and the results should open in a new browser tab, select all the text and paste them into a GitHub Gist and select SQL from the language dropdown (You don’t need an account for this)

    https://gist.github.com/

    It should look similar to this (this is based on my above screenshot)

    https://gist.github.com/ntwb/04ed1cf9cf5f67528fcb

    The Repair Forums option ‘Recalculate the position of each reply’ does not work. It creates a white screen. Load problem?

    Yes, sadly, this is because of the size of the forums, kind of a know issue so that is good, sortof. Anyway the best thing is is that you do not need to run this tool. This is for only repairing ‘threaded replies’ which MyBB did not have so you can just skip running this tool. I’ll update the verbiage for this text in the next release and make some improvements when used with ‘large’ sites and some error handling.

    myBB accepts spaces in the login name. Unfortunately WordPress does not. The conversion leaves the spaces in the login name. Is it possible to correct this with an SQL? Is there another workaround? Correcting them one by one is not my preference

    I’ll work out a fix for that shortly, second time I’ve hit this issue today, @Halo-DieHard also did on BuddyPress.org.

    Two forums are created double. The copies do not contain any topics or replies, so it is easy to remove them. Just thought you should know

    That shouldn’t be the case, though as you point out easily fixed, most likely one of two things, either those forums were still there from a previous import OR there is something funky with the forum categories versus the forum forums though checking my import I just did, everything is fine for me ;)

    The original myBB forum contained 1mlj+ messages and 36k+ discussions. The converted forum contains 109k subjects. That can be explained because of the first issue. But it only contains 830k replies. It could be that myBB adds subjects and replies to messages, but the difference is a bit too big. Is it possible to do an SQL checksum to check the expected output?

    It does sound a little off, some forum software combine the total topics and replies to create a total post count.

    On my MyBB I have:

    
    Our members have made a total of 22 posts in 10 threads.
    We currently have 6 members registered.
    

    After importing using the [bbp-stats] shortcode I have
    Topics 10, Replies 12

    So that works out correct in my case, total of 22 posts = 10 topics + 12 replies :)

    It’s hard to match up yours because the issue above as you state, ball park 109k + 830k = 93k which is a few short of the 1m, this will be due to those issues above and we can take another look after they’re fixed.


    EzYRiDaH
    Participant

    @ezyridah

    Hi Stephen / @netweb,
    Many thanks for your replies! I am trying to convert MyBB 1.6.11. It is a very old forum which (unfortunately) was customized improperly by the previous owner. It was lagging many many versions. I paid a student to do an upgrade about a year ago, but there is some funky **** still in the PHP. With my limited knowledge of MyBB I even tried to a fresh install of MyBB and imported the complete forum in it. Pulled all my hairs out when I found out MyBB stores code in the database…

    I’ve stressed my dear wife to start cleaning the old forum, which she’s (finally) diligently doing now.

    Don’t worry too much about making any updates on the code :) I also have some kind of life and job next to trying to convert a MyBB forum ;) After all the cleaning of the old forum, I will do another test run to see if I can find any other issues. It is very likely that some errors are just because it is an ancient forum (10 years old). Read up on the conversion of @Halo-DieHard. Will make sure I do a repair/optimization of the old forum. To be honest, I am not sure if I did that with the last conversion, sorry.

    I will report back on the forum on what I was able to squeeze out on performance with MAMP, with what settings and the results.

    Another (most likely dumb) question I forgot to ask. This could be the answer to the missing(?) topics/replies. At the end of the conversion, the program shows in short intervals ‘-‘ on the screen. This takes forever and lasted almost as long as the initial part of the conversion which grabbed the Rows. I thought the conversion was finished when the ‘-‘ stopped appearing on screen ‘————‘ etc. Was I supposed to wait for some kind of other message which indicates the conversion ended?

    In the meantime I am busy integrating the plugin USERPRO and GD bbPress Toolbox. Never coded in PHP and already hate the syntax ;) I will get there and report back on the results!

    Thanks again for your time!

    Another (most likely dumb) question I forgot to ask. This could be the answer to the missing(?) topics/replies. At the end of the conversion, the program shows in short intervals ‘-’ on the screen. This takes forever and lasted almost as long as the initial part of the conversion which grabbed the Rows. I thought the conversion was finished when the ‘-’ stopped appearing on screen ‘————’ etc. Was I supposed to wait for some kind of other message which indicates the conversion ended?

    This is basically WordPress losing the connection with the MySQL database and is a right pain, once this occurs “nothing” is now happening, the import is essentially stopped.

    So in answer to your question, most likely this is the result of the missing topic/replies.

    So yes, at the end of the conversion, you’ll see “Conversion Complete” with a link to the repair tools.

    With most of the databases I have had to test to get compatibility with the importers many of the sample databases I have been sent (I don’t have a license for most of the commercial forum packages) I make a copy of the database and then prune/delete all but ~10 topics, the topics replies, the topic and reply users. This makes testing a breeze as it completes in a couple of minutes, once I know all is good then I give it a whirl on the full database.


    EzYRiDaH
    Participant

    @ezyridah

    Thanks for your reply! That explains a lot. Strange it happened, but I will check it again with the next trial run. Will keep you posted ;)

Viewing 10 replies - 1 through 10 (of 10 total)

You must be logged in to reply to this topic.