To be honest, I don’t know what the dashes mean, I have seen them myself and tried to duplicate the issue so we can ‘fix it’ but it happens so rarely to me I haven’t been able to find the ‘problem’ to create a ‘fix’.
I do think once ‘the dashes’ do appear there is no easy way to recover from it though without restarting everything…
With imports as big as yours they take a very long time and is frustrating to start an import and many hours later find it didn’t work, change one setting and start the waiting game again!
So before I go through a few options, a couple of questions:
- What is your skill level of MySQL?
- Are you comfortable using phpMyAdmin?
- You live in phpMyAdmin!
- You dream MySQL queries in your sleep!
- What is your skill level of Linux/Apache/MySQL/PHP?
- Could you setup a local LAMP install on your ‘puter?
- I already have a local LAMP install!
- I have a LAMP install with a 4GB MySQL RAM Drive!
Thanks for your reply…
Last night after I posted, I ended up completely logging out and logging back in…and then restarted the import. That seemed to get rid of the dashes. So I’m not sure if I could reproduce them at this point.
The importer appears to have picked up where it left off before at 1600, but it has been very sporadic as far as how far it gets before it stalls. It did around 20,000 before I left it for the night, but then it only did about 100 more between that time and checking it this morning. It’s now going again and I’m up to about 31,000. I tried adjusting the limits lower on the rows and higher on the delay, but it doesn’t seem to matter.
And yes, I do realize that this is a large import…but even so, this is not an active, booming community anymore. I’m just trying to get the site into a more sustainable system to keep the content out there for people to find. So if it takes a long time to do the import and it doesn’t complete perfectly, I’m actually ok with that. As long as the bulk of it is there in some semblance of organization, I’m good. Maybe someday it will be active again…I can hope.
To answer your questions…yes, I’m comfortable using phpMyAdmin. As far as the LAMP goes, all I have is a mac here at home and from what I understand, it’s kinda complicated to get a LAMP running on a mac? If that’s not the case…or even if it’s a somewhat lengthy process, but not too hair-pulling…I’m willing to give it a shot.
Another thing I have not tried, but could… I actually did a conversion from vBulletin to Vanilla with no issues…and sorry, but it was fast! But I’ve decided not to use Vanilla. I could try importing from that db to see what it ends up looking like if I can’t get this one to work.
I had a larger vbulletin forum that I tried to import from and ended up having to migrate from vbulletin to phpbb and then from phpbb to bbpress.
I’m having similar issues. I actually had to convert an old ikonboard to vbulletin. Then vbulletin to bbpress. It’s a massive one too (180k users, 1.8mil posts). I’m doing this all locally with the memory set very high.
All the users managed to get converted but the posts died at 600k. I’m having difficulty continuing the conversion, it won’t start back up. It says it’s working but just sits there.
yeah… I’ve ended up doing it locally too. I tried the Vanilla conversion from the migration I did previously and ended up getting tons of SQL errors. So I ended up just pulling out the tables from my vBulletin db that the bbPress converter actually uses and am using those to run the conversion locally.
So far so good on the local conversion. I’m up to about 225,000 on the replies with no stalls yet just using the default set of 100 and 1 sec delay.
Rachael
I’d like to know if you have any success with this, i posted a similar issue here but i havent had any luck rectifying it.
The last time test i tried i got to >300k (forum has ~400k) posts after the convertor stalled and requiring a restart a couple of times, until the most recent time tried to restart I got this error in the output window:
Repair any missing information: Continue
WordPress database error: [Table 'db_name.pbpbb_posts' doesn't exist]
SELECT convert(posts.post_id USING "utf8") AS post_id,convert(topics.topic_id USING "utf8") AS topic_id,convert(posts.forum_id USING "utf8") AS forum_id,convert(posts.topic_id USING "utf8") AS topic_id,convert(posts.poster_ip USING "utf8") AS poster_ip,convert(posts.poster_id USING "utf8") AS poster_id,convert(posts.post_subject USING "utf8") AS post_subject,convert(posts.post_text USING "utf8") AS post_text,convert(posts.post_time USING "utf8") AS post_time FROM pbpbb_posts AS posts LEFT JOIN pbpbb_topics AS topics USING (topic_id) WHERE posts.post_id != topics.topic_first_post_id LIMIT 304200, 100
Starting Conversion
Im currently running this on AWS (Amazon Servers) and had upgraded to the server to something with a bit more grunt but its not helping.
Ive also discovered, not sure if this is related, that none of the replies that are imported are assigned to a forum or a topic and when viewing a topic only the first post appears and none of the related replies.
looking forward to response.
mchl
Well… after my last post, the conversion got up to around 308,000 and stalled, and it wouldn’t start again. I started it again from scratch and it stalled again at the exact same group, so I figured there was something wrong with that batch of posts. I figured out where it had stalled and sure enough, there was a group of posts in one thread that had a bunch of hex numbers in the post title. So I cleared those out and started the conversion again and left it for the night.
When I got up this morning, it was up to around 471,000…so well past the mark that it had stalled before. So hopefully (majorly crossing my fingers here) it will be done when I get home.
As for the second part of your question about the posts not being with their topic… I think all that is supposed to be rectified once you complete the conversion and run all the clean-up tools that are on the Repair tab (think that’s the name of the tab).
Rachael
Ive managed to get a full forum migration to complete. After upgrading our server and trying on my local machine (mac laptop) i thought i’d have a try on my local machine at work (iMac with decent processor & ram), and after ~6.5hrs the whole forum had been migrated over.
After speaking with a couple of people it turns out that wp_insert_post has a memory leak, and could be causing the import process to slow down and eventually stall on a large migration. I’m happy for anyone to jump in and clarify whether this may be the case and expand on this.
After the migration I ran all the ‘repair forum’ functions and it assigned all the topics and replies to their correct parents.
Hope this helps.
Thanks
@mchl Excellent that you now have a working migration. I don’t know of any outstanding issues for any {{{wp_insert_post}}} memory leak, now could I see any issue regarding this in WP Trac.
@leahcar8 Looks like your getting closer and closer to getting there too 🙂
I am doing lots of work on all ~20 importers at the moment that with all this feedback I will try to look at optimizing both the importer and the SQL queries used for inclusion in bbPress 2.5.
All these little gotchyas, tips, tricks and feedback is extremely helpful so I will also get the docs updated to include as much of this info to. 🙂
Stephen, if you want admin/test access to a mostly working import from vbulletin to phpbb to bbpress, let me know if it would be helpful. There are quite a few users that have import_ in from of their usernames and duplicate users.
Cool… I Try to subscribe to all the importer topics here but occasionally miss some.
As to any ‘imported_username’ this is a feature of the importer so if a user already exists with that username an additional username is created with the prefix ‘imported_’ in front of the username. If you go to the WordPress User admin and select the ‘imported_username’ user and select delete on that user you will then be prompted to delete the user but attribute all posts to another user, select the original (i.e Non ‘imported_’ username) and all their posts, topics, replies etc will then be attributed to that user.
Yes, just tedious with hundreds/thousands of imported_users.
Just as a follow-up… my conversion did complete successfully! and I got it moved back to my hosting server. Everything seems to be working fine. I ran all the repair tools and that certainly fixed all the mis-matched posts…so that’s good. I’m not really sure how long it took, but I’m guessing it was about 8-10 hours. …and it seemed to go at a pretty stable pace throughout. I didn’t really notice a slow-down. I ran it on a macbook pro i7 w/ 4gb ram.
I have noticed that the post counts shown in the admin cp do not necessarily match what is shown on the user side, with the user side being correct (even after running the re-counters). It’s not that big of a deal, it just almost gave me a heart attack since I thought a whole group of posts was missing. But they’re there safe and sound after all. So I’m not sure what’s up with that.
So now I’m just trying to figure out what type of theme I want to use…and hopefully I can get the site active again.
@Stephen Edgar – if you are making changes to the vBulletin importer and would like to have someone test on a larger db, I would be glad to do it.
Thanks!
Rachael
@cashman wrote:
Yes, just tedious with hundreds/thousands of imported_users.
Agreed, I can only hack the URL of the WordPress admin dashboard to get me ~250 users at a time and yes it is tedious and in the past I have done most of this directly in phpMyAdmin with SQL.
@leahcar8/
Great news that the conversion finished 🙂
The post counts in the backend and frontend should match so that has me a little curious to what is going on with that… Any chance you could post a couple of screenshots showing those numbers?
Thanks for the testing offer, I’ll let you know 🙂
As I’ve spent the day playing with passwords, did anyone have any issues with users logging into WordPress after the import? Did their existing vBulletin passwords work?
I’d also appreciate a comment on this ticket in Trac for any password issues you found.
https://bbpress.trac.wordpress.org/ticket/1969
