[Robin W]

Converting a Snitz access database forum to BBpress

This documents how I transferred from a website using Snitz and an access database to a new site with BBress using Mysql.

This briefing assumes knowledge of
• FTP programmes and uploading/downloading files
• A way to access your wordpress database eg myphpadmin
• A way to see your access database

Introduction

BBpress has a conversion tool that lets you import data from several other forum packages. However two things are needed for this to work.
1. The data in a MySQL database format
2. A convertor programme that understands the structure of the source (in this case Snitz) forum.

If you already have a MySQL verison of Snitz (Snitz supports Access, MS SQL, and MySQL), then you’ll still need to fix the date, and run a specific Snitz converter, but don’t need the upload to MySql in step 2.

Plan

The plan therefore is to
1. Fix date format (it is different between the two forums)
2. Convert the data to sql format
3. Run a snitz converter
4. Run repair to fix the links

Execution

I can only describe how I achieved this, I suspect there are lots of ways, but this one worked.

I had an access database on site x with hosting company a, and needed to transfer to a Mysql database on site y with hosting company b (basically I was taking over running someone else’s website management)

Step 1 fixing the data format

Snitz stores dates in the format yyyymmddhhmmss
BBpress stores dates in the format yyyy-mm-dd hh:mm:ss

You can fix this
• before you transfer the data to sql,
• after you have transferred it
• or during conversion,

this will depend on where you knowledge lies. But you will need to fix it. If you don’t fix this, all the entries you import will show as posted 2014 years ago (basically year 0)

I converted before transferring to sql, as I an old and used to working with flat files! If you are more capable in sql, you could no doubt write a small programme to change the data format after you’ve uploaded to sql, and if more capable in php you could do it within the converter programme (see later)

The Snitz database has two tables with dates in that need changing. These are :

FORUM_TOPICS
FORUM_REPLY

This is how they showed in my Snitz database, but the prefix may be different with yours.

In FORUM_TOPICS you need to change
T_DATE
T_LAST_POST

In FORUM_REPLY you need to change
T_REPLY

I downloaded the access database from the old site to my PC. I then exported the two tables to an excel spreadsheet (in Access go to external data, and look for export to Excel. You’ll need to have created a blank spreadsheet with the name you want to use first) and ran the formula

=CONCATENATE(MID(N2,1,4),"-",MID(N2,5,2),"-",MID(N2,7,2)," “,MID(N2,9,2),":",MID(N2,11,2),":",MID(N2,13,2))

Where column N contained the old date.

I then prefixed the original heading with OLD (eg heading T_DATE became OLDT_DATE) and headed the new column with the heading of the old one (eg T_DATE). I did this to a) preserve the original columns just in case, and b) to use for the clean slug in the conversion (see later).

I then deleted the old table in access, and imported the two excel ones back into Access (as before external data, then import).

I then had the same access database, but with the correct date formulas and a couple of extra columns with the old date format in them.

Step 2 – Convert access database to MySql database

The first thing you’ll need is a MySql database to upload to.

You can use the one you presumably already have with WordPress/BBpress. However you’ll just need to check that none of the names co-incide. Since my Snitz database used capitals (I don’t know if they all Snitz databases do), and BBpress uses lower case, I suspect that there is no risk in doing this, but one for you to consider.

I took the safety of creating a fresh blank database (my hosting package allows the creation of databases and manages them through phpMyadmin).
There are numerous programmes on the web that do access to Mysql conversion, but for no particular reason I chose Bullzip http://www.bullzip.com/.

However this needs a 32bit ODBC driver to work, so unless you already have Mysql on your local PC, you’ll need to download the driver from http://www.mysql.com/downloads/connector/odbc/ – choose the 32 bit one.

Once installed it is a doddle to do the conversion.
You’ll need :
The access database on your pc
The host address of your MySql database
The database name
The username
The password.

Run the wizard, and it’ll upload in minutes.

Finally if you’re going to do the date conversion discussed in Step 1 at this stage, now’s the time to do it.

Step 3 – Importing/converting the data

This is easy once you’ve got your head around it, but initially it can be very frustrating.

In essence you need to translate what one forum calls something to what the other needs.

For Snitz, I used the following translation

Forum section FORUM_FORUM forum
Forum id	FORUM_ID	_bbp_forum_id
Forum parent id	CAT_ID	_bbp_parent_id
Forum title	F_SUBJECT	post_title
Forum Slug  (see note 1)	FORUM_ID	post_name
Forum description	F_DESCRIPTION	post_content
Forum Display Order	F_ORDER	Menu_order
Forum Date Update		Post_date
		
Topic section	FORUM_TOPICS	topic
Topic id	TOPIC_ID	_bbp_topic_id
Forum id 	FORUM_ID	_bbp_forum_id
Topic author	T_AUTHOR	post_author
Topic title	T_SUBJECT	post_title
Topic Slug (see note 1)	T_DATE	post_name
Forum id (if no parent 0) 	FORUM_ID	post_parent
Topic date update 	T_DATE	post_date
	T_DATE	post_date_gmt
	T_LAST_POST_DATE	post_modified
	T_LAST_POST_DATE	post_modified_gmt
		
Tags section		
Ignored		
		
		
Post section	FORUM_REPLY	reply
Post id	REPLY_ID	_bbp_post_id
Forum id	FORUM_ID	_bbp_forum_id
Topic  id	TOPIC_ID	_bbp_topic_id
Author ip	R_IP	_bbp_author_ip
Post author	R_AUTHOR	post_author
Topic slug	R_DATE	post_name
Post Content	R_MESSAGE	post_content
Topic id	TOPIC_ID	post_parent
Topic date	R_DATE	post_date
		
User Section	FORUM_MEMBERS	user
Store old user id	MEMBER_ID	_bbp_user_id
	M_NAME	user_login
	M_PASSWORD	_bbp_password
	M_EMAIL	user_email
	M_HOMEPAGE	user_url
	M_AIM	aim
	M_YAHOO	yim
		

[NOTE if you copy this and post in a word doc, you should be able to get a nice tabbed table]
Note 1

I used OLDR_DATE & OLDT_DATE from the step 1 as my clean slugs. A slug is what BBpress uses for a URL to reference a post. It needs to be clean – ie not have formatting, odd characters like spaces etc. in it. I did debate using the topic ID, but was unsure if this was unique for BBpress – it may well be fine. So I used the references above as I was pretty sure that with our forum being not that busy, that no two people would have posted in the same second. If you choose to use some other field, such as topic ID, you’ll need to change the field in the converter document – see below.

This is then used against the example converter that BBpress supplies.

Whilst initially looking daunting, this is fairly easy to change to get it to work.

Step 3.1 Creating a Snitz.php

Firstly go to your site. Using FTP go to web/wp-content/plugins/bbpress/includes/admin/converters and locate example.php. FTP this to your PC and using an editor (notepad ++ is good – download it from http://notepad-plus-plus.org/) you need to change the default “froms” to those above

But first save it locally as “snitz.php”, and change the first couple of lines to read

/**
 * Implementation of snitz converter.
 */
class snitz extends BBP_Converter_Base

If you don’t do this, the converter won’t recognise it.

Then you need to work through each of the lines of code changing the “default” source database tables and names to the correct ones.

So looking at you new snitz.php, you’ll see for instance the first conversion is :

// Forum id. Stored in postmeta.
$this->field_map[] = array(
'from_tablename' => 'forum', 'from_fieldname' => 'forumid',
'to_type' => 'forum', 'to_fieldname' => '_bbp_forum_id'

So to get this to work for Snitz you need to change the source table from ‘forum’ to “FORUM_FORUM” and the from field needs changing from ‘forumd’ to’ _bbp_forum_id’

Making the lines now read :

// Forum id. Stored in postmeta.
$this->field_map[] = array(
'from_tablename' => 'FORUM_FORUM', 'from_fieldname' => 'FORUM_ID',
'to_type' => 'forum', 'to_fieldname' => '_bbp_forum_id'

Using the translation table above you work through each of the lines, doing the substitution.

I commented out the tags section, as I couldn’t see that Snitz had tags. I also commented out the “
Store old user salt” and “User password verify class” sections as passwords won’t come across – see later on this.

CRITICALLY I also added the following as the example doesn’t had a bit that brings the actual topic content across !

// Topic content.
		// Note: We join the posts table because topics do not have content.
		$this->field_map[] = array(
			'from_tablename'  => 'FORUM_TOPICS',
			'from_fieldname'  => 'T_MESSAGE',
			'join_tablename'  => 'thread',
			'join_type'       => 'INNER',
			'join_expression' => 'USING (threadid) WHERE post.parentid = 0',
			'to_type'         => 'topic',
			'to_fieldname'    => 'post_content',
			'callback_method' => 'callback_html'

Once all that is done, save the file locally.

Then using FTP upload it to the same directory that example came from
Viz : web/wp-content/plugins/bbpress/includes/admin/converters
Now you’re ready to run the conversion!

You’ll need the following info
Database server (localhost if your using you local Mysql database)
Database name
Database User
Database password

Then in Dashboard, go to tools>forums>import forums

Select “snitz” from the converter list, and fill in the database info from above.

Click to select users, presuming that you want to bring these across.

Then click start, and sit back and watch. If you’ve got everything right, you’ll see a steady stream of lines such as “converting topics 1-100”

If Fails :

If it falls over, then you’ll need to work out which part of the snitz converter it doesn’t like (it’ll give you a line number). Usually you’ve got punctuations wrong, or called something by a wrong name. Remember snitz (in my case anyway) uses capitals, and bbpress needs lowercase, so ensure that’s all ok.

If nothing comes through (it usually says “starting conversion…conversion complete”), then you have got the connection wrong, as it’s not finding the database and tables.

If Success :

Then you’ll need to repair the links (Dashboard>tools>forums>repair forums). Otherwise in my case the forum listings came through, but with no topics listed against them.

Do as the tool suggests – I ran several at the same time, and wrote most of this document whilst waiting !

Finally – users

If you successfully imported your users, they’ll have accounts, but their passwords won’t work – Bbpress can’t bring these across.

The easiest way is to tell them to click “lost password” and enter their email address and then they can reset their own.

And that’s how it’s done !

[Robin W]

have you followed step 1 in the first thread on changing date format ?

[stefyonweb]

Yes, dates are in the correct format yyyy-mm-dd

[Robin W]

ok, not sure what to suggest next

[stefyonweb]

Hi, I think I did it: the problem with the dates dipended on topics table structure.
Everything seems to work fine now.
Thank you for your precious support!

[Robin W]

that’s great – I’m glad you got there.

Could I ask you just to elaborate on ‘the problem with the dates depended on topics table structure’ to help others who might find this thread and have the same issue

[stefyonweb]

Sure: I had an mdb backup. For some reason I inverted two columns in my table structure, so when I imported from csv file I had 0000-00-00 00:00:00 in column LAST_POST. Just a stupid mistake ^__^
Unfortunately I still have some troubles. Lots of replies are in the database but not connected with their parent topics. I still have to figure out why, the database is huge, it’s very difficult to do some debug.
Anyway, I’ll tell you when everything is ok.
Thanks!

[Robin W]

thanks

[estakhrchi ali]

irstly I only included the fields Robin had supplied, I do not have a copy of Snitz to test either. So I commented out but didn’t remove any of the code so that it would stand out and you could update any of these as needed which you have done.

The forum ‘counts’ I think you may have incorrect in your post above:
* _bbp_topic_count – The number of topics that are ‘publicly’ visible
* _bbp_reply_count – The number of replies that are ‘publicly’ visible
* _bbp_total_topic_count – The total number of topics (includes topics that are are unapproved or hidden)
* _bbp_total_reply_count – The total number of replies (includes replies that are are unapproved or hidden)

A clearer explanation of each of these would be helpful and also are each of these fields in the FORUM_FORUM database table:
* F_TOPICS
* F_COUNT
* T_COUNT
* P_COUNT

The topic ‘counts’ is the same:ساخت استخر
* _bbp_reply_count – The number of replies that are ‘publicly’ visible
* _bbp_total_reply_count – The total number of replies (includes replies that are are unapproved or hidden)