The default install has a single topic (“Your first topic — First Post. Woot!”) that is tagged bbPress. If you deleted that but the tag persists, you might need to go into the admin and Manage > Recount /bb-admin/site.php. There is one checkbox there that says “DELETE tags with no topics. Only functions if the above checked.” Try that.