will be fine, but you’ll likely want to avoid having private or hidden forums. bbPress performs a postmeta query to exclude topics and replies from private and hidden forums, which will be a pretty bad bottleneck.
Hopefully in bbPress 2.4 or 2.5, I’d like to move the last-active-time meta queries to use post_modified in the posts table. That will remove some of our dependency on JOINing tables, and speed things up for you pretty significantly (at the cost of bastardizing the core post schema and its intentions a bit.)
If you have other specific questions as you go, happy to help and be a resource.