I thought I’d leave the solution I used in case it can help anyone else that might come across this. I was already using Private groups like suggested above so poked around in the settings to see how I could make it work. This might not be the most elegant way but it works.
Group Name Settings: Add a general group for all users and a group for admins.
Topic Permissions: Check “Activate Forum Permissions”
User Management: Here you can bulk add all the users to the general group.
Forum: Open the forum you would like to restrict to admin posting only and scroll to the “Forum Groups” meta box. Click the general and admin groups and update the page (Click any additional groups you might already have set up as well). When the page reloads you’ll see additional visibility options for each group. For the general user (and any other users) choose “Only View Topics/Replies” and for the admin leave it as “Create/Edit Topics and Replies (and view all topics and/replies)”.
I had to set it up this way because the “Non-logged in users” option wouldn’t work because we were restricting the forum to logged-in users only.
Hopefully, this helps anyone that might be facing the same issue. This one took me a bit to sort out.
