Oh Damn!!
I made a mistake, actually WP Super Cache was active. Previously it showed a “Delete Cache” link at the admin bar. Now it’s not showing and I thought it was not activated 🙁
Anyway, I wonder why the cache is not being invalidated when creating new replies.
Yeah, comes in backend, but not on frontend.