I had the same idea but commented out the entire contents of the login-form.php.
Removing or renaming will cause and error so leave it in place. Just open it and comment out the contents from top to bottom. It’s the login form shown on the front page. This way when you are logged in, you will still see the your name and the admin link and logout option. This is working for me anyway.
