There is compilation caching so only the sketch should be recompiled every time, unless you have changed the board selection or modified library files.
The caching system has been improved over time and you are using an outdated version of the Arduino IDE so you might experience improved efficiency if you update to the latest version.