This is not caused by idleserver, it's caused by having the game minimized. You can have idleserver on without minimizing the game (and you can have the game minimized without idleserver on, which results in the same extra lag).