There is no way to prevent the first load delay. At this time (during startup) the cache on the web server is being populated from the database.
Regarding subsequent delays, what happens is that IIS shuts down the web application when it has been idle for a certain period of time, then the web application needs some time to re-load all its data. Thus, you may want to consider extending the timeout: