When the backlog runs into the millions of messages, repeatedly opening each queue file and trying to deliver to each left-over recipient takes too much time. The situation can be improved by keeping some status information in quickly-accessible memory, so that the mail system knows when a deferred queue file is ready to be tried again.
Memory is a finite resource. A naive program keeps information in memory for every deferred queue file. This approach is bound to fail when the number of deferred queue files grows sufficiently large. When that happens, the mail system becomes wedged, keeps crashing due to lack of memory, and stays wedged until someone throws away enough messages that the queue fits in memory again.
When a message cannot be delivered upon the first attempt, the queue manager gives the queue file a time stamp into the future by some configurable amount of time. Queue files with future time stamps are ignored by the queue manager.
Whenever a repeat delivery attempt fails, the queue file time stamp is moved into the future by an amount of time equal to the age of the message. Thus, the time between delivery attempts doubles each time. This strategy effectively implements exponential backoff.