Little's law
The average number of requests in a system equals arrival rate times time spent inside. For a serving stack it fixes the relationship between throughput, latency and how many requests must be in flight — you cannot improve all three at once.
- Requests resident in the system on average.
- Arrival rate, in requests per second.
- Average time a request spends in the system.