# [TIL-4] Little’s law, Tuning the thread pool size

Back then, when the company where I work was creating an event that expected to have 10 x incoming traffic, I was assigned a task by my Tech Lead to conduct stress tests for our application services.

This is necessary to know how many services we needed to scale out and find out the bottleneck at our application services.

During the stress tests, we found that our application services have problems with the thread pool. And we decided to tune the number of our thread pool.

If we follow “**measure, don’t assume**” we need to initially take a gander at the technology that is being referred to and ask what measurements make sense and how we instrument our framework to acquire them. We also need to carry some mathematics to the table.

I’m asking my Tech Lead what is the **formula **or **rule of thumb** to know how many thread pools we need, he said that there’s a law that is called **Little’s law**, and then we calculated together and found out the suitable size for our thread pool.

After that, I’m trying to dig out that **Little’s law**. And here’s what I’ve found by exploring internet sources.

# Thread Pool

Before we go through Little’s law, it’s a good idea to know about “What is a thread pool?”

In web applications,** thread pool** **size **decides the number of concurrent requests that can be taken care of at any given time.

In the event that a web application gets a greater number of requests than thread pool size, overabundance requests are either queued or rejected.

# Little’s law

In queueing theory, a discipline within the mathematical theory of probability,

Little’s result,theorem,lemma,law, orformulais a theorem by John Little which states that the long-term average numberLof customers in a stationary system is equal to the long-term average effective arrival rateλmultiplied by the average timeWthat a customer spends in the system

Expressed algebraically the law is

L = λW

In context web applications we could say

The average number of threads in a system (Threads) is equal to the average number of request arrival rate (Request per sec), multiplied by the average response time (Response time).

**Threads **= Number of Threads**Request per sec **= Number of Web Requests that can be processed in one second**Response time **= Time is taken to process one web request

Threads=Request per secxResponse time

# Example

Requests are showing up at a worker at the pace of 10 Requests/second. Each request takes 2 seconds to finish. What number of threads are required?

λ =10 rps, W = 2 seconds

L = 10 * 2

L = 20

Therefore for the case above we need 20 threads to handle the pace