initial response time is too slow

the first request to the server takes atleast 3-5 seconds and subsequent requests if made within 5-10 seconds takes 400-700ms

it works fine for single request only if multiple requests are made at the same time this issue occurs

You first said that the first request took 3-5 seconds and the other requests took 400-700ms, but now you say the problem occurs only when you make multiple requests at the same time...? 

This may be related to the sessions. If you make 5000 requests from postman, they will all use the same session and will block each others because it all come from the same client. Check your session parameters in CI and in php.ini. It may be what is causing these issues.

the delay issue still exists even for 1 request but i can live with that
And i am not using sessions
i think issue is with the php in cpanel. Before i was testing with a html file outside CI directory. When i created a php file just to echo hardcoded string the issue still exist its not even passing 100 request at a time.
So its not a CI issue