Tuning for +10k TCP Connections

[IvanDyachenko]

• Coping with the TCP TIME-WAIT state on busy Linux servers
• Connection Management FAQ
• Running 1,000 Containers in Docker Swarm
• A Million-user Comet Application with Mochiweb, Part 1
• Tuning your Linux kernel and HAProxy instance for high loads
• Increase “Open Files Limit”
• Performance Tuning on Linux — TCP
• Linux on IBM Systems/ TCP/IP IPv4 settings
• Scaling to 12 Million Concurrent Connections: How MigratoryData Did It
• https://stackoverflow.com/a/3923785/5592388
• https://stackoverflow.com/a/40352004/5592388
• http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2013-January/025007.html
• http4s blaze client with connection pool is closing connection after processing a large response http4s/blaze#641

curl --location --verbose --request GET 'https://api.ozon.ru/composer-api.bx/page/json/v1?url=/category/9040/?layout_container=default%26layout_page_index=1%26page=0%26sold_out_page=1' \
> --header 'Accept: application/json' \
> --header 'User-Agent: OzonStore/430' \
> --header 'Host: api.ozon.ru'
Note: Unnecessary use of -X or --request, GET is already inferred.
* Expire in 0 ms for 6 (transfer 0x560e3d805fb0)
* ...
* Expire in 3 ms for 1 (transfer 0x560e3d805fb0)
*   Trying 185.73.195.192...
* TCP_NODELAY set
* Immediate connect fail for 185.73.195.192: Cannot assign requested address
* Closing connection 0
curl: (7) Couldn't connect to server

docker inspect -f '{{.State.Pid}}' marketplace-scheduler-185.73.195.192
nsenter -t 21284 -n netstat -atn | grep TIME_WAIT | wc -l
nsenter -t 21284 -n ss -tan state time-wait | wc -l

#366