Scaling Servers

While much of the focus of the early PERT has been on the performance of individual TCP connections, frequently one is interested in the aggregate performance of a server that handles many requests/connections in parallel. Performance aspects include:

• The capability of serving large numbers of clients
• Maintaining quick response time
• Maintaining good throughput

When tuning a server to be able to serve tens of thousands of simultaneous users, issues such as InterruptCoalescence, scheduling, application and threading performance, sufficient TCP listen backlogs, etc. become important factors.

An important approach to server scaling consists of "scaling out" to multiple servers that share the load. This can be done e.g. using cluster technology using Load Balancers^?, or through the use of Content Distribution Networks (CDNs).

Related Case Studies

• ApacheScaling - Scaling Apache 2.x beyond 20,000 concurrent downloads (on ftp.heanet.ie)

References

• The C10K problem, D. Kegel et al. This Web site deals with many aspects of server scaling beyond 10'000 connections (C10K).
• Building Scalable Web Services, T. Killalea, ACM Queue, October 2008

-- SimonLeinen - 02 Nov 2008 - 17 Dec 2008
-- PekkaSavola - 02 Oct 2008