Ключевые слова:bsd, apache, optimization, (найти похожие документы)
From: Aaron Gifford <[email protected]>
Subject: Настрока FreeBSD для повышения производительности apache httpd (eng)
Running a High-Performance Web Server for BSD
Like other OS's, the listen queue is often the first limit hit. The
following are comments from "Aaron Gifford <[email protected]>" on
how to fix this on BSDI 1.x, 2.x, and FreeBSD 2.0 (and earlier):
Edit the following two files:
/usr/include/sys/socket.h
/usr/src/sys/sys/socket.h
In each file, look for the following:
/*
* Maximum queue length specifiable by listen.
*/
#define SOMAXCONN 5
Just change the "5" to whatever appears to work. I bumped the two
machines I was having problems with up to 32 and haven't noticed the
problem since.
After the edit, recompile the kernel and recompile the Apache server
then reboot.
FreeBSD 2.1 seems to be perfectly happy, with SOMAXCONN set to 32
already.
Addendum for very heavily loaded BSD servers
from Chuck Murcko <[email protected]>
If you're running a really busy BSD Apache server, the following are
useful things to do if the system is acting sluggish:
* Run vmstat to check memory usage, page/swap rates, etc.
* Run netstat -m to check mbuf usage
* Run fstat to check file descriptor usage
These utilities give you an idea what you'll need to tune in your
kernel, and whether it'll help to buy more RAM. Here are some BSD
kernel config parameters (actually BSDI, but pertinent to FreeBSD and
other 4.4-lite derivatives) from a system getting heavy usage. The
tools mentioned above were used, and the system memory was increased
to 48 MB before these tuneups. Other system parameters remained
unchanged.
maxusers 256
Maxusers drives a lot of other kernel parameters:
* Maximum # of processes
* Maximum # of processes per user
* System wide open files limit
* Per-process open files limit
* Maximum # of mbuf clusters
* Proc/pgrp hash table size
The actual formulae for these derived parameters are in
/usr/src/sys/conf/param.c. These calculated parameters can also be
overridden (in part) by specifying your own values in the kernel
configuration file:
# Network options. NMBCLUSTERS defines the number of mbuf clusters and
# defaults to 256. This machine is a server that handles lots of traffic,
# so we crank that value.
options NMBCLUSTERS=4096 # mbuf clusters at 4096
#
# Misc. options
#
options CHILD_MAX=512 # maximum number of child processes
options OPEN_MAX=512 # maximum fds (breaks RPC svcs)
In many cases, NMBCLUSTERS must be set much larger than would appear
necessary at first glance. The reason for this is that if the browser
disconnects in mid-transfer, the socket fd associated with that
particular connection ends up in the TIME_WAIT state for several
minutes, during which time its mbufs are not yet freed. Another reason
is that, on server timeouts, some connections end up in FIN_WAIT_2
state forever, because this state doesn't time out on the server, and
the browser never sent a final FIN. For more details see the
[1]FIN_WAIT_2 page.
Some more info on mbuf clusters (from sys/mbuf.h):
/*
* Mbufs are of a single size, MSIZE (machine/machparam.h), which
* includes overhead. An mbuf may add a single "mbuf cluster" of size
* MCLBYTES (also in machine/machparam.h), which has no additional overhead
* and is used instead of the internal data area; this is done when
* at least MINCLSIZE of data must be stored.
*/
CHILD_MAX and OPEN_MAX are set to allow up to 512 child processes
(different than the maximum value for processes per user ID) and file
descriptors. These values may change for your particular configuration
(a higher OPEN_MAX value if you've got modules or CGI scripts opening
lots of connections or files). If you've got a lot of other activity
besides httpd on the same machine, you'll have to set NPROC higher
still. In this example, the NPROC value derived from maxusers proved
sufficient for our load.
To increase the size of the listen() queue, you need to adjust the
value of SOMAXCONN. SOMAXCONN is not derived from maxusers, so you'll
always need to increase that yourself. We use a value guaranteed to be
larger than Apache's default for the listen() of 128, currently. The
actual value for SOMAXCONN is set in sys/socket.h. The best way to
adjust this parameter is run-time, rather than changing it in this
header file and thus hardcoding a value in the kernel and elsewhere.
To do this, edit /etc/rc.local and add the following line:
/usr/sbin/sysctl -w kern.somaxconn=256
We used 256 but you can tune it for your own setup. In many cases,
however, even the default value of 128 (for later versions of FreeBSD)
is OK.
Caveats
Be aware that your system may not boot with a kernel that is
configured to use more resources than you have available system RAM.
ALWAYS have a known bootable kernel available when tuning your system
this way, and use the system tools beforehand to learn if you need to
buy more memory before tuning.
RPC services will fail when the value of OPEN_MAX is larger than 256.
This is a function of the original implementations of the RPC library,
which used a byte value for holding file descriptors. BSDI has
partially addressed this limit in its 2.1 release, but a real fix may
well await the redesign of RPC itself.
Finally, there's the hard limit of child processes configured in
Apache.
For versions of Apache later than 1.0.5 you'll need to change the
definition for HARD_SERVER_LIMIT in httpd.h and recompile if you need
to run more than the default 150 instances of httpd.
From conf/httpd.conf-dist:
# Limit on total number of servers running, i.e., limit on the number
# of clients who can simultaneously connect --- if this limit is ever
# reached, clients will be LOCKED OUT, so it should NOT BE SET TOO LOW.
# It is intended mainly as a brake to keep a runaway server from taking
# Unix with it as it spirals down...
MaxClients 150
Know what you're doing if you bump this value up, and make sure you've
done your system monitoring, RAM expansion, and kernel tuning
beforehand. Then you're ready to service some serious hits!
Thanks to Tony Sanders and Chris Torek at BSDI for their helpful
suggestions and information.
"M. Teterin" <[email protected]> writes:
It really does help if your kernel and frequently used utilities
are fully optimized. Rebuilding the FreeBSD kernel on an AMD-133
(486-class CPU) web-server with
-m486 -fexpensive-optimizations -fomit-frame-pointer -O2
helped reduce the number of "unable" errors, because the CPU was
often maxed out.
Accept filtering on FreeBSD
Versions of FreeBSD from August 2000 onwards include a feature called
"accept filters" which delay the return from accept() until a
condition has been met, e.g. an HTTP request has arrived. This
postpones the requirement for a child process to handle the new
connection which therefore increases the number of connections that a
given number of child processes can handle. It also allows a child
process to accomplish more immediately after accept() returns (because
the request is already available to be read) so there is less context
switching.
There are two filters in FreeBSD at the time of writing: "dataready"
and "httpready". The former just waits for the first packet to arrive
from the client; the latter waits for the end of the HTTP headers.
Unfortunately the "httpready" filter breaks support for HTTP/0.9
(which doesn't have headers) so Apache doesn't use it, but the
"dataready" filter gives the same benefit in the majority of cases so
Apache attempts to use that instead.
Accept filters provide the most benefit on servers that are already so
busy that they are configured with "KeepAlive Off". [2]HTTP KeepAlive
(aka persistent connections) avoids the cost of setting up a new
connection for every request, but connections that are being kept
alive use up one of the available child processes. Since there is a
limited number of child processes this can significantly reduce the
capacity of the server. The viewers of a web site will still get a lot
of the benefit of persistent connections even with a very small
KeepAliveTimeout so you should try reducing it before turning it off
altogether.
To enable accept filtering, you must either load the appropriate
accept filter module, e.g. with the command kldload accf_data, or
compile a kernel with options ACCEPT_FILTER_DATA. Apache will then
enable filtering when it is restarted.
If you are more concerned about performance than compatibility with
absurdly obsolete HTTP/0.9 user agents then you can recompile Apache
to use the "httpready" filter. This may be particularly helpful if
your web site uses really big cookies, for example. If you are using
src/Configure then add -DACCEPT_FILTER_NAME=\"httpready\" to the
EXTRA_CFLAGS line in the src/Configuration file. If you are using
APACI (aka ./configure) then use the command
CFLAGS=-DACCEPT_FILTER_NAME=\'\"httpready\"\' ./configure (with all
the funky backslashed quotes).
