工作,学习,生活,这里将会有一些记录. 备用域名:http://meisw.51099.com 注册 | 登陆
浏览模式: 标准 | 列表分类:squid/缓存

squid 配置参数

 #./configure --prefix=/usr/local/squid3 //指定软件的安装路径
--disable-internal-dns //使用自己内部DNS查询
--disable-ident-lookups //防止系统使用RFC931规定的身份识别方
--disable-carp //Cache数组路由协议(CARP)用来转发丢失的cache到父cache的数组或cluste
--disable-wccp //用于阻止或分发HTTP请求到一个或多个caches
--enable-gnuregex //支持GNU正则表达式
--enable-async-io=240 //等同于同时开启./configure如下三个选项:
--with-aufs-threads=N_THREADS
--with-pthreads
--enable-storeio=ufs,aufs


这个主要是设置async模式来运行squid,我的理解是设置用线程来运行squid,如果服务器配置很不错,有1G以上内存,cpu使用SMP的方式的话可以考虑设成160或者更高。
如果服务器比较糟糕就根据实际情况设了。另外此项还另cache文件支持aufs
--enable-icmp //加入ICMP支持
--enable-kill-parent-hack //关掉squid的时候,要不要连同父进程一起关掉
--with-maxfd=65535 //指定最大文件描述
--with-dl //
--with-large-files //
--enable-poll //指定使用Poll()函数,提升性能就是啦
--enable-linux-netfilter //可以支持透明代理
--enable-large-cache-files //开启大文件支持,支持2GB以上的文件
--enable-delay-pools //开启squid延时池功能
--enable-snmp //此选项可以让MRTG使用SNMP协议对服务器的流量状态进行监测,因此必须选择此项,使Squid支持SNMP接口
--enable-underscore //允许解析的URL中出现下划线,因为默认squid会认为带下划线的URL地址是非法的,并拒绝访问该地址
--enable-arp-acl //可以在规则设置中直接通过客户端的MAC地址进行管理,防止客户使用IP欺骗
--enable-cache-digests //加快请求时,检索缓存内容的速度
--enable-err-language=”Simplify_Chinese” //
--enable-default-err-languages=”Simplify_Chinese” //指定出错是显示的错误页面为简体中文

squid限制文件下载或访问

 禁止终端用户在任何客户机上下载文件扩展名为mp3、exe、zip和rar类型的文件
acl badfile urlpath_regex –i .mp3$ .exe$ .zip$ .rar$ .rmvb$ .rm$ .mp4$ ……
http_access deny badfile
禁止客户机IP地址在192.168.2.0子网的所有终端客户在星期一到星期五的9:00到18:00访问Internet资源
acl clientnet src 192.168.2.0/24
acl worktime time MTWHF 9:00-18:00
http_access deny clientnet worktime
限制IP地址为192.168.1.102的客户机并发连接的最大连接数为5
acl clientip src 192.168.1.102
acl clientmaxconn maxconn 5
http_access deny clientip clientmaxconn
禁止所有终端用户访问域名包含为google.com的网站
acl badurl url_regex –i google.com
http_access deny badurl
禁止所有终端用户访问域名为www.google.com的网站
acl baddomain dstdomain –i www.google.com
http_access deny baddomain

squid_https

 按照节点上的安装方法,安装完成后,再如下操作,完成https的添加和配置

 
yum install -y openssl-devel
 
cd /tmp
cd squid-3.1.22
make clean
./configure \
        --prefix=/www/wdlinux/squid-3.1.22 \
        --with-pthreads \
        --enable-storeio="aufs,ufs" \
        --enable-async-io \
        --disable-internal-dns \
        --enable-stacktraces \
        --disable-ident-lookups \
        --enable-removal-policies='heap,lru' \
        --with-aio \
        --with-filedescriptors=65536 --enable-ssl
 
make
make install
 
完成后,修改下配置文件
vim /www/wdlinux/squid/etc/squid.conf
在 http_port 80 vhost vport 行下面,添加如下一行
https_port 443 cert=/www/wdlinux/squid/etc/3_user_www.zgnpo.com.crt key=/www/wdlinux/squid/etc/4_user_www.zgnpo.com.key defaultsite=www.zgnpo.com
 
将证书文件传到
/www/wdlinux/squid/etc/目录下
 
然后重起下服务
service squid restart
 
就可以了
 
 

squid视频缓存

 --prefix=/usr/local/squid --enable-dlmalloc --with-pthreads --enable-epoll --enable-stacktrace --enable-removal-policies=heap,lru --enable-delay-pools --enable-storeio=aufs,coss,diskd,ufs --enable-snmp 

 

 

range_offset_limit -1

quick_abort_min -1 KB

refresh_pattern -i \.flv$ 1440 50% 2880 ignore-reload

 

http://bbs.linuxtone.org/thread-1933-1-1.html

 

 

使用Squid筹建内网视频缓存系统

http://www.myexception.cn/operating-system/2024535.html

HAProxy服务器配置手册

本文档覆盖了如上版本HAProxy的所有配置命令. 但是不提供任何的提示(hint)、示例(example)或建议.
如果你要获得此类信息,请查看“参考手册”和“架构手册”。

HAProxy的配置过程分为3个主要部分:
 - 命令行参数,这是最优先的
 - global(全局)段,设置进程级参数
 - 代理配置段,通常位于"default", "listen", "fronted", "backend"这样的形式内。

配置文件的语法是由以本手册描述的关键字后跟可选的一个或者多个参数(参数之间空格)
组成。如果字符串中包含空格,必须使用'\'进行转义。\本身需要使用\进行转义。

一些参数值为时间,比如说timeout。时间值通常单位为毫秒(ms),但是也可以通过加后缀
来使用其他的单位。这是很重要的,因为我并不会在每个关键字中说明这点。支持的单位为:

  - us : microseconds. 1 microsecond = 1/1000000 second
  - ms : milliseconds. 1 millisecond = 1/1000 second. This is the default.
  - s  : seconds. 1s = 1000ms
  - m  : minutes. 1m = 60s = 60000ms
  - h  : hours.   1h = 60m = 3600s = 3600000ms
  - d  : days.    1d = 24h = 1440m = 86400s = 86400000ms


1. Global(全局)参数
--------------------

"global" 段的参数是进程级的,通常和操作系统(OS)相关。这些参数一般只设置一次,如果
配置无误,就不需要再次配置进行修改。(译注:也就是说如果设置多个正确的,还是以第一个
为准,后面的无效)。 这些参数其中一些有对应的命令行参数。

global段支持的关键字如下:

 * 进程管理和安全
   - chroot
   - daemon
   - gid
   - group
   - log
   - nbproc
   - pidfile
   - uid
   - ulimit-n
   - user
   - stats
  
 * 性能调优
   - maxconn
   - noepoll
   - nokqueue
   - nopoll
   - nosepoll
   - spread-checks
   - tune.maxaccept
   - tune.maxpollevents
  
 * 调试
   - debug
   - quiet


1.1) 进程管理和安全
------------------------------------

chroot <jail dir>
  切换运行目录到<jail dir>并且在放弃超级权限前执行chroot()。这会增强HAProxy的安全
  级别。仅在HAProxy以超级用户启动时才能使用。要注意的是<jail dir>必须为空并且其他
  任何用户不可写。
  
  
daemon
  使HAProxy进程进入后台运行。这是推荐的运行模式。等同于命令行-D参数。命令行参数
  -db则为禁止此选项。

gid <number>
  进程运行的组ID。建议将HAProxy的运行gid设置为一个专有的,或者设置为和其他类似进程
  一样的gid。 HAProxy必须以属于此组的用户启动,或者使用root来启动。
  参考group 和 uid.
  
group <group name>
  和gid类似,不过这里使用/etc/group中对应组名的gid。
  参考gid和 uid.
  
log <address> <facility> [max level]
  添加一个全局的syslog服务器。 最多允许定义2个全局服务器。这些服务器将接收HAProxy的
  启动和退出日志,如果代理配置里设置了log global,可以接收所有日志。

  <address> 可以为:

        - 一个IPv4 IP地址后跟一个:和一个UDP端口。如果端口不指定, 默认使用514
          (标准的syslog端口)

        - 一个到UNIX domain socket文件的路径。注意chroot(此文件在chroot内能访问)
          和uid/gid(此文件可写).

  <facility> 必须为如下24种标准syslog设备的一种:

          kern   user   mail   daemon auth   syslog lpr    news
          uucp   cron   auth2  ftp    ntp    audit  alert  cron2
          local0 local1 local2 local3 local4 local5 local6 local7

  还可以设置一个可选的level来过滤发出日志。默认会发出所有日志。如果指定了level,
  等于或高于此级别的日志才会发出。 8个级别为:

      emerg  alert  crit   err    warning notice info  debug

nbproc <number>
  创建<number>个进程进入deamon模式运行。此参数要求将运行模式设置为"daemon"。
  默认只创建一个进程, 这也是推荐的运行模式。 对那些每个进程的文件描述符被限制为
  很小值的系统来说, 可能需要开启多个daemon进程。但是使用多个进程将使调试变
  得非常艰难,这是非常不鼓励使用的。
  参考"daemon"。
  
pidfile <pidfile>
  将所有进程的pid写入文件<pidfile>。 等同于命令行参数"-p"。 启动进程的用户必须
  有权限访问此文件。
  参考"daemon"

stats socket <path> [{uid | user} <uid>] [{gid | group} <gid>] [mode <mode>]
  在<path>下创建流模式的UNIX套接字. 任何已存的套接字会备份后会被替换。连接到
  此套接字并输入show stat后会得到一个CSV格式的输出,输出包括进程信息。输入show
  info将返回更详细的相关信息。

stats timeout <timeout, in milliseconds>
  默认stats套接字的超时时间为10s。可以用此参数修改,单位可以为us,ms,s,m,h,d后缀。

stats maxconn <connections>
  默认,stats套接字(socket)仅限10个并发连接。这个值可由此参数修改。

uid <number>
  设置进程用户ID为<number>.建议将HAProxy的运行uid设置为一个专有的,或者设置为和
  其他类似进程一样的uid。 HAProxy必须以root来启动从而切换到其他用户运行。
  参考gid和user.

ulimit-n <number>
  设置每个进程的最大文件描述符数。默认的, 此值会自动计算,所以不推荐设置此参数。

user <user name>
  和uid类似,不过这里使用/etc/passwd中对应用户名的uid。
  参考uid和group.


1.2) 性能调优
-----------------------

maxconn <number>
  Sets the maximum per-process number of concurrent connections to <number>. It
  is equivalent to the command-line argument "-n". Proxies will stop accepting
  connections when this limit is reached. The "ulimit-n" parameter is
  automatically adjusted according to this value. See also "ulimit-n".
  设置每个进程的最大并发连接数. 等于命令行参数"-n"。 如果达到此值,代理会拒绝接受
  新的连接。ulimit-n会根据这个值自动调整。

noepoll
  Disables the use of the "epoll" event polling system on Linux. It is
  equivalent to the command-line argument "-de". The next polling system
  used will generally be "poll". See also "nosepoll", and "nopoll".
  在Linux上禁止使用epoll。等于命令行"-de"参数。 下一个会用到的polling系统为
  poll。
  查看"noepoll"和"nopoll".
  

nokqueue
  Disables the use of the "kqueue" event polling system on BSD. It is
  equivalent to the command-line argument "-dk". The next polling system
  used will generally be "poll". See also "nopoll".
  在BSD上禁止kqueue polling系统。等于命令行"-dk"参数. 下一个会使用的是poll.
  查看"nopoll".

nopoll
  Disables the use of the "poll" event polling system. It is equivalent to the
  command-line argument "-dp". The next polling system used will be "select".
  It should never be needed to disable "poll" since it's available on all
  platforms supported by HAProxy. See also "nosepoll", and "nopoll" and
  "nokqueue".
  禁止poll,等于命令行"-dp"。下一个使用的是select。poll在任何能运行HAProxy的机器
  上都支持,因此不要禁止poll。
  查看"nospoll"和"nopoll"以及"nokquque".

nosepoll
  Disables the use of the "speculative epoll" event polling system on Linux. It
  is equivalent to the command-line argument "-ds". The next polling system
  used will generally be "epoll". See also "nosepoll", and "nopoll".
  在Linux上禁止使用speculative epoll系统。等于"-ds". 下一个使用的是epoll.
  查看"nospell", "nopoll".

spread-checks <0..50, in percent>
  Sometimes it is desirable to avoid sending health checks to servers at exact
  intervals, for instance when many logical servers are located on the same
  physical server. With the help of this parameter, it becomes possible to add
  some randomness in the check interval between 0 and +/- 50%. A value between
  2 and 5 seems to show good results. The default value remains at 0.

tune.maxaccept <number>
  Sets the maximum number of consecutive accepts that a process may perform on
  a single wake up. High values give higher priority to high connection rates,
  while lower values give higher priority to already established connections.
  This value is unlimited by default in single process mode. However, in
  multi-process mode (nbproc > 1), it defaults to 8 so that when one process
  wakes up, it does not take all incoming connections for itself and leaves a
  part of them to other processes. Setting this value to zero or less disables
  the limitation. It should normally not be needed to tweak this value.

tune.maxpollevents <number>
  Sets the maximum amount of events that can be processed at once in a call to
  the polling system. The default value is adapted to the operating system. It
  has been noticed that reducing it below 200 tends to slightly decrease
  latency at the expense of network bandwidth, and increasing it above 200
  tends to trade latency for slightly increased bandwidth.


1.3) 调试
---------------

debug
  启动debug模式,此模式会dump所有的交互信息到stdout,并运行于前台(译注:不进入deamon
  模式)。等效于命令行参数-d. 此参数绝不要用于生产环境,因为它会在系统启动时被挂起.

quiet
  在启动时不显示任何信息。等于命令行参数-v。


2) 代理(Proxies)
----------

Proxy configuration can be located in a set of sections :
代理相关配置位于下列配置段中:
 - defaults <name>
 - frontend <name>
 - backend  <name>
 - listen   <name>

A "defaults" section sets default parameters for all other sections following
its declaration. Those default parameters are reset by the next "defaults"
section. See below for the list of parameters which can be set in a "defaults"
section. The name is optional but its use is encouraged for better readability.
"defaults"段为其后的所有其他配置段设置默认参数。 "defaults"段可以有多个,后设置的总
是会覆盖之前的配置。 查看下面的列表可以知道"defaults"段可以使用哪些配置参数。"defaults"
关键字是可选的,但是为了更好的可读性,建议加上。

A "frontend" section describes a set of listening sockets accepting client
connections.
"frontend"段描述了一组监听的套接字,它们接受客户端连接。

A "backend" section describes a set of servers to which the proxy will connect
to forward incoming connections.
"backend"段描述了一组服务器,代理(Haproxy)会连接这些服务器并转发客户端请求到这些服
务器上。

A "listen" section defines a complete proxy with its frontend and backend
parts combined in one section. It is generally useful for TCP-only traffic.
"listen"段定义了一个完整的代理,它的前段(frontend)和后端(frontend)都在这个配置
段里。这种配置通常用于仅TCP的流量.

All proxy names must be formed from upper and lower case letters, digits,
'-' (dash), '_' (underscore) , '.' (dot) and ':' (colon). ACL names are
case-sensitive, which means that "www" and "WWW" are two different proxies.
代理名必须由大(小)写字母、数字、'-'、'_'、'.'、':'组成。ACL名字是大小写敏感的,
也即www和WWW分别指不同的代理。

Historically, all proxy names could overlap, it just caused troubles in the
logs. Since the introduction of content switching, it is mandatory that two
proxies with overlapping capabilities (frontend/backend) have different names.
However, it is still permitted that a frontend and a backend share the same
name, as this configuration seems to be commonly encountered.
由于历史原因,所有的代理名字是可以重叠的,这种仅仅会导致日志有些问题。 后来内容交换
(Content Switching)的加入使得两个有重复功能的代理(frontend/backend)必须使用不
同的名字。 然而, 仍然允许frontend和backend使用同一个名字,因为这种配置会经常遇到。


Right now, two major proxy modes are supported : "tcp", also known as layer 4,
and "http", also known as layer 7. In layer 4 mode, HAProxy simply forwards
bidirectionnal traffic between two sides. In layer 7 mode, HAProxy analyzes the
protocol, and can interact with it by allowing, blocking, switching, adding,
modifying, or removing arbitrary contents in requests or responses, based on
arbitrary criteria.
当前,HAProxy支持两种主要的代理模式: "tcp"也即4层,和"http",即7层。在4层模式下,
HAproxy仅在客户端和服务器之间转发双向流量。7层模式下,HAProxy会分析协议,并且能通过
允许、拒绝、交换、增加、修改或者删除请求(request)或者回应(response)里指定内容来控制
协议,这种操作要基于特定规则。



2.1) Quick reminder about HTTP
------------------------------

When a proxy is running in HTTP mode, both the request and the response are
fully analyzed and indexed, thus it becomes possible to build matching criteria
on almost anything found in the contents.

However, it is important to understand how HTTP requests and responses are
formed, and how HAProxy decomposes them. It will then become easier to write
correct rules and to debug existing configurations.


2.1.1) The HTTP transaction model
---------------------------------

The HTTP protocol is transaction-driven. This means that each request will lead
to one and only one response. Traditionnally, a TCP connection is established
from the client to the server, a request is sent by the client on the
connection, the server responds and the connection is closed. A new request
will involve a new connection :

  [CON1] [REQ1] ... [RESP1] [CLO1] [CON2] [REQ2] ... [RESP2] [CLO2] ...

In this mode, called the "HTTP close" mode, there are as many connection
establishments as there are HTTP transactions. Since the connection is closed
by the server after the response, the client does not need to know the content
length.

Due to the transactional nature of the protocol, it was possible to improve it
to avoid closing a connection between two subsequent transactions. In this mode
however, it is mandatory that the server indicates the content length for each
response so that the client does not wait indefinitely. For this, a special
header is used: "Content-length". This mode is called the "keep-alive" mode :

  [CON] [REQ1] ... [RESP1] [REQ2] ... [RESP2] [CLO] ...

Its advantages are a reduced latency between transactions, and less processing
power required on the server side. It is generally better than the close mode,
but not always because the clients often limit their concurrent connections to
a smaller value. HAProxy currently does not support the HTTP keep-alive mode,
but knows how to transform it to the close mode.

A last improvement in the communications is the pipelining mode. It still uses
keep-alive, but the client does not wait for the first response to send the
second request. This is useful for fetching large number of images composing a
page :

  [CON] [REQ1] [REQ2] ... [RESP1] [RESP2] [CLO] ...

This can obviously have a tremendous benefit on performance because the network
latency is eliminated between subsequent requests. Many HTTP agents do not
correctly support pipelining since there is no way to associate a response with
the corresponding request in HTTP. For this reason, it is mandatory for the
server to reply in the exact same order as the requests were received.

Right now, HAProxy only supports the first mode (HTTP close) if it needs to
process the request. This means that for each request, there will be one TCP
connection. If keep-alive or pipelining are required, HAProxy will still
support them, but will only see the first request and the first response of
each transaction. While this is generally problematic with regards to logs,
content switching or filtering, it most often causes no problem for persistence
with cookie insertion.


2.1.2) HTTP request
-------------------

First, let's consider this HTTP request :

  Line     Contents
  number 
     1     GET /serv/login.php?lang=en&profile=2 HTTP/1.1
     2     Host: www.mydomain.com
     3     User-agent: my small browser
     4     Accept: image/jpeg, image/gif
     5     Accept: image/png


2.1.2.1) The Request line
-------------------------

Line 1 is the "request line". It is always composed of 3 fields :

  - a METHOD      : GET
  - a URI         : /serv/login.php?lang=en&profile=2
  - a version tag : HTTP/1.1

All of them are delimited by what the standard calls LWS (linear white spaces),
which are commonly spaces, but can also be tabs or line feeds/carriage returns
followed by spaces/tabs. The method itself cannot contain any colon (':') and
is limited to alphabetic letters. All those various combinations make it
desirable that HAProxy performs the splitting itself rather than leaving it to
the user to write a complex or inaccurate regular expression.

The URI itself can have several forms :

  - A "relative URI" :

      /serv/login.php?lang=en&profile=2

    It is a complete URL without the host part. This is generally what is
    received by servers, reverse proxies and transparent proxies.

  - An "absolute URI", also called a "URL" :

      http://192.168.0.12:8080/serv/login.php?lang=en&profile=2

    It is composed of a "scheme" (the protocol name followed by '://'), a host
    name or address, optionally a colon (':') followed by a port number, then
    a relative URI beginning at the first slash ('/') after the address part.
    This is generally what proxies receive, but a server supporting HTTP/1.1
    must accept this form too.

  - a star ('*') : this form is only accepted in association with the OPTIONS
    method and is not relayable. It is used to inquiry a next hop's
    capabilities.
        
  - an address:port combination : 192.168.0.12:80
    This is used with the CONNECT method, which is used to establish TCP
    tunnels through HTTP proxies, generally for HTTPS, but sometimes for
    other protocols too.

In a relative URI, two sub-parts are identified. The part before the question
mark is called the "path". It is typically the relative path to static objects
on the server. The part after the question mark is called the "query string".
It is mostly used with GET requests sent to dynamic scripts and is very
specific to the language, framework or application in use.


2.1.2.2) The request headers
----------------------------

The headers start at the second line. They are composed of a name at the
beginning of the line, immediately followed by a colon (':'). Traditionally,
an LWS is added after the colon but that's not required. Then come the values.
Multiple identical headers may be folded into one single line, delimiting the
values with commas, provided that their order is respected. This is commonly
encountered in the "Cookie:" field. A header may span over multiple lines if
the subsequent lines begin with an LWS. In the example in 2.1.2, lines 4 and 5
define a total of 3 values for the "Accept:" header.

Contrary to a common mis-conception, header names are not case-sensitive, and
their values are not either if they refer to other header names (such as the
"Connection:" header).

The end of the headers is indicated by the first empty line. People often say
that it's a double line feed, which is not exact, even if a double line feed
is one valid form of empty line.

Fortunately, HAProxy takes care of all these complex combinations when indexing
headers, checking values and counting them, so there is no reason to worry
about the way they could be written, but it is important not to accuse an
application of being buggy if it does unusual, valid things.

Important note:
   As suggested by RFC2616, HAProxy normalizes headers by replacing line breaks
   in the middle of headers by LWS in order to join multi-line headers. This
   is necessary for proper analysis and helps less capable HTTP parsers to work
   correctly and not to be fooled by such complex constructs.


2.1.3) HTTP response
--------------------

An HTTP response looks very much like an HTTP request. Both are called HTTP
messages. Let's consider this HTTP response :

  Line     Contents
  number 
     1     HTTP/1.1 200 OK
     2     Content-length: 350
     3     Content-Type: text/html


2.1.3.1) The Response line
--------------------------

Line 1 is the "response line". It is always composed of 3 fields :

  - a version tag : HTTP/1.1
  - a status code : 200
  - a reason      : OK

The status code is always 3-digit. The first digit indicates a general status :
 - 2xx = OK, content is following   (eg: 200, 206)
 - 3xx = OK, no content following   (eg: 302, 304)
 - 4xx = error caused by the client (eg: 401, 403, 404)
 - 5xx = error caused by the server (eg: 500, 502, 503)

Please refer to RFC2616 for the detailed meaning of all such codes. The
"reason" field is just a hint, but is not parsed by clients. Anything can be 
found there, but it's a common practice to respect the well-established
messages. It can be composed of one or multiple words, such as "OK", "Found",
or "Authentication Required".


2.1.3.2) The response headers
-----------------------------

Response headers work exactly like request headers, and as such, HAProxy uses
the same parsing function for both. Please refer to paragraph 2.1.2.2 for more
details.


2.2) 代理关键字矩阵
----------------------------

The following list of keywords is supported. Most of them may only be used in a
limited set of section types. Some of them are marked as "deprecated" because
they are inherited from an old syntax which may be confusing or functionally
limited, and there are new recommended keywords to replace them. Keywords
listed with [no] can be optionally inverted using the "no" prefix, ex. "no
option contstats". This makes sense when the option has been enabled by default
and must be disabled for a specific instance.
下面是HAProxy目前支持的关键字列表。 大部分只能用于部分配置段。 有一些标注了"deprecate"
是因为它们是从老的语法继承下来的,它们通常容易让人混淆或者功能受限,而且目前都有新的替代
关键字使用。有[no]标注的关键字表明用no作为前缀可以作为此关键字的相反设置,比如"no option
contstats". 比如说一个配置默认是启用的,但是在后面的某个具体的实例中需要关闭,这就非常有用了。


keyword                 defaults   frontend   listen    backend
----------------------+----------+----------+---------+---------
acl                         -          X         X         X   
appsession                  -          -         X         X   
backlog                     X          X         X         -
balance                     X          -         X         X   
bind                        -          X         X         -   
block                       -          X         X         X
capture cookie              -          X         X         -
capture request header      -          X         X         -
capture response header     -          X         X         -
clitimeout                  X          X         X         -  (deprecated)
contimeout                  X          -         X         X  (deprecated)
cookie                      X          -         X         X
default_backend             -          X         X         -
disabled                    X          X         X         X
dispatch                    -          -         X         X
enabled                     X          X         X         X
errorfile                   X          X         X         X
errorloc                    X          X         X         X
errorloc302                 X          X         X         X
errorloc303                 X          X         X         X
fullconn                    X          -         X         X
grace                       -          X         X         X
http-check disable-on-404   X          -         X         X
log                         X          X         X         X
maxconn                     X          X         X         -
mode                        X          X         X         X
monitor fail                -          X         X         -
monitor-net                 X          X         X         -
monitor-uri                 X          X         X         -
[no] option abortonclose    X          -         X         X
[no] option allbackups      X          -         X         X
[no] option checkcache      X          -         X         X
[no] option clitcpka        X          X         X         -
[no] option contstats       X          X         X         -
[no] option dontlognull     X          X         X         -
[no] option forceclose      X          -         X         X
option forwardfor           X          X         X         X
[no] option http_proxy      X          X         X         X
option httpchk              X          -         X         X
[no] option httpclose       X          X         X         X
option httplog              X          X         X         X
[no] option logasap         X          X         X         -
[no] option nolinger        X          X         X         X
[no] option persist         X          -         X         X
[no] option redispatch      X          -         X         X
option smtpchk              X          -         X         X
[no] option srvtcpka        X          -         X         X
option ssl-hello-chk        X          -         X         X
option tcpka                X          X         X         X
option tcplog               X          X         X         X
[no] option tcpsplice       X          X         X         X
[no] option transparent     X          X         X         -
redisp                      X          -         X         X  (deprecated)
redispatch                  X          -         X         X  (deprecated)
reqadd                      -          X         X         X
reqallow                    -          X         X         X
reqdel                      -          X         X         X
reqdeny                     -          X         X         X
reqiallow                   -          X         X         X
reqidel                     -          X         X         X
reqideny                    -          X         X         X
reqipass                    -          X         X         X
reqirep                     -          X         X         X
reqisetbe                   -          X         X         X
reqitarpit                  -          X         X         X
reqpass                     -          X         X         X
reqrep                      -          X         X         X
reqsetbe                    -          X         X         X
reqtarpit                   -          X         X         X
retries                     X          -         X         X
rspadd                      -          X         X         X
rspdel                      -          X         X         X
rspdeny                     -          X         X         X
rspidel                     -          X         X         X
rspideny                    -          X         X         X
rspirep                     -          X         X         X
rsprep                      -          X         X         X
server                      -          -         X         X
source                      X          -         X         X
srvtimeout                  X          -         X         X  (deprecated)
stats auth                  X          -         X         X
stats enable                X          -         X         X
stats realm                 X          -         X         X
stats refresh               X          -         X         X
stats scope                 X          -         X         X
stats uri                   X          -         X         X
stats hide-version          X          -         X         X
timeout check               X          -         X         X
timeout client              X          X         X         -
timeout clitimeout          X          X         X         -  (deprecated)
timeout connect             X          -         X         X
timeout contimeout          X          -         X         X  (deprecated)
timeout http-request        X          X         X         -
timeout queue               X          -         X         X
timeout server              X          -         X         X
timeout srvtimeout          X          -         X         X  (deprecated)
timeout tarpit              X          X         X         X
transparent                 X          X         X         -  (deprecated)
use_backend                 -          X         X         -
----------------------+----------+----------+---------+---------
keyword                 defaults   frontend   listen    backend


2.2.1) 关键字索引(按字母顺序排序)
-----------------------------------------------

This section provides a description of each keyword and its usage.
本节描述每个关键字及其用法。


acl <aclname> <criterion> [flags] [operator] <value> ...
  Declare or complete an access list.
  声明或完整一个访问控制列表(ACL).
  可用于配置段:                defaults | frontend | listen | backend
                                  no    |    yes   |   yes  |   yes
  示例:
        acl invalid_src  src          0.0.0.0/7 224.0.0.0/3
        acl invalid_src  src_port     0:1023
        acl local_dst    hdr(host) -i localhost
2.5) HTTP头操作
-----------------------------


在HTTP模式下,可以基于正则表达式来重写、增加或者删除请求或者回应包头,如果一个某特定的
头匹配正则,也可以基于此来拒绝此请求或回应,这样就足以解决大部分简单的基于协议的攻击了,
也可以防止信息从内部网泄露出去。但是这样做却有一个限制: 因为HAproxy的HTTP引擎不支持
keep-alive, 在一个TCP会话中仅仅第一个请求中发送的头信息能被HAProxy看到,所有后续头
仅当做是数据而不会进行分析。而且,HAProxy从来不接触数据内容,它在分析完所有的头后会立即
停止分析。

本节包含了下列在2.2.1中详细描述过的关键字的用法:

  - reqadd     <string>
  - reqallow   <search>
  - reqiallow  <search>
  - reqdel     <search>
  - reqidel    <search>
  - reqdeny    <search>
  - reqideny   <search>
  - reqpass    <search>
  - reqipass   <search>
  - reqrep     <search> <replace>
  - reqirep    <search> <replace>
  - reqtarpit  <search>
  - reqitarpit <search>
  - rspadd     <string>
  - rspdel     <search>
  - rspidel    <search>
  - rspdeny    <search>
  - rspideny   <search>
  - rsprep     <search> <replace>
  - rspirep    <search> <replace>


所有这些关键字都使用相同的约定。<search>参数是一个POSIX扩展的正则表达式(regex),支持
使用括号()进行分组(括号不需要使用\转义)。空格和其他域(field)分隔符必须使用\开头以避免
和域分隔符混淆。其他字符也可以结合\使用来改变其本身的含义:

  \t   tab
  \r   回车(CR)
  \n  换行(LF)
  \    空格(而不是域分隔符)
  \#   输出'#'而不当做是注释
  \\   在正则中使用\
  \\\\ 在文本中使用\(*2 for regex, *2 for haproxy)
  \xXX 就像C语言里一样输出XX的十六进制ASCII码。

The <replace> parameter contains the string to be used to replace the largest
portion of text matching the regex. It can make use of the special characters
above, and can reference a substring which is delimited by parenthesis in the
regex, by writing a backslash ('\') immediately followed by one digit from 0 to
9 indicating the group position (0 designating the entire line). This practice
is very common to users of the "sed" program.

The <string> parameter represents the string which will systematically be added
after the last header line. It can also use special character sequences above.

Notes related to these keywords :
---------------------------------
  - these keywords are not always convenient to allow/deny based on header
    contents. It is strongly recommended to use ACLs with the "block" keyword
    instead, resulting in far more flexible and manageable rules.

  - lines are always considered as a whole. It is not possible to reference
    a header name only or a value only. This is important because of the way
    headers are written (notably the number of spaces after the colon).

  - the first line is always considered as a header, which makes it possible to
    rewrite or filter HTTP requests URIs or response codes, but in turn makes
    it harder to distinguish between headers and request line. The regex prefix
    ^[^\ \t]*[\ \t] matches any HTTP method followed by a space, and the prefix
    ^[^ \t:]*: matches any header name followed by a colon.

  - for performances reasons, the number of characters added to a request or to
    a response is limited at build time to values between 1 and 4 kB. This
    should normally be far more than enough for most usages. If it is too short
    on occasional usages, it is possible to gain some space by removing some
    useless headers before adding new ones.

  - keywords beginning with "reqi" and "rspi" are the same as their couterpart
    without the 'i' letter except that they ignore case when matching patterns.

  - when a request passes through a frontend then a backend, all req* rules
    from the frontend will be evaluated, then all req* rules from the backend
    will be evaluated. The reverse path is applied to responses.

  - req* statements are applied after "block" statements, so that "block" is
    always the first one, but before "use_backend" in order to permit rewriting
    before switching. 


2.6) 日志
------------

[待补充]

2.7) CSV格式

  0. pxname: 代理名
  1. svname: service name (FRONTEND 指前端, BACKEND指后端,其他名字指服务器)
  2. qcur: 当前的请求队列
  3. qmax: 最大请求队列
  4. scur: 当前会话(session)
  5. smax: 最大会话(session)
  6. slim: 会话限制
  7. stot: 总会话
  8. bin: bytes in
  9. bout: bytes out
 10. dreq: 拒绝的请求
 11. dresp: 拒绝的回应(responses)
 12. ereq: 请求错误
 13. econ: 连接错误
 14. eresp: 回应错误
 15. wretr: 重试次数(warnings)
 16. wredis: redispatches (warning)
 17. status: 状态s (UP/DOWN/...)
 18. weight: 服务器权重(server), 总权重(backend)
 19. act: 服务器为主(active)(server), 主服务器个数(backend)
 20. bck: 服务器为备(backup) (server),被服务器个数(backend)
 21. chkfail: 失败的检查次数
 22. chkdown: 从UP->DOWN 切换的次数
 23. lastchg: 最后状态改变时间(seconds)
 24. downtime: 总的宕机时间(秒)
 25. qlimit: 队列限制
 26. pid: 进程id (0 第一个实例, 1 为第二个, ...)
 27. iid: 唯一代理id
 28. sid: service id (unique inside a proxy)
 29. throttle: warm up status
 30. lbtot: 服务器被选择的总次数
 31. tracked: id of proxy/server if tracking is enabled
 32. type (0=frontend, 1=backend, 2=server)

2.8) Unix套接字命令(Unix Socket commands)

 - "show stat [<iid> <type> <sid>]": 以csv格式导出状态信息。通过传入id,type和sid可以
    导出指定项:
     - iid为proxy id, -1导出所有
     - type 选择可导出对象类型: 1 为 frontend, 2 指 backend, 4 指
       server, -1 为所有. 这些指可以是OR(bit运算),如:
          1+2=3   -> frontend+backend.
          1+2+4=7 -> frontend+backend+server.
     - sid 为service id, -1 从选定的proxy导出所有service.

 - "show info": 导出当前haproxy的状态信息。

quick_abort_min

近来在研究大文件 Cache 方面,squid 对大文件处理方面的优化.其中发现二个最重要的参数 range_offset_limit 和 quick_abort*.其实从名字中就能看得出来,二个最主要的分别就是一个是对大文件的 range (断点续传)和 abort (中断)的处理.注意这些都是由客户端请求过来的 request 来驱动产生的。


1. squid 中的 range_offset_limit

其实在前面几次提到过这个参数,也会有很多问题,一定要注意就是这个设置的比你的maximum_object_size要小.不然下载完,又放弃.

另外这个参数也会有个问题,就是当十个线程过来,它会同一个文件请求10次,不过总的来讲,这个参数少设置为好.

2. squid 中的 quick_abort

quick_abort_min (KB)
quick_abort_max (KB)
quick_abort_pct (percent)


控制squid是否继续传输被用户中断的请求.当用户中断请求时,squid把检测 quick_abort 的值.如果剩余部分小于“quick_abort_min”指定的值,squid 把继续完成剩余部分的传输;如果剩余部分大于“quick_abort_max”指定的值,squid 把终止剩余部分的传输;如果已完成“quick_abort_pct”指定的百分比,squid把继续完成剩余部分的传输.

比如我们对大量的大文件的 cache 系统可以这样处理,如果想有请求过来就一直请求完的话,就使用 quick_abort_min -1 来让请求断开后还不断的回源,这的副作用是有可能引起服务器的流量比真实用户的请求还多。
quick_abort_min 16 MB
quick_abort_max 16 MB
quick_abort_pct 80

 

3. squid 中的 collapsed_forwarding

多个回源流量变成一个回源,这个在大文件时,也比较有用,但是如果第一个用户很慢的话,会严重影响速度.如果有开发能力最好修复。

4. read_ahead_gap
这个是让 Squid 来预读多少的意思。

squid-3.2 多核

wget http://www.squid-cache.org/Versions/v3/3.2/squid-3.2.0.18.tar.gz
./configure --prefix=/usr/local/squid --enable-gnuregex --enable-async-io=80 --enable-cache-digests --enable-err-language="zh-cn" --enable-default-err-language="zh-cn" --enable-epoll --disable-internal-dns  --with-filedescriptors=655360 --enable-kill-parent-hack --enable-esi --enable-ssl --with-large-files --enable-build-info='VeryCDN'  --enable-follow-x-forwarded-for
make
make install

 

 

squid.conf
机器为6核.所以 workers 为6
也可以为每个squid 进程.绑定到某个CPU核上
设置方法见cpu_affinity_map参数

http://www.squid-cache.org/Doc/config/cpu_affinity_map/

 

 

acl ControlCentre src 114.80.226.0/24 112.65.227.0/24 127.0.0.0/24
http_port 80 accel vhost vport
max_filedescriptors 65535
 

cpu_affinity_map代替workers,能正常开启多进程,squid工作正常。我们的服务器是双CPU,12核24进程,下面是我们的参数:

1 cpu_affinity_map process_numbers=1,2,3,4,5,6,7,8,9,10,11,12
2 cores=1,3,5,7,9,11,13,15,17,19,21,23
这个参数的意思就是把squid进程和CPU核绑定。

了解squid3.2支持多核设计考虑

1. 当前squid的状态和架构
Squid-3.3使用wokers 支持基本的多核,管理员可以通过配置启动一个squid来派生多个worker进程利用所有可利用的CPU。

一个worker 接收一个新的http request 并且处理每一个请求直到它完成。Worker之间可以共享http_ports 但是它们不会将一个请求事务传到其他worker间。一个worker支持的功能与单squid(不支持SMP的squid)完全相同可但是worker之间或许配置不同,如配置为监听不同的http_ports。参数cpu_affinity_map项允许CPU core绑定到每个worker。

2. Worker之间是如何协调的?
一个指定的协调进程会启动workers 和在它们需要时,协调他们的行为。以下是协调程序的主要职责:
重启工作异常的worker进程。
允许worker进程直接共享监听socket
广播reload和shutdow命令到其他worker
为Cahce Manager统计一些信息。
协调程序并不参与常规的事务处理,也决定不了哪worker来处理新进来的请求。
通常,这个协调程序都是空闲的。

3. Worker进程之间可以共享啥?
可以使用协调器和通用的配置文件,squid的worker进程们可以收到完全相同的配置信息和同步它们之间的特性。默认情况下,squid worker 可以共享的如下:
Squid可执行程序。
通用配置文件
监听的端口
Logs
内存缓存对象(支持大多数的环境)
磁盘缓存对象(仅限于Rock Store)
缓存管理统计等。

限定的配置和worker依赖的宏可以用来限制共享。例如,每个worker可以指定监听某一个http_port。

目前,squid 的worker和同步狭义的资源或服务,包括如下:
1)、内存缓存对象(在一些特定的环境下)
2)、磁盘缓存对象(除了Rock Store)
3)、DNS caches (ipcache 和 fqdncache)
4)、SNMP 统计—目前正有一个活跃的项目在推竞这项工作。
5)、Helper 进程和 守护进程。
6)、SSL session 缓存

缓存索引可以共享而无需另行拷贝。其他的信息共享通常是在RAM中进行,本质上是用拷贝和复制来进行,以避免加锁及其他的性能开销。

4. 为什么用进程?难道线程不是更好么?
以下几个原因决定了使用进程而不是线程:
1)、在当前squid的框架下使用线程会消耗更多的时间,缘于线程不安全。用户现在急需支持多核也无法等待彻底从新开始重现一个squd。
2)、线程提供了更快的上下文切换,但在一个典型的多核的squid部署环境中,squid的每个worker进程都绑定到了一个特定的cpu,上下文切换的开销并不是非常重要。
3)、进程和线程同步和共享机制对足以支持多处理器功能的扩展。
概括起来,我们使用进程代替线程的原因是:它让我们在有限的时间内实现了对多处理器的支持。此时使用线程不太现实。

5. 由谁决定worker 接受request?
所有的worker都共享http_port 监听同一个IP地址和同一个tcp port。操作系统会加锁保护这个监听的socket。并且决定哪个worker接受这个HTTP请求。一旦连接被这个worker接受,则这个连接就交给对应的这个worker

原文地址:http://wiki.squid-cache.org/Features/SmpScale

Records:6412345678