工作,学习,生活,这里将会有一些记录. 备用域名:http://meisw.wdlinux.cn 注册 | 登陆
浏览模式: 标准 | 列表全部文章

id image及boot和recovery文件系统剖析及其制作!

手机文件系统分为不同的部分,可能如下,不同的系统顺序不同,
#cat /proc/mtd
dev:     size    erasesize   name
mtd0: 00040000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00280000 00020000 "boot"
mtd3: 04380000 00020000 "system"
mtd4: 04380000 00020000 "cache"
mtd5: 04ac0000 00020000 "userdata"
我们主要涉及recovery和boot,在我们操作之前最好是备份他俩。
# cat /dev/mtd/mtd1 > /sdcard/mtd1.img
# cat /dev/mtd/mtd2 > /sdcard/mtd2.img

System存放/中挂载的所有文件,userdata中存放data目录中的数据,主要是你安装的apps及其文件。
我们最好把自己喜欢的update.zip发在sdcard的root目录中,以在弄乱boot分区后可以恢复。
有另外一个重要的文件 /system/recovery.img,他是mtd1的完全拷贝。每次关机他都会重新刷写mtd1一次,这代表你对recovery即/dev/mtd/mtd1的修改再重启后都无效。如果要改变mtd1,最好是插入image在 /system/recovery.img中,如果你升级系统没有替换 /system/recovery.img会很惨的。
Boot和recovery不是正常的文件系统,是android自定的格式,有一个2k的头,接着是gzipped的kernel和ramdisk,一个可选的second stage loader(少见)。定义在mkbootimg.h中。
+-----------------+
| boot header      | 1 page
+-----------------+
| kernel           | n pages  
+-----------------+
| ramdisk          | m pages  
+-----------------+
| second stage     | o pages
+-----------------+

n = (kernel_size + page_size - 1) / page_size
m = (ramdisk_size + page_size - 1) / page_size
o = (second_size + page_size - 1) / page_size

0. all entities are page_size aligned in flash
1. kernel and ramdisk are required (size != 0)
2. second is optional (second_size == 0 -> no second)
Ramdisk是一个小型文件系统包含用来初始化系统的核心文件,它包含关键的初始化进程,比如init.rc(可以设定系统范围的属性),下面是ramdisk中的一系列的文件。关于ramdisk的文档如下地址
./init.trout.rc
./default.prop
./proc
./dev
./init.rc
./init
./sys
./init.goldfish.rc
./sbin
./sbin/adbd
./system
./data
Recovery image还有额外的文件,包含recovery二进制程序及其支持文件(在home+power按下重启时运行),文件如下
./res
./res/images
./res/images/progress_bar_empty_left_round.bmp
./res/images/icon_firmware_install.bmp
./res/images/indeterminate3.bmp
./res/images/progress_bar_fill.bmp
./res/images/progress_bar_left_round.bmp
./res/images/icon_error.bmp
./res/images/indeterminate1.bmp
./res/images/progress_bar_empty_right_round.bmp
./res/images/icon_firmware_error.bmp
./res/images/progress_bar_right_round.bmp
./res/images/indeterminate4.bmp
./res/images/indeterminate5.bmp
./res/images/indeterminate6.bmp
./res/images/progress_bar_empty.bmp
./res/images/indeterminate2.bmp
./res/images/icon_unpacking.bmp
./res/images/icon_installing.bmp
./sbin/recovery
解压编辑和压缩镜像
镜像中有一串00接着1F 8B,0之前为kernel,1F 8B之后为ramdisk,我们可以编辑ramdisk,不过得先un gizp它,再un cpio它。我们可以用如下命令

 

gunzip -c ../your-ramdisk-file | cpio -i
之后在工作目录中会有所有的ramdisk的文件,我们可以编辑它。

为了重新创建ramdisk,我们的gzip和re cpio它,所有工作目录中的文件都将被加入ramdisk中。
find . | cpio -o -H newc | gzip > ../newramdisk.cpio.gz

最后我们可以组合内核,使用mkbootimg程序,命令如下
mkbootimg --cmdline 'no_console_suspend=1 console=null' --kernel your-kernel-file --ramdisk newramdisk.cpio.gz -o mynewimage.img

把image刷写到手机上:
如果有新的recovery image,直接插入手机中,重启。如果是boot,得使用adb如下。
adb push ./mynewimage.img /sdcard

得到root权限。使用如下两条命令:

# cat /dev/zero > /dev/mtd/mtd2
    write: No space left on device [this is ok, you can ignore]
# flash_image boot /sdcard/mynewimage.img
Reboot
如果有错误,使用update.zip重新升级。

Alternative Method
Download split_bootimg.zip . This Zip file contains one Perl file, split_bootimg.pl, which reads the boot.img header (according to the bootimg.h of the Android source code) to extract the kernel and ramdisk. The script also outputs the kernel command line and board name (if specified).
(Note: Do not use a boot.img image extracted directly from /dev/mtd/mtd2. This image may become corrupted during the read process.)
The following example uses the boot.img from the full TC4-RC28 update:
% ./split_bootimg.pl boot.img
Page size: 2048 (0x00000800)
Kernel size: 1388548 (0x00153004)
Ramdisk size: 141518 (0x000228ce)
Second size: 0 (0x00000000)
Board name:
Command line: no_console_suspend=1
Writing boot.img-kernel ... complete.
Writing boot.img-ramdisk.gz ... complete.
Extract the ramdisk.
% mkdir ramdisk
% cd ramdisk
% gzip -dc ../boot.img-ramdisk.gz | cpio -i
% cd ..
Make any changes necessary (e.g., set ro.secure=0 in default.prop).
Recreate the cpio archive using the mkbootfs binary produced from building the Android source code (The cpio utility in OS X does not recognize the newc format, therefore mkbootfs is the best option for OS X users).
% mkbootfs ./ramdisk | gzip > ramdisk-new.gz
Recreate the image file using the mkbootimg binary produced from building the Android source code.
% mkbootimg --cmdline 'no_console_suspend=1 console=null' --kernel boot.img-kernel --ramdisk ramdisk-new.gz -o boot-new.img
For Nexus One : Add --base 0x20000000 to mkbootimg command-line.

(Note: the console=null command line option was introduced in the TC4-RC30 boot images to remove the root shell (TODO: add link))
Flashing your new image back onto the phone
You will probably only ever be flashing boot images directly to the phone, given the fact that /system/recovery.img automatically flashes the recovery device for you (as noted above). If you have created a new recovery image, just stick it in /system/recovery.img and reboot. If you are flashing a boot image, stick it on your phone via adb (a tool included in the Android SDK):
adb push ./mynewimage.img /sdcard
Then, open a shell to your phone via 'adb shell', get root, and do the following two commands to flash your new boot image:
# cat /dev/zero > /dev/mtd/mtd2
    write: No space left on device [this is ok, you can ignore]
# flash_image boot /sdcard/mynewimage.img
Reboot.
If your phone starts all the way up, congratulations. If not, you did something wrong and you'll need to boot into recovery mode and apply your update.zip file (reboot while holding down home+power, when you get the recovery screen press alt+L and then alt+S).
Something fun to do with your new found power
If you place a file titled initlogo.rle in the root directory of your boot image, the phone will display this image upon boot (after the "G1" image and before the Android animation). In order to create this file, you need to create a 320x480 image in Photoshop or Gimp and save it as a "raw image" file. You then need to compress that image with the program to565. More details on thathere.
This is not the same thing as applying an update.zip
You will see other places on the forums that describe how to create customized update.zip files, as well as update.zip files that people are sharing. For example, there is a recent update.zip which is a modified version of rc30 (with the anti-root aspects disabled). The update.zip files include new boot images, recovery images, and typically replacements for the entire system/ directory as well as other updates. If you are creating a custom boot or recovery image, it is typically a good idea to start with the image distributed with the most recent update you have applied (flashing an image from an older release could have unintended consequences).

Android启动脚本init.rc

摘自《Android系统原理及开发要点详解》

Android启动脚本init.rc

在 Android中使用启动脚本init.rc,可以在系统的初始化过程中进行一些简单的初始化操作。这个脚本被直接安装到目标系统的根文件系统中,被 init可执行程序解析。 init.rc是在init启动后被执行的启动脚本,其语法主要包含了以下内容:

Commands:命令
Actions: 动作
Triggers:触发条件
Services:服务
Options: 选项
Propertise:属性

(1) Commands是一些基本的操作,例如:
    mkdir /sdcard 0000 system system
    mkdir /system
    mkdir /data 0771 system system
    mkdir /cache 0770 system cache
    mkdir /config 0500 root root
    mkdir /sqlite_stmt_journals 01777 root root
    mount tmpfs tmpfs /sqlite_stmt_journals size=4m
这些命令在init可执行程序中被解析,然后调用相关的函数来实现。

(2) Actions(动作)表示一系列的命令,通常在Triggers(触发条件)中调用,动作和触发条件的形式为:
   on <trigger>
      <command>
      <command>
      <command>

动作的使用示例如下:
    on init
    export PATH /sbin:/system/sbin:/system/bin:/system/xbin
    mkdir /system
init表示一个触发条件,这个触发事件发生后,进行设置环境变量和建立目录的操作称为一个“动作”


(3) Services(服务)通常表示启动一个可执行程序,Options(选项)是服务的附加内容,用于配合服务使用。

service vold /system/bin/vold
    socket vold stream 0660 root mount

service bootsound /system/bin/playmp3
    user media
    group audio
    oneshot

vold和bootsound分别是两个服务的名称,/system/bin/vold和/system /bin/playmp3分别是他们所对应的可执行程序。
socket、user、group、oneshot就是配合服务使用的选项。其中oneshot选项表示该服务只启动一次,而如果没有oneshot选项,
这个可执行程序会一直存在--如果可执行程序被杀死,则会重新启动。

(4) Properties(属性)是系统中使用的一些值,可以进行设置和读取。

    setprop ro.FOREGROUND_APP_MEM 1536
    setprop ro.VISIBLE_APP_MEM 2048
    on property:ro.kernel.qemu=1
    start adbd
setprop 用于设置属性,on property可以用于判断属性,这里的属性在整个Android系统运行中都是一致的。

init脚本的关键字可以参考init进程的system/core/init/keyword.h文件。
init.rc的使用方法,可以参考说明文件system/core/init/readme.txt

如果想要修改启动过程只需要修改init.c(system/core/init)或者init.rc里的内容即可.


附录:system/core/init/readme.txt


Android Init Language
---------------------

The Android Init Language consists of four broad classes of statements,
which are Actions, Commands, Services, and Options.

All of these are line-oriented, consisting of tokens separated by
whitespace. The c-style backslash escapes may be used to insert
whitespace into a token. Double quotes may also be used to prevent
whitespace from breaking text into multiple tokens. The backslash,
when it is the last character on a line, may be used for line-folding.

Lines which start with a # (leading whitespace allowed) are comments.

Actions and Services implicitly declare a new section. All commands
or options belong to the section most recently declared. Commands
or options before the first section are ignored.

Actions and Services have unique names. If a second Action or Service
is declared with the same name as an existing one, it is ignored as
an error. (??? should we override instead)


Actions
-------
Actions are named sequences of commands. Actions have a trigger which
is used to determine when the action should occur. When an event
occurs which matches an action's trigger, that action is added to
the tail of a to-be-executed queue (unless it is already on the
queue).

Each action in the queue is dequeued in sequence and each command in
that action is executed in sequence. Init handles other activities
(device creation/destruction, property setting, process restarting)
"between" the execution of the commands in activities.

Actions take the form of:

on <trigger>
   <command>
   <command>
   <command>


Services
--------
Services are programs which init launches and (optionally) restarts
when they exit. Services take the form of:

service <name> <pathname> [ <argument> ]*
   <option>
   <option>
   ...


Options
-------
Options are modifiers to services. They affect how and when init
runs the service.

critical
   This is a device-critical service. If it exits more than four times in
   four minutes, the device will reboot into recovery mode.

disabled
   This service will not automatically start with its class.
   It must be explicitly started by name.

setenv <name> <value>
   Set the environment variable <name> to <value> in the launched process.

socket <name> <type> <perm> [ <user> [ <group> ] ]
   Create a unix domain socket named /dev/socket/<name> and pass
   its fd to the launched process. <type> must be "dgram" or "stream".
   User and group default to 0.

user <username>
   Change to username before exec'ing this service.
   Currently defaults to root. (??? probably should default to nobody)
   Currently, if your process requires linux capabilities then you cannot use
   this command. You must instead request the capabilities in-process while
   still root, and then drop to your desired uid.

group <groupname> [ <groupname> ]*
   Change to groupname before exec'ing this service. Additional
   groupnames beyond the (required) first one are used to set the
   supplemental groups of the process (via setgroups()).
   Currently defaults to root. (??? probably should default to nobody)

oneshot
   Do not restart the service when it exits.

class <name>
   Specify a class name for the service. All services in a
   named class may be started or stopped together. A service
   is in the class "default" if one is not specified via the
   class option.

onrestart
    Execute a Command (see below) when service restarts.

Triggers
--------
   Triggers are strings which can be used to match certain kinds
   of events and used to cause an action to occur.

boot
   This is the first trigger that will occur when init starts
   (after /init.conf is loaded)

<name>=<value>
   Triggers of this form occur when the property <name> is set
   to the specific value <value>.

device-added-<path>
device-removed-<path>
   Triggers of these forms occur when a device node is added
   or removed.

service-exited-<name>
   Triggers of this form occur when the specified service exits.


Commands
--------

exec <path> [ <argument> ]*
   Fork and execute a program (<path>). This will block until
   the program completes execution. It is best to avoid exec
   as unlike the builtin commands, it runs the risk of getting
   init "stuck". (??? maybe there should be a timeout?)

export <name> <value>
   Set the environment variable <name> equal to <value> in the
   global environment (which will be inherited by all processes
   started after this command is executed)

ifup <interface>
   Bring the network interface <interface> online.

import <filename>
   Parse an init config file, extending the current configuration.

hostname <name>
   Set the host name.

chdir <directory>
   Change working directory.

chmod <octal-mode> <path>
   Change file access permissions.

chown <owner> <group> <path>
   Change file owner and group.

chroot <directory>
Change process root directory.

class_start <serviceclass>
   Start all services of the specified class if they are
   not already running.

class_stop <serviceclass>
   Stop all services of the specified class if they are
   currently running.

domainname <name>
   Set the domain name.

insmod <path>
   Install the module at <path>

mkdir <path> [mode] [owner] [group]
   Create a directory at <path>, optionally with the given mode, owner, and
   group. If not provided, the directory is created with permissions 755 and
   owned by the root user and root group.

mount <type> <device> <dir> [ <mountoption> ]*
   Attempt to mount the named device at the directory <dir>
   <device> may be of the form mtd@name to specify a mtd block
   device by name.
   <mountoption>s include "ro", "rw", "remount", "noatime", ...

setkey
   TBD

setprop <name> <value>
   Set system property <name> to <value>.

setrlimit <resource> <cur> <max>
   Set the rlimit for a resource.

start <service>
   Start a service running if it is not already running.

stop <service>
   Stop a service from running if it is currently running.

symlink <target> <path>
   Create a symbolic link at <path> with the value <target>

sysclktz <mins_west_of_gmt>
   Set the system clock base (0 if system clock ticks in GMT)

trigger <event>
   Trigger an event. Used to queue an action from another
   action.

write <path> <string> [ <string> ]*
   Open the file at <path> and write one or more strings
   to it with write(2)


Properties
----------
Init updates some system properties to provide some insight into
what it's doing:

init.action
   Equal to the name of the action currently being executed or "" if none

init.command
   Equal to the command being executed or "" if none.

init.svc.<name>
   State of a named service ("stopped", "running", "restarting")


Example init.conf
-----------------

# not complete -- just providing some examples of usage
#
on boot
   export PATH /sbin:/system/sbin:/system/bin
   export LD_LIBRARY_PATH /system/lib

   mkdir /dev
   mkdir /proc
   mkdir /sys

   mount tmpfs tmpfs /dev
   mkdir /dev/pts
   mkdir /dev/socket
   mount devpts devpts /dev/pts
   mount proc proc /proc
   mount sysfs sysfs /sys

   write /proc/cpu/alignment 4

   ifup lo

   hostname localhost
   domainname localhost

   mount yaffs2 mtd@system /system
   mount yaffs2 mtd@userdata /data

   import /system/etc/init.conf

   class_start default

service adbd /sbin/adbd
   user adb
   group adb

service usbd /system/bin/usbd -r
   user usbd
   group usbd
   socket usbd 666

service zygote /system/bin/app_process -Xzygote /system/bin --zygote
   socket zygote 666

service runtime /system/bin/runtime
   user system
   group system

on device-added-/dev/compass
   start akmd

on device-removed-/dev/compass
   stop akmd

service akmd /sbin/akmd
   disabled
   user akmd
   group akmd

Debugging notes
---------------
By default, programs executed by init will drop stdout and stderr into
/dev/null. To help with debugging, you can execute your program via the
Andoird program logwrapper. This will redirect stdout/stderr into the
Android logging system (accessed via logcat).

For example
service akmd /system/bin/logwrapper /sbin/akmd

busybox及bash在android中的安装及init.rc修改

ramdisk的制作
2.6内核开始,initrd.img采用cpio压缩,ramdisk.img也一样,使用gunzip解压缩,然后再使用 cpio解包。

1)将ramdisk.img复制其他目录,名称改为ramdisk.img.gz,解压
#gunzip ramdisk.img.gz


//新建一个文件夹ramdisk,进入
#cpio -i -F ../ramdisk.img
这时,可到ramdisk中看看去~

2)修改init.rc,在PATH中加上busybox 路径
//busybox安装在 /data/busybox
## Global environment setup
##
env {
       #其中, /data/busybox 为busybox安装路径,bash也是放在其中

       PATH /data/busybox:/sbin:/system/sbin:/system/bin
       LD_LIBRARY_PATH /system/lib
       ANDROID_BOOTLOGO 1
       ANDROID_ROOT /system
       ANDROID_ASSETS / system/app
       ANDROID_DATA /data
       EXTERNAL_STORAGE /sdcard
       DRM_CONTENT /data/drm/content
       HOME /                                          #这个也是新添加环境变量
}

3)重新打包成镜像,并使用新镜像启动emulator
#cpio -i -t -F ../ramdisk.img > list
#cpio -o -H newc -O rd_busybox.img < list

//使用 -ramdisk 参数,指定所使用的镜像文件
#emulator -noskin -ramdisk rd_ramdisk.img


2. 安装busybox及bash
下载地址:http://www.billrocks.org/android_libs/bin/
注:也可自行交叉编译,不过需静态编译。

#adb shell mkdir /data/busybox
#adb push busybox /data/busyobx
#adb push bash /data/busybox


//adb shell,进入android
#cd /data/busyobx
#chmod +x busybox bash
#./busybox --install                                 //将程序安装在当前目录下

//重启emulator,进入bash
bash-3.2# export
declare -x ANDROID_ASSETS="/system/app"
declare -x ANDROID_BOOTLOGO="1"
declare -x ANDROID_DATA="/data"
declare -x ANDROID_ROOT="/system"
declare -x DRM_CONTENT="/data/drm/content"
declare -x EXTERNAL_STORAGE="/sdcard"
declare -x HOME="/"
declare -x LD_LIBRARY_PATH="/system/lib"
declare -x OLDPWD
declare -x PATH="/data/busybox:/sbin:/system/sbin:/system/bin"
declare -x PWD="/data/bin/tst"
declare -x SHLVL="1"

注:在1中修改 init.rc 增加的环境变量及路径已经生效。

android 修改ramdisk.img和init.rc && android启动后设置/data权限为770 .

有时候希望在启动后做点事情,比如在sdcard上建立目录,修改/data的访问权限等,这些都可以放在init.rc中去完成,那么如何来完成呢,最近由于工作的需要,在这方面进行了一些研究,特写于此,供大家参考。

      通过下面的例子来讲述,实现一个功能:在android系统启动后,修改/data的访问权限为770.

共4步:1、解压ramdisk.img 2、用c语言完成要实现的功能 3、修改init.rc,启动c语言可执行程序  4、制作ramdisk.img

      1、解压ramdisk.img:

  解压、修改Android的ramdisk.img的手动方法:

  将ramdisk.img复制一份到任何其他目录下,将其名称改为ramdisk.img.gz,并使用命令

  gunzip ramdisk.img.gz

  然后新建一个文件夹,叫ramdisk吧,进入,输入命令
  cpio -i -F ../ramdisk.img
  这下,你就能看见并操作ramdisk里面的内容了。

  此步可参考:http://blog.csdn.net/liushaogeng/archive/2010/10/14/5941259.aspx
 

   2、完成一个C语言程序chpermission.c,来实现修改权限的功能。代码如下:

     #include <stdlib.h>

      int main()

     {

        system("chmod 700 /data");

         return 0;

 

     }

    然后编译此程序,不过需要编译为android下的可执行程序,可参考我的博客:http://blog.csdn.net/liushaogeng/archive/2010/09/01/5855925.aspx ,采用博客中介绍的两种方法均可以。
    编译完后将可执行文件push到模拟器的/data目录中去:
     #adb push chpermission /data

     3、 修改ramdisk目录下init.rc文件,让其启动 chpermission程序, 增加以下代码:

     #add by me on 2010-10-18   
     service
chpermission   /data/ chpermission
          oneshot

     用于启动 chpermission,完成相应的功能--修改data权限。

     init.rc的语法可参考博客: http://blog.csdn.net/liushaogeng/archive/2010/10/18/5949244.aspx

     4、制作ramdisk.img,

  init.rc修改之后,可以使用下列命令重新打包成镜像
  cpio -i -t -F ../ramdisk.img > list
  cpio -o -H newc -O lk.img < list

  当前目录下生成的lk.img就是我们的新镜像了。

  可参考博客: http://blog.csdn.net/liushaogeng/archive/2010/10/14/5941259.aspx
 

   将ramdisk.img拷贝到sdk指向的目录。

    全部ok,重启自己的模拟器,查看data权限是否为770.

android中通过代码实现文件权限修改(chmod)

在Unix和Linux的各种操作系统下,每个文件(文件夹也被看作是文件)都按读、写、运行设定权限。

读、写、运行三项权限可以用数字表示,就是r=4,w=2,x=1。所以,rw-r--r--用数字表示成644。
反过来说777就是rwxrwxrwx,意思是该登录用户(可以用命令id查看)、他所在的组和其他人都有最高权限。
Android中可用通过adb shell 方法修改文件的权限,有时候我们需要在代码中实现改功能,

 

    try {
              String command = "chmod 777 " + destFile.getAbsolutePath();
              Log.i("zyl", "command = " + command);
              Runtime runtime = Runtime.getRuntime(); 

              Process proc = runtime.exec(command);
             } catch (IOException e) {
              Log.i("zyl","chmod fail!!!!");
              e.printStackTrace();
             }

Android下修改hosts文件

由于某些原因,可能需要指定域名对应的IP地址。Android是基于Linux的系统,与Linux类似,通过hosts文件来设置。

在Android下,/etc是link到/system/etc的,我们需要修改/system/etc/hosts来实现。但是这个文件是只读,不能通过shell直接修改。可以通过连接到PC上使用adb来修改。步骤如下:

1、获得root权限:adb root

2、设置/system为可读写:adb remount

3、将hosts文件复制到PC:adb pull /system/etc/hosts <PC机上文件名>

4、修改PC机上文件

5、将PC机上文件复制到手机:adb push <PC机上文件名> /system/etc/hosts

如果要查看是否修改成功,可以在PC上执行adb shell,运行cat /system/etc/hosts;或者在手机上运行cat /system/etc/hosts。

在Android 1.6系统中,hosts文件格式有一点与PC机Linux不同:不能在一行中一个IP对应多个域名,比如:

127.0.0.1      host1.example.com host2.example.com host3.example.com

在大多PC机Linux系统是合法的,但不能在Android 1.6上起作用,需要拆成每个域名一行才能使用:

127.0.0.1      host1.example.com

127.0.0.1      host2.example.com

127.0.0.1      host3.example.com

 

refresh_pattern 的一些理解和建议

Squid 中 refresh_pattern 的作用

 

用于确定一个页面进入cache后,它在cache中停留的时间。refresh_pattern 规则仅仅应用到没有明确过时期限的响应。原始服务器能使 用 Expires 头部,或者 Cache-Control:max-age 指令来指定过时期限 ,只要没有在设置 override-expire。

语法:

refresh_pattern [-i] regexp min percent max [options]

 min 参数是分钟数量。它是过时响应的最低时间限制。如果某个响应驻留在 cache 里的时间没有超过这个最低限制,那么它不会过期。类似的,max 参数是存活响应的最高时间限制。如果某个响应驻留在 cache 里的时间高于这个最高限制,那么它必须被刷新。

在最低和最高时间限制之间的响应,会面对 squid 的最后修改系数 LM-factor 算法 LM-factor=(response age)/(resource age)。对这样的响应,squid 计算响应的年龄和最后修改系数,然后将它作为百分比值进行比较。响应年龄简单的就是从原始服务器产生,或最后一次验证响应后,经历的时间数量。源年龄在 Last-Modified 和 Date头 部之间是不同的。LM-factor 是响应年龄与源年龄的比率。这个基本不用详细了解,这是不是一个精确控制过期的参数,如果要精确控制过期,就不要使用这个。

 

Refresh_pattern 有14个参数

 

我讲讲常用的几个参数的意思

override-expire
该选项导致 squid 在检查 Expires 之前,先检查 min 值。这样,一个非零的 min 时间让 squid 返回一个未确认的 cache 命中,即使该响应准备过期。

override-lastmod
改选项导致 squid 在检查 LM-factor 百分比之前先检查min ,它生效在expire 之后

reload-into-ims
该选项让 squid 在确认请求里,以 no-cache 指令传送一个请求。换句话说,squid 在转发请求之前,对该请求增加一个 If-Modified- Since 头部。注意这点仅仅在目标有 Last-Modified 时间戳时才能工作。外面进来的请求保留 no-cache 指令,以便它到达原始服务器。
一般情况可以使用 reload-into-ims。它其实是强行控制对象的超时时间,这违反了http协议的精神,但是在带宽较窄的场合,可以提高明显系统相应时间。
举例:

refresh_pattern -i \.css$ 1440 50% 129600 reload-into-ims
refresh_pattern -i \.xml$ 1440 50% 129600 reload-into-ims
refresh_pattern -i \.html$ 1440 90% 129600 reload-into-ims-
refresh_pattern -i \.shtml$ 1440 90% 129600 reload-into-ims
refresh_pattern -i \.hml$ 1440 90% 129600 reload-into-ims
refresh_pattern -i \.jpg$ 1440 90% 129600 reload-into-ims
refresh_pattern -i \.png$ 1440 90% 129600 reload-into-ims
refresh_pattern -i \.gif$ 1440 90% 129600 ignore-reload
refresh_pattern -i \.bmp$ 1440 90% 129600 reload-into-ims
refresh_pattern -i \.js$ 1440 90% 129600 reload-into-ims

 ignore-reload
该选项导致 squid 忽略请求里的任何 no-cache 指令。
所以。如果希望内容一进入 cache 就不删除,直到被主动 purge 掉为止,可以加上 ignore-reload 选项,这个我们常用在mp3,wma,wmv,gif之类。
Examples:

refresh_pattern -i \.mp3$ 1440 50% 2880 ignore-reload
refresh_pattern -i \.wmv$ 1440 50% 2880 ignore-reload
refresh_pattern -i \.rm$ 1440 50% 2880 ignore-reload
refresh_pattern -i \.swf$ 1440 50% 2880 ignore-reload
refresh_pattern -i \.mpeg$ 1440 50% 2880 ignore-reload
refresh_pattern -i \.wma$ 1440 50% 2880 ignore-reload

ignore-no-cache
该选项导致 Squid 强制忽略从源站而来的“Pragma: no-cache”和“Cache-control: no-cache”

ignore-private
该选项导致 Squid 强制忽略从源站而来的“Cache-control: private”

ignore-auth:强制将一个请求认为是源站发送的带有“Cache-control: public”

ignore-auth
该选项导致 Squid 强制将一个请求认为是源站发送的带有“Cache-control: public”

 

Refresh_pattern  percent 的计算

resource age =对象进入cache的时间-对象的last_modified
response age =当前时间-对象进入cache的时间
LM-factor=(response age)/(resource age)

 

举个例子,这里只考虑percent, 不考虑min 和max

例如:refresh_pattern 20%

假设源服务器上www.aaa.com/index.htm —–lastmodified 是 2007-04-10 02:00:00
squid上 proxy.aaa.com/index.htm index.htm进入cache的时间 2007-04-10 03:00:00

1)如果当前时间 2007-04-10 03:00:00
resource age =3点-2点=60分钟
response age =0分钟
index.htm还可以在cache停留的时间(resource age)*20%=12分钟
也就是说,index.htm进入cache后,可以停留12分钟,才被重新确认。

2)如果当前时间 2007-04-10 03:05:00
resource age =3点-2点=60分钟
response age =5分钟
index.htm还可以在cache停留的时间(resource age)*20%=12分钟-5=7
LM-factor=5/60=8.3%<20%

一直到2007-04-10 03:12:00 LM-factor=12/60=20% 之后,cache中的页面index.htm终于stale。
如果这时没有 index.htm 的请求,index.htm 会一直在缓存中,如果有 index.htm 请求,squid 收到该请求后,由于已经过期, squid 会向源服务器发一个 index.htm 是否有改变的请求,源服务器收到后,如果 index.htm 没有更新,squid 就不用更新缓存,直接把 缓存的内容放回给客户端,同时,重置对象进入 cache 的时间为与源服务器确认的时间,比如 2007-04-10 03:13:00,如果正好在这个后重新确认了页面。重置后,resource age 变长,相应在 cache 中存活的时间也变长。

如果有改变则把最新的 index.htm 返回给 squid,squid 收到会更新缓存,然后把新的 index.htm 返回给客户端,同时根据新页面 中的Last_Modified 和取页面的时间,重新计算 resource age,进一步计算出存活时间。

实际上,一个页面进入 cache 后,他的存活时间就确定了,即 (resource age) * 百分比,一直到被重新确认。

Linux rsync目录文件同步

rysnc是一个数据镜像及备份工具,具有可使本地和远程两台主机的文件,目录之间,快速同步镜像,远程数据备份等功能。在同步过程中,rsync是根据自己独特的算法,只同步有变化的文件,甚至在一个文件里只同步有变化的部分,所以可以实现快速的同步数据的功能。

第一种:依赖ssh服务同步目录

rsync -aSvH /home/N22/test1/* /home/N22/test2/

参数解释:

a 等同于 -rlptgoD 归档模式,就是保持文件所有属性、权限不变
S 有效的处理零散文件
v verbose模式
H 保持hard links

上面这个表示 将本机的/home/N22/test1/目录下所有的文件,同步到/home/N22/test2/目录下。这种方式我只能同步新增和更新,不能同步删除,不知道原因,修改为

rsync -aSvH –delete /home/N22/test1/* /home/N22/test2/

时,虽然添加了delete仍然不能同步删除的文件。

和其它服务器同步时,需要账户和密码:rsync -aSvH /home/* root@192.168.1.1:/home/

第二种:服务器和客户端

经过测试,这种服务器和客户端的配置能完美解决删除文件和目录的同步功能。

服务器端配置:

1、rysnc一般是通过xinetd进行启动的。

修改/etc/xinetd.d/rsync,只修改一个地方
# default: off
# description: The rsync server is a good addition to am ftp server, as it \
#       allows crc checksumming etc.
service rsync
{
disable = yes  修改为disable = no
socket_type     = stream
wait            = no
user            = root
server          = /usr/bin/rsync
server_args     = –daemon
log_on_failure  += USERID

}

2、接下来编辑配置文件

[root@mailsvr ~] # vi /etc/rsyncd.conf
uid = nobody

gid = nobody

use chroot = no

max connections = 4

pid file = /var/run/rsyncd.pid

lock file = /var/run/rsyncd.lock

log file = /var/log/rsyncd.log

[web]

path = /opt/web

ignore errors

read only = true

list = false

hosts allow = 192.168.0.5  # 允许的IP地址

hosts deny = 0.0.0.0/32    # 禁止的IP 地址

auth users = backup        # 认证用户名,此例是 backup

secrets file = /etc/backup.pass #认证用户的密码文件

3、编辑服务器的密码文件 /etc/backup.pass

vi /etc/backup.pass

backup:123456

4、然后设置权限chmod 400 /etc/backup.pass

客户端配置

1. 编辑rsync连接时的密码文件 /etc/rsync_client.pass

vi /etc/rsync_client.pass

123456                    # 只需要配置连接时使用的密码即可,必须与A服务器上定义的密码相同.

2. chmod 600 /etc/rsync_client.pass

3. 使用 rsync 命令连接服务器,实现文件同步

rsync -vzrtopg –progress –delete –password-file=/etc/rsync_client.passbackup@192.168.1.2::web /opt/test

4. 使用 –execlude= 排除不需要同步的文件后缀名

rsync -vzrtopg –progress –delete –password-file=/etc/rsync_client.pass –exclude=”*.tmp”backup@192.168.1.2::web /home/test

5. 使用 –execlude-from= 排除不需要同步的目录

rsync -vzrtopg –progress –delete –password-file=/etc/rsync_client.pass –exclude-from=/opt/pcdir backup@192.168.1.2::web /home/test

6、定时同步

#vi rsyncd.sh

rsync -aSvH  –delete –password-file=/home/N22/rsync_client.pass backup@192.168.1.252::web /home/N22/test2/

#crontab -e
01 04 * * * /etc/rsyncd.sh
每天凌晨4点01分执行

http://hi.baidu.com/colorsight/blog/item/f0ce3816d874ec104b90a75f.html