【nginx】 关于nginx的优化调整和本地代理的讨论

首先

我之前想要一个快速的HTTP服务器,所以决定重新复制约10年前进行的nginx调优。
另外,我还会介绍当时作为前端服务器性能改善的本地代理。
值得注意的是,调优没有标准答案。这只是我偶然成功的故事。

nginx的速度很快的说法。

这次的组织

Untitled Diagram (1).jpg

默认设置(HTTP)

首先,以默认状态进行比较。
即使是默认设置,nginx已经变得更快了。我记得以前默认设置下差距没有这么大。

# ab -c 10 -n 10000 -k http://apache1/index.html |grep "Requests per second"
Requests per second:    15801.16 [#/sec] (mean)
# ab -c 10 -n 10000 -k http://nginx1/index.html |grep "Requests per second"
Requests per second:    39050.75 [#/sec] (mean)

默認設置(https)

https的样子大概是这样的。差距大约是两倍左右。

# ab -c 10 -n 10000 -k https://apache1/index.html |grep "Requests per second"
Requests per second:    11478.42 [#/sec] (mean)
# ab -c 10 -n 10000 -k https://nginx1/index.html |grep "Requests per second"
Requests per second:    21561.81 [#/sec] (mean)

对nginx进行优化调整

将nginx1的nginx.conf替换为↓。

user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log crit ;
pid /run/nginx.pid;
worker_rlimit_nofile 100000;

events {
    worker_connections 32000;
    multi_accept        on;
    use                 epoll;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  off;

    sendfile           on;
    keepalive_timeout  30;
    access_log        off;
    tcp_nopush         on;
    tcp_nodelay        on;
    reset_timedout_connection on;
    client_body_timeout 10;
    send_timeout 2;
    keepalive_requests 100000;
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
    types_hash_max_size 2048;
    gzip on;
    gzip_disable msie6;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 32 16k;
    gzip_min_length 250;
    gzip_types image/jpeg image/bmp image/svg+xml text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript image/x-icon;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
        root         /usr/share/nginx/html;
        include /etc/nginx/default.d/*.conf;

        location / {
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }

# Settings for a TLS enabled server.

    server {
        listen       443 ssl http2 default_server;
        listen       [::]:443 ssl http2 default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        ssl_certificate "/etc/pki/server.crt";
        ssl_certificate_key "/etc/pki/server.key";
        ssl_session_cache shared:SSL:1m;
        ssl_session_timeout  10m;
        ssl_ciphers PROFILE=SYSTEM;
        ssl_prefer_server_ciphers on;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }

}

服务器指令没有从默认值进行更改,所以可以根据需要任意设定。
worker_processes设为1是有意为之的,如果只是为了调整性能,也可以设为auto。
worker_connection可以更少,实际上很少使用(甚至不使用)。

不要忘记将以下配置添加到nginx中并重新启动。

ulimit -n 100000
sysctl -w net.ipv4.ip_local_port_range='1024 65535'
echo "100000" > /proc/sys/fs/file-max

调音后

为了进行重新比较,也将默认设置下的值列出来。
最下面的是调整后的结果。
与nginx的默认设置相比,提升了50%左右。与apache的默认设置相比,则提升了4倍。

# ab -c 10 -n 10000 -k http://apache1/index.html |grep "Requests per second"
Requests per second:    15801.16 [#/sec] (mean)
# ab -c 10 -n 10000 -k http://nginx1/index.html |grep "Requests per second"
Requests per second:    39050.75 [#/sec] (mean)
# ab -c 10 -n 10000 -k http://nginx1/index.html |grep "Requests per second"
Requests per second:    60312.90 [#/sec] (mean)

调整后(https)。

对比一下https也是值得的。
虽然从nginx默认值增长了2倍以上,但这太过份了。
由于存在一些波动,至少应该认识到它从nginx默认值增长了大约50%以上,跟http类似。

# ab -c 10 -n 10000 -k https://apache1/index.html |grep "Requests per second"
Requests per second:    11478.42 [#/sec] (mean)
# ab -c 10 -n 10000 -k https://nginx1/index.html |grep "Requests per second"
Requests per second:    21561.81 [#/sec] (mean)
# ab -c 10 -n 10000 -k https://nginx1/index.html |grep "Requests per second"
Requests per second:    50646.50 [#/sec] (mean)

大型文件

比较原计划只到这里,但是我觉得有点在意,所以我试着将index.html文件压缩到大约40kb。这个链接也是https。令人意外的是,尽管存在一些波动,但文件大小并没有显著影响放大倍率。至少从文件大小方面来看,这似乎不是一个显著影响性能的调优措施。

# ab -c 10 -n 10000 -k https://apache1/index.html |grep "Requests per second"
Requests per second:    6686.85 [#/sec] (mean)
# ab -c 10 -n 10000 -k https://nginx1/index.html |grep "Requests per second"
Requests per second:    14857.72 [#/sec] (mean)
# ab -c 10 -n 10000 -k https://nginx1/index.html |grep "Requests per second"
Requests per second:    18544.55 [#/sec] (mean)

调音的话就说到这里吧。

本地代理

从这里开始是关于将更改后的nginx.conf应用于前端服务器时的故事。
将刚才更改的nginx.conf文件按以下方式进行本地代理化。

user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log crit ;
pid /run/nginx.pid;
worker_rlimit_nofile 100000;

events {
    worker_connections 32000;
    multi_accept        on;
    use                 epoll;
}

http {
    log_format  main  ' -  [] "" '
                      '  "" '
                      '"" ""';
    access_log  off;

    sendfile           on;
    keepalive_timeout  30;
    access_log        off;
    tcp_nopush         on;
    tcp_nodelay        on;
    reset_timedout_connection on;
    client_body_timeout 10;
    send_timeout 2;
    keepalive_requests 100000;
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
    types_hash_max_size 2048;
    gzip on;
    gzip_disable msie6;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 32 16k;
    gzip_min_length 250;
    gzip_types image/jpeg image/bmp image/svg+xml text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript image/x-icon;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    server {
        listen  8080 default_server;
        server_name  _;

        location / {
            proxy_pass http://backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
        }
     }

    server {
        listen  8443 ssl http2 default_server;
        ssl on;
        ssl_certificate /etc/pki/server.crt;
        ssl_certificate_key /etc/pki/server.key;
        server_name  _;
        ssl_session_cache shared:SSL:1m;
        ssl_session_timeout  10m;
        ssl_ciphers PROFILE=SYSTEM;
        ssl_prefer_server_ciphers on;

        location / {
            proxy_pass http://backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
        }
     }

     upstream backend {
          server 127.0.0.1:80;
          keepalive 10240;
     }
}

包括了与内核相关的部分,确保正好将它们列入工作中↓。

nginx本地代理
– defaultTab: 节点
描述: 使用调优后的nginx配置进行本地代理。
executionEnabled: true
ID: 7f9f2e50-52ca-4b39-bffc-ba542c1b5766
日志级别: 信息
名称: nginx本地代理
可编辑节点筛选: false
节点筛选:
dispatch:
排除优先: true
保持执行: false
排序顺序: 升序
节点为空时成功: false
线程数: ‘1’
筛选: ‘name: apache1′
默认选择的节点: true
插件:
ExecutionLifecycle: null
已启用调度: true
执行顺序:
commands:
– 描述: 修改ulimit
exec: sudo ulimit -n 100000
– 描述: 修改net.ipv4.ip_local_port_range
exec: sudo sysctl -w net.ipv4.ip_local_port_range=’1024 65535’
– 描述: 修改/proc/sys/fs/file-max
exec: sudo sysctl fs.file-max=100000
– 描述: 清除yum缓存(由于某种原因yum无法工作)
exec: sudo yum clean packages
– 描述: 安装nginx
exec: sudo yum install -y nginx
– 描述: 修改nginx.conf
script: |-
sudo tee /etc/nginx/nginx.conf </dev/null
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log crit ;
pid /run/nginx.pid;
worker_rlimit_nofile 100000;events {
worker_connections 32000;
multi_accept on;
use epoll;
}

http {
log_format main ‘ – [] “” ‘
‘ “” ‘
‘”” “”‘;
access_log off;

sendfile on;
keepalive_timeout 30;
access_log off;
tcp_nopush on;
tcp_nodelay on;
reset_timedout_connection on;
client_body_timeout 10;
send_timeout 2;
keepalive_requests 100000;
open_file_cache max=200000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
types_hash_max_size 2048;
gzip on;
gzip_disable msie6;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 32 16k;
gzip_min_length 250;
gzip_types image/jpeg image/bmp image/svg+xml text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript image/x-icon;

include /etc/nginx/mime.types;
default_type application/octet-stream;

server {
listen 8080 default_server;
server_name _;

location / {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Connection “”;
}
}

server {
listen 8443 ssl http2 default_server;
ssl on;
ssl_certificate /etc/pki/server.crt;
ssl_certificate_key /etc/pki/server.key;
server_name _;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 10m;
ssl_ciphers PROFILE=SYSTEM;
ssl_prefer_server_ciphers on;

location / {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Connection “”;
}
}

upstream backend {
server 127.0.0.1:80;
keepalive 10240;
}
}

EOF
– 描述: 启动nginx
exec: sudo service nginx start
保持执行: false
策略: 节点优先
UUID: 7f9f2e50-52ca-4b39-bffc-ba542c1b5766

所以,当时将其应用到前端服务器上后,速度提高了30%(负载减轻)。
但是,在当前环境下,实际应用到apache1的EC2上并没有提速。
从行业角度来看,apache方面应该也做了相应的调优,所以可能代理也有兼容性问题吧。
我尽力进行了尝试和摸索,但是毕竟对当时的情况记忆不深。我又一次感到沮丧。
(CentOS 8的apache据说使用的是eventMPM,这也可能产生了影响)

结尾

这次作为重现本地代理的挑战,结果非常失败,但最终达到了目标,所以还算好。我想要建立一些Web服务器,然后将其轻松整合到一个容易使用的系统中,希望能够舒适地生活。然而,也许现在正是试试“ansible with rundeck”这种并不容易的机会。我想要轻松一些。另外,最近我感觉太过注重输出,导致幽默感不足。需要尽快采取措施。

bannerAds