Monthly Archives: janvier 2014

Nginx and wordpress permalinks

In order to have modified wordpress permalinks working on NGinx, you have to add a « try_files » line in the location block of the config file:

        location / {
                index index.php;
                # a trick for Wordpress
                try_files $uri $uri/ /index.php?$args;
                }

Speeding up wordpress

Here are some tips to speed up your wordpress site.

First, add theses lines to your wp-config.php file:

/* Tuning */
define('WP_CACHE', true);
define('WP_POST_REVISIONS', 2);
define('WP_POST_REVISIONS',false);
define('AUTOSAVE_INTERVAL', 300);
define('EMPTY_TRASH_DAYS', 3 );
define('WP_MEMORY_LIMIT', '128M');

The memory limit has to be adusted to your server configuration.

haproxy + nginx reverse proxy

How to access/protect some web services or applications ?

Assumptions used in this article:

– all connections are done through SSL
– the loadbalancer HAproxy (LB1, LB2, LBn) will be setup with keepalived for the failover
– the reverse proxy NGinx (NG1, NG2, NGn) will be a SSL termination and will forward SSL traffic to the web server
– the webserver NGinx provides SSL connection

Big Picture:

haproxynginx01

 

1) Web server setup

Use of NGinx with PHP and SSL:

apt-get install nginx php-fpm openssl

Edit file /etc/nginx/nginx.conf

# nginx.conf

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;

events {
        worker_connections 768;
}

http {

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        server_tokens off;

        index index.php index.html

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

Edit virtual host file, in this example /etc/nginx/sites-available/naze.mine.nu.80 for http site based on the domain name (and do not forget to link it in sites-enabled directory):

server {
    listen      80;
    server_name naze.mine.nu vradis.naze.mine.nu;
    root        /var/www/naze.mine.nu/80/ONLINE;

    include /etc/nginx/mime.types;
        default_type application/octet-stream;

        # Get primary client info (not in ssl)
        set_real_ip_from 192.168.45.0/24;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;

    access_log /var/log/nginx/naze.mine.nu.80.access.log;
    error_log /var/log/nginx/naze.mine.nu.80.error.log;

        error_page 404 /404.html;
        location = /404.html {
                root   /var/www/naze.mine.nu/error;
                }

        location / {
                index index.php;
                # a trick for Wordpress permalinks
                try_files $uri $uri/ /index.php?$args;
                }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        location ~ .php$ {
                try_files $uri =404;
                fastcgi_split_path_info ^(.+.php)(/.+)$;
                # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini

                # With php5-cgi alone:
                #fastcgi_pass 127.0.0.1:9000;
                # With php5-fpm:
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                fastcgi_index index.php;
                include fastcgi_params;
            }

    }

For SSL website we have to create certificates, two files are needed in the config: domain.key and domain.crt. Change « domain » with whatever you want:

openssl genrsa -des3 -out server.key 4096
openssl req -new -key server.key -out domain.csr
openssl rsa -in server.key -out domain.key
openssl x509 -req -days 3650 -in domain.csr -signkey domain.key -out domain.crt

All certificate files will be stored in a dedicated ssl directory.

Edit virtual host file, in this example /etc/nginx/sites-available/naze.mine.nu.443 for https site based on the domain name:

server {
    listen      443;
    server_name naze.mine.nu;
    root        /var/www/naze.mine.nu/443/ONLINE;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ssl on;
        ssl_certificate /etc/nginx/ssl/naze.mine.nu.crt;
        ssl_certificate_key /etc/nginx/ssl/naze.mine.nu.key;

        # Get primary client info (not in ssl)
        set_real_ip_from 192.168.45.0/24;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;

    access_log /var/log/nginx/naze.mine.nu.443.access.log;
    error_log /var/log/nginx/naze.mine.nu.443.error.log;

        error_page 404 /404.html;
        location = /404.html {
                root   /var/www/naze.mine.nu/error;
                }

    location / {
        index index.php;
        # a trick for Wordpress permalinks
        try_files $uri $uri/ /index.php?$args;
        }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        location ~ .php$ {
                try_files $uri =404;
                fastcgi_split_path_info ^(.+.php)(/.+)$;
                # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini

                # With php5-cgi alone:
                #fastcgi_pass 127.0.0.1:9000;
                # With php5-fpm:
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                fastcgi_index index.php;
                include fastcgi_params;
            }

    }

We also want to cath all access to the webserver when not done through defined domain.

For example instead of using http://naze.mine.nu but http://82.226.254.187. Then we have to define default sections, edit file /etc/nginx/site-enabled/default.80:

server {

    listen       80  default_server;
    server_name  _;
    root /var/www/default/80;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        # Get primary client info (not in ssl)
        set_real_ip_from 192.168.45.0/24;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;

    access_log /var/log/nginx/default.80.access.log;
    error_log /var/log/nginx/default.80.error.log;

    # Back to site when error
    error_page 404 /index.php;

        location / {
                index index.php;
                # a trick for Wordpress permalinks
                try_files $uri $uri/ /index.php?$args;
                }

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        location ~ .php$ {
                try_files $uri =404;
                fastcgi_split_path_info ^(.+.php)(/.+)$;
                # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini

                # With php5-cgi alone:
                #fastcgi_pass 127.0.0.1:9000;
                # With php5-fpm:
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                fastcgi_index index.php;
                include fastcgi_params;
                }

    }

You can also define a default website for SSL connections.

And do not forget to make symlinks in directory site-available to have these files loaded.

RESULT:

We have a webserver serving naze.mine.nu domain in http & https. We do catch other requests with a default website. Configuration is done to keep client IP in log for http connections.

 

2) Reverse proxy setup

The reverse proxy is located on a different server than the webserver.

Use of NGinx:

apt-get install nginx

I will not install openssl since I will reuse the same certificates than the webserver. I copy all certificates from one server to another.

Edit file nginx.conf:

user www-data;
worker_processes 2;
pid /var/run/nginx.pid;

events {
    worker_connections 512;
    accept_mutex        on;   # serially accept() connections and pass to workers, efficient if workers gt 1
    # multi_accept on;
}

http {


    # Timeouts, do not keep connections open longer then necessary to reduce
    # resource usage and deny Slowloris type attacks.
    client_body_timeout      3s; # maximum time between packets the client can pause when sending nginx any data
    client_header_timeout    3s; # maximum time the client has to send the entire header to nginx
    keepalive_timeout       75s; # timeout which a single keep-alive client connection will stay open
    send_timeout             9s; # maximum time between packets nginx is allowed to pause when sending the client data

    # General Options
    charset                   utf-8; # adds the line "Content-Type" into response-header, same as "source_charset"
    default_type              application/octet-stream;
    ignore_invalid_headers    on;
    include                   /etc/nginx/mime.types;
    keepalive_requests        50;  # number of requests per connection, does not affect SPDY
    keepalive_disable         none; # allow all browsers to use keepalive connections
    max_ranges                0;   # disabled to stop range header DoS attacks as resumed downloads are denied
    msie_padding              off;
    open_file_cache           max=1000 inactive=2h;
    open_file_cache_errors    on;
    open_file_cache_min_uses  1;
    open_file_cache_valid     1h;
    output_buffers            1 512;
    postpone_output           1440;   # postpone sends to match our machine's MSS
    read_ahead                512K;   # kernel read head set to the output_buffers
    recursive_error_pages     on;
    reset_timedout_connection on;  # reset timed out connections freeing ram
    sendfile                  on;  # on for decent direct disk I/O
    server_tokens             off; # version number in error pages
    server_name_in_redirect   off; # if off, nginx will use the requested Host header
    source_charset            utf-8; # same value as "charset"
    tcp_nodelay               on; # Nagle buffering algorithm, used for keepalive only
    tcp_nopush                off;

    # Log Format
    log_format  main  '$remote_addr $host $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $ssl_cipher $request_time';


    proxy_cache_path  /var/lib/nginx/proxy/my-cache  levels=1:2   keys_zone=my-cache:10m;


    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Edit site file reverse.naze.mine.nu.80 to reverse for url http://naze.mine.nu

server {
    listen  naze.mine.nu:80;
        server_name  naze.mine.nu;

        access_log  /var/log/nginx/reverse.naze.mine.nu.80.access.log;
        error_log  /var/log/nginx/reverse.naze.mine.nu.80.error.log error;

        set_real_ip_from 192.168.45.0/24;
        set_real_ip_from 192.168.45.85;
        real_ip_header X-Forwarded-For;
        # real_ip_recursive on;

        if ($request_method !~ ^(GET|HEAD|POST)$ ) {
                return 403;
                }

        error_page 301 /301.html;
                location = /301.html {
                root   /var/www/reverse/naze.mine.nu/80;
                }
        error_page 302 /302.html;
                location = /302.html {
                root   /var/www/reverse/naze.mine.nu/80;
                }
        error_page 404 /404.html;
                location = /404.html {
                root   /var/www/reverse/naze.mine.nu/80;
                }
        error_page 500 /500.html;
                location = /500.html {
                root   /var/www/reverse/naze.mine.nu/80;
                }
        error_page 502 /502.html;
                location = /502.html {
                root   /var/www/reverse/naze.mine.nu/80;
                }
        error_page 503 /503.html;
                location = /503.html {
                root   /var/www/reverse/naze.mine.nu/80;
                }

    # Where to go
        location / {
        proxy_pass  http://192.168.45.10:80;
        proxy_cache my-cache;
        proxy_cache_valid 12h;
        expires 12h;
        proxy_cache_use_stale error timeout invalid_header updating;
                }

    location ~*^.+(swf|jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)$ {
        proxy_pass http://192.168.0.10:80;
        proxy_cache my-cache;
        proxy_cache_valid 2d;
        expires max;
        }

        }

Edit site file for https domain, here to reverse for url: https://naze.mine.nu

server {
    listen  naze.mine.nu:443;
        server_name  naze.mine.nu;

        access_log  /var/log/nginx/reverse.naze.mine.nu.443.access.log;
        error_log  /var/log/nginx/reverse.naze.mine.nu.443.error.log error;

        proxy_cache off;

        ssl on;
        ssl_certificate /etc/nginx/ssl/naze.mine.nu.crt;
        ssl_certificate_key /etc/nginx/ssl/naze.mine.nu.key;

        # set_real_ip_from 192.168.45.0/24;
        set_real_ip_from 192.168.45.65;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;

        if ($request_method !~ ^(GET|HEAD|POST)$ ) {
                return 403;
                }

        error_page 301 /301.html;
                location = /301.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }
        error_page 302 /302.html;
                location = /302.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }
        error_page 400 /400.html;
                location = /400.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }
        error_page 401 /401.html;
                location = /401.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }
        error_page 500 /500.html;
                location = /500.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 502 /502.html;
                location = /502.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 503 /503.html;
                location = /503.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        location / {
                # proxy_pass              http://naze.mine.nu;
        # proxy_pass    https://192.168.45.105:443;
        proxy_pass    https://192.168.45.10:443;
                }
        }

Edit site file for default website to reverse url not corresponding to any other defined site:

server {
    listen  naze.mine.nu:443;
        server_name  naze.mine.nu;

        access_log  /var/log/nginx/reverse.naze.mine.nu.443.access.log;
        error_log  /var/log/nginx/reverse.naze.mine.nu.443.error.log error;

        proxy_cache off;

        ssl on;
        ssl_certificate /etc/nginx/ssl/naze.mine.nu.crt;
        ssl_certificate_key /etc/nginx/ssl/naze.mine.nu.key;

        # set_real_ip_from 192.168.45.0/24;
        set_real_ip_from 192.168.45.65;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;

        if ($request_method !~ ^(GET|HEAD|POST)$ ) {
                return 403;
                }

        error_page 301 /301.html;
                location = /301.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 302 /302.html;
                location = /302.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 400 /400.html;
                location = /400.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 401 /401.html;
                location = /401.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 403 /403.html;
                location = /403.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 404 /404.html;
                location = /404.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 500 /500.html;
                location = /500.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 502 /502.html;
                location = /502.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        error_page 503 /503.html;
                location = /503.html {
                root   /var/www/reverse/naze.mine.nu/80;
                allow all;
                }

        location / {
                # proxy_pass              http://naze.mine.nu;
        # proxy_pass    https://192.168.45.105:443;
        proxy_pass    https://192.168.45.10:443;
                }
        }

 

RESULT:

We have setup the reverse proxy that will send requests to the web server.

 

3) Loadbalancer HAProxy setup

 

The loadbalancer will be installed on another server than the reverse proxy.

Install HAproxy:

apt-get install haproxy

Edit file /etc/haproxy/haproxy.cfg:

global
log /dev/log local0
log 127.0.0.1 local1 notice
chroot /var/lib/haproxy
maxconn 4096
user haproxy
group haproxy
daemon

defaults
log global
mode http
# option httplog
option dontlognull
contimeout 5000
clitimeout 50000
srvtimeout 50000
# option httpclose
# option forwardfor
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http

listen lb1.naze.mine.nu 0.0.0.0:80
mode http
stats enable
option httplog
# stats auth user:pass
balance roundrobin
cookie JSESSIONID prefix
option httpclose
option forwardfor

# put ip of resverse proxies
server reverse_proxy_1 192.168.45.85:80 cookie reverse_proxy_1 check
server reverse_proxy_2 192.168.45.95:80 cookie reverse_proxy_2 check

frontend https_frontend
bind *:443
mode tcp
default_backend https_reverse

backend https_reverse
mode tcp
balance roundrobin
stick-table type ip size 200k expire 30m
stick on src

# put ip of resverse proxies
server reverse_proxy_1_ssl 192.168.45.85:443
server reverse_proxy_2_ssl 192.168.45.95:443

We said we would do some failover for the loadbalancer. Let’s set up keepalived:

apt-get install keepalived

Edit config file /etc/keepalived/keepalived.conf:

vrrp_script chk_haproxy { # Requires keepalived-1.1.13
script "killall -0 haproxy" # cheaper than pidof
interval 2 # check every 2 seconds
weight 2 # add 2 points of prio if OK
}

vrrp_instance VI_1 {
interface eth0
state MASTER
virtual_router_id 51
priority 101 # 101 on master, 100 on backup
virtual_ipaddress {
192.168.45.99
}
track_script {
chk_haproxy
}
}

This means all web services will be accessible through http(s) using the IP address 192.168.45.99. The DNS has to be configured accordingly, that means naze.mine.nu must resolv to 192.168.45.99.

You can bind as many domains as you want on this IP address, then configure reverse proxies accordingly.

 

DO NOT FORGET:

You have to replicate the haproxy and keepalived configuration on the second loadbalancer.

You have to replicate the nginx configuration on the second reverse proxy.