Nginx Multi-Workload Config - Website & Reverse Proxy

时间:2019-04-23 15:08:45

标签: nginx nginx-reverse-proxy nginx-config

TLDR: I'm trying to configure Nginx to host a website AND act as a reverse proxy at the same time. The website is on 'www.example.com:443' and the reverse proxy is to a separate OpenVPN server on 'vpn.example.com:443'. I think I know the answer is "Nginx cannot be configured to listen on a single port for two different types of traffic" (web traffic and reverse proxy traffic), but thought I'd ask those more knowledgeable.

Now, on to the more long winded version. I do have a working setup that looks like this:

               +-----------------------+
               |                       |
               |         Nginx         |
               |    (Rev Proxy Only)   |
               |                       |
               |     192.168.0.100     |
               |  vpn.example.com:443  |
               |  www.example.com:443  |
               |                       |
               +-----------------------+
                 /                   \
                /                     \
+-----------------------+    +-----------------------+
|                       |    |                       |
|        OpenVPN        |    |       LAMP Stack      |
|         (VPN)         |    |        (Website)      |
|                       |    |                       |
|     192.168.0.101     |    |     192.168.0.102     |
|  vpn.example.com:443  |    |  www.example.com:443  |
|                       |    |                       |
+-----------------------+    +-----------------------+

Here are a couple of notes. The Nginx (v1.14.2) server is CentOS v7.6. It is configured with "--with-stream" and "--with-stream_ssl_preread_module". It also uses all defaults in the /etc/nginx/nginx.conf, except it includes a stream{} block with proxy_pass. The conf files look like this:

/etc/nginx/nginx.conf

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}

include /etc/nginx/proxy.conf;

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

/etc/nginx/proxy.conf

stream {

    map $ssl_preread_server_name $name {
        www.example.com lamp;
        vpn.example openvpn;
        default https_default_backend;
    }

    server {
        listen 192.168.0.100:443;
        proxy_pass $name;
        ssl_preread on;
    }

    upstream lamp {
        server 192.168.0.102:443;
    }

    upstream openvpn {
        server 192.168.0.101:443;
    }

    upstream https_default_backend {
        server 192.168.0.101:443;
    }

    log_format basic '[$time_local] $remote_addr - $protocol '
                     'STATUS: $status BYTES_SENT: $bytes_sent '
                     'BYTES_RECEIVED: $bytes_received '
                     'SENT_TO: $upstream_addr MSECS: $msec '
                     'SESSION_TIME: $session_time '
                     'UPSTREAM_BYTES_SENT: $upstream_bytes_sent '
                     'UPSTREAM_BYTES_RECEIVED: $upstream_bytes_received '
                     'UPSTREAM_CONNECTION_TIME: $upstream_connect_time"';

    access_log /var/log/nginx/stream.mfgs.dev_access.log basic;
    error_log  /var/log/nginx/stream.mfgs.dev_error.log;

}

Again, this is my current config, and everything here works. Website traffic is streamed to the web server and VPN traffic is streamed to the VPN server. All of this works.

My goal is to change the LAMP stack (Linux, Apache, MySQL, PHP) to a LEMP stack (Linux, Nginx, MySQL, PHP), and combine the Nginx reverse proxy and website onto the same server. Any traffic specifically directed to 'www.example.com:443' would go to the Nginx website and everything else would be passed over to the VPN server, like this:

+-----------------------+
|                       |
|         Nginx         |
| (Website & Rev Proxy) |
|                       |
|     192.168.0.100     |
|  vpn.example.com:443  |
|  www.example.com:443  |
|                       |
+-----------------------+
            |
            |
+-----------------------+
|                       |
|        OpenVPN        |
|         (VPN)         |
|                       |
|     192.168.0.101     |
|  vpn.example.com:443  |
|                       |
+-----------------------+

I've tried installing Nginx on a separate server (192.168.0.105). I got the website up and running on this server. However, when I include the proxy.conf, it tests correctly using 'nginx -t', but the Nginx service fails to restart on this server. The failure message is:

nginx: [emerg] bind() to 192.168.0.105:443 failed (98: Address already in use)

I believe that's because both the http and stream blocks are listening on the same IP & port. Is there some way to configure Nginx to act as a web server for a specific FQDN:port and proxy (via streaming) everything else to another server?

Thank you in advance.

1 个答案:

答案 0 :(得分:0)

我认为您正在尝试将www.example.com与新的nginx主机托管在同一主机中:192.168.0.105

如果是这样,则在启动address already in use服务时会遇到nginx错误。

这是由于nginx http 服务都配置为侦听端口: 443

http {
    server {
        listen 192.168.0.105:443;
        server_name www.example.com;
    }
}

stream {
    server {
        listen 192.168.0.105:443;
        proxy_pass $name;
        ssl_preread on;
    }
}

http服务不同,该服务允许基于命名的服务器使用相同的端口。 Nginx将httpstream视为不同的服务。

我有2种解决方案可以解决同一节点中的主机streamhttp

解决方案1 ​​:让www.example.com127.0.0.1:443

由于大多数节点默认情况下应具有回送接口。我相信您想直接公开stream:443服务而不是www.example.com:443。让www.example.com倾听127.0.0.1:443并不是解决address already in use问题的好主意。

http {
    server {
        listen 127.0.0.1:443;
        server_name www.example.com;
    }
}

解决方案2 :让www.example.com监听Unix套接字

类似于解决方案1,但为www.example.com使用unix套接字以避免额外的TCP开销,从而获得一点性能提升。

http {
    server {
        listen unix:/var/run/nginx.sock;
        server_name www.example.com;
    }
}

对于这两种解决方案,请记住使用upstream或unix套接字重新配置stream部分中的127.0.0.1

我希望可以找到更多的选择,但这是到目前为止我能想到的。