我将AWS EKS用作我的k8s控制平面,并部署了一个3节点自动缩放组作为我的工作节点(K8s节点)。该自动缩放组位于我的VPC中。而且我确保安全组的开放程度至少足以允许对等节点和ELB进行通信。
我正在尝试使用nginx-ingress路由来自外部k8s集群的流量。我使用helms通过values.yaml部署我的nginx入口。
我的values.yaml看起来像这样:
serviceAccount:
create: true
name: nginx-ingress-sa
rbac:
create: true
controller:
kind: "Deployment"
service:
type: "LoadBalancer"
# targetPorts:
# http: 80
# https: http
loadBalancerSourceRanges:
- 1.2.3.4/32
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "https"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-1:123456789:certificate/my-cert
service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: sg-12345678
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/enable-access-log: "true"
config:
log-format-escape-json: "true"
log-format-upstream: '{"real_ip" : "$the_real_ip", "remote_user": "$remote_user", "time_iso8601": "$time_iso8601", "request": "$request", "request_method" : "$request_method", "status": "$status", "upstream_addr": $upstream_addr", "upstream_status": "$upstream_status"}'
extraArgs:
v: 3 # NGINX log level
我的入口Yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress-1
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/enable-access-log: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: "s1.testk8.dev.mydomain.net"
http:
paths:
- path: /
backend:
serviceName: s1-service
servicePort: 443
- host: "s2.testk8.dev.mydomain.net"
http:
paths:
- path: /
backend:
serviceName: s2-service
servicePort: 443
tls:
- hosts:
- "s1.testk8.dev.mydomain.net"
- "s2.testk8.dev.mydomain.net"
secretName: "testk8.dev.mydomain.net"
请注意,此机密是域* .mydomain.net上的自签名TLS证书。
此设置现在的行为是,如果输入
https://s1.testk8.dev.mydomain.net在Chrome中只是挂起。它说正在等待左下角的s1.testk8.dev.mydomain.net。
如果我使用:
curl -vk https://s1.testk8.dev.mydomain.net
它返回:
* Trying x.x.x.x...
* TCP_NODELAY set
* Connected to s1.testk8.dev.mydomain.net (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=*.mydomain.net
* start date: Apr 25 00:00:00 2018 GMT
* expire date: May 25 12:00:00 2019 GMT
* issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* SSL certificate verify ok.
> GET / HTTP/1.1
> Host: s1.testk8.dev.steelcentral.net
> User-Agent: curl/7.54.0
> Accept: */*
>
它似乎也在等待服务器响应。
我还尝试调整values.yaml以及更改时
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http" # instead of https as above
然后我点击了https://s1.testk8.dev.mydomain.net URL,至少可以看到来自入口控制器窗格的HTTP 400消息(向HTTPS端口发送的纯HTTP请求)。
如果取消注释values.yaml中的这些行:
# targetPorts:
# http: 80
# https: http
我能够到达后端pod(由状态集控制,此处未列出。),我可以看到具有新条目的后端pod的访问日志。
不确定我的用例是否奇怪,因为我看到很多人在AWS上使用nginx-ingress,他们在ELB终止了TLS。但是我需要让后端Pod终止TLS。
我也尝试了ssl-passthrough标志,但没有帮助。当后端协议为https时,我的请求甚至似乎都没有到达入口控制器,因此谈论ssl-passthrough可能仍然毫无意义。
如果您一直阅读这里的全部内容,请多谢!!
答案 0 :(得分:1)
据我所知,即使使用当前的nginx-ingress大师,也无法使用自签名证书。模板https://github.com/kubernetes/ingress-nginx/blob/master/rootfs/etc/nginx/template/nginx.tmpl
缺少任何必需的指令,例如:
location / {
proxy_pass https://backend.server.ip/;
proxy_ssl_trusted_certificate /etc/nginx/sslcerts/backend.server.pem;
proxy_ssl_verify off;
... other proxy settings
}
因此请尝试使用让我们来加密证书。
答案 1 :(得分:0)
我的猜测是您的后端服务正在使用HTTPS,并且中间流量是通过HTTP发送的。您的values.yaml中的这一行似乎很奇怪:
targetPorts:
http: 80
https: http
您可以尝试这样的事情吗?
targetPorts:
http: 80
https: 443