How do I use a private Docker registry in LKE with Proxy Protocol?
Hi,
I have changed my LKE config to work with Proxy Protocol following this guide. For external traffic this is working fine now, but I'm running into some problems trying to get our in-cluster private Docker registry to work.
Pushing images to the registry from an external connection is working fine, but creating pods fails as K8s is not able to pull images from the registry.
The docker registry is in namespace default. My services are in different namespaces.
I've tried various different configs (and set the --docker-server in the registry-credentials secret accordingly):
image: registry.mydomain.com/foo-bar:latest results in a 503 Bad Gateway
image: docker-registry.default.svc.cluster.local/foo-bar:latest
errors with ImagePullBackOff and
dial tcp: lookup docker-registry.default.svc.cluster.local: no such host
Using the Pod IP (which I rather not use directly) seems to get me somewhere but fails because TLS is offloaded in the NGINX Ingress controller.
image: 10.2.0.15:5000/foo-bar:latest
http: server gave HTTP response to HTTPS client
Any advice on how to get this set up properly is much appreciated.
Edit: I managed to work around this by setting up a dedicated NodeBalancer for our Docker registry without the Proxy Protocol. This is working fine now but feels a bit unnecessary.
3 Replies
rgerke
Linode Staff
Hey there -
This might help you out. We have a new annotation in for the CCM for "hostname-only-ingress" that you can add which should work:
https://github.com/linode/linode-cloud-controller-manager/blob/master/README.md
The full annotation would look like this:
service.beta.kubernetes.io/linode-loadbalancer-hostname-only-ingress: true
What this will do is prevent kube-proxy from circuiting the requests directly to the backend pods. When using TLS and it is terminating on the NodeBalancer, the requests will need to go to the NodeBalancer - and adding that annotation should get this working for you.
Hey there I need the annotation you mentioned, however I set up a Linode LKE cluster through rancher. The loadbalancer is automatically generated and every time I edit the yaml to add the annotation it does not save. Can I still add the annotation?
Edit: Nevermind, I see you can add an annotation by clicking edit on the nodebalancer through the UI and adding the annotation there. Directly editing the YAML does not work.
I made the mistake of reading your documentation and expected it to be complete, cause it is not missing details such as seen in this post.
The git hub repo has all the info in it that is needed but you do not reference it within your own documentation.
You have information spread all over the place, which can be ok if you tell people and do not make them search through a forum to find it. Very frustrating.