Kubernetes error with "kubernetes_get_host.py" in monasca-agent

[nseyvet]

Running the script as is and with monasca pike creates the following error:

  raise exceptions.KubernetesAPIConnectionError(exception_message)
monasca_agent.common.exceptions.KubernetesAPIConnectionError: Could not get agent pod from Kubernetes API to get host IP with error - HTTPSConnectionPool(host='10.0.0.1', port=443): Max retries exceeded with url: //api/v1/namespaces/default/pods/monasca-agent-deployment-1746399571-4l84t (Caused by SSLError(CertificateError("hostname '10.0.0.1' doesn't match either of 'kubernetes.default.svc.cluster.local', 'kubernetes.default.svc', 'kubernetes.default', 'kubernetes', 'localhost', '192.168.99.101', '10.0.0.1'",),))

The kubernetes service in 1.7.5 is defined as:

$ kubectl describe service kubernetes
Name:              kubernetes
Namespace:         default
Labels:            component=apiserver
                   provider=kubernetes
Annotations:       <none>
Selector:          <none>
Type:              ClusterIP
IP:                10.0.0.1
Port:              https  443/TCP
TargetPort:        8443/TCP
Endpoints:         10.0.2.15:8443
Session Affinity:  ClientIP
Events:            <none>

This could be tied to kubernetes-client/python#36 ?

The python script "kubernetes_get_host.py" used in monasca-agent can be replaced with the DownwardAPI (see https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/) and the variable injected via en environment variable:

   env:
        - name: AGENT_HOSTNAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName

I would propose to remove the python script and associated code in Dockerfile, and start.sh, etc and instead document how to inject the nodename.

If that is acceptable, I can prepare a fix.

[timothyb89]

I think kubernetes_get_host.py predates nodeName being available in the downward API, but it would be good to switch. Maybe @mhoppal can confirm?

[timothyb89]

That said, you're right, the cause of the error is probably due to outdated ipaddress and urllib3 packages (see also). Removing kubernetes_get_host.py will fix this issue and let the agent start, but the plugins that use the kubernetes API should still fail.

If you're getting this from the monasca/agent containers that's pretty surprising... no idea how we haven't hit that in the agent plugins.

[timothyb89]

Output from the latest agent-collector image shows compatible versions:

[tim:~]↥ 3s % docker run --rm=true -it monasca/agent-collector:master-20171121-122311 pip list | grep -E '(urllib3|ipaddress)'
ipaddress (1.0.18)
urllib3 (1.22)

[nseyvet]

The plugin is fixed IFF the ipaddress package is added to dependencies. At the moment ipaddress is NOT included. I raised an issue on monasca story board for this. https://review.openstack.org/#/c/519651/

…

Sent from my iPhone

On 21 Nov 2017, at 18:58, Tim Buckley ***@***.***> wrote: Output from the latest agent-collector image shows compatible versions: [tim:~]↥ 3s % docker run --rm=true -it monasca/agent-collector:master-20171121-122311 pip list | grep -E '(urllib3|ipaddress)' ipaddress (1.0.18) urllib3 (1.22) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[witekest]

The upstream bugfix has been merged. Are any actions still required here?

[nseyvet]

Kubernetes_get_host.py is still included in the monasca agent base image. So the problem remains and using the downward API would be a safer option.

…

On Wed, 17 Jan 2018 at 10:48, Witek Bedyk ***@***.***> wrote: The upstream bugfix has been merged. Are any actions still required here? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#327 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AdweXApYWaG0zkA8jeLB1yVekHo43CYGks5tLcHWgaJpZM4Qc-9D> .