Kubernetes Developer Tools - Retrieving Application Logs

Use the Kubernetes API to pull back pod logs

Kubernetes API - Logs

For cloud admins, it's fairly straightforward to run "kubectl get pods" then "kubectl logs <pod name>" to pull back logs for a given pod but for regular developers, needing to provide access and kubectl training is not necessarily desired.  Alternatively, in AKS at least, a log analytics workspace can be used but this is costly and clunky.  In my previous post, I outlined how to associate a pod with a service account to call the Kubernetes API.  We'll be updating that account to grant permissions to access pod logs.  This assumes all steps in said post have been followed.

Configuring the service account to access pod logs

Granting the service account permissions to pod logs is fairly straightforward.  Simply update and rerun the Role yaml file as follows, note, the only change is adding "pods/log" in the rules.resources array:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: dev-tools-reader
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods", "pods/log"]
  verbs: ["get", "watch", "list"]

Validating access to the API

Similar to validating basic connections, the following commands can be run from inside the pod to validate access to the pod logs.  You'll need to know and replace in the last line the pod name as well as the container name which can be retrieved with "kubectl describe pods".

SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount
TOKEN=$(cat ${SERVICEACCOUNT}/token)
CACERT=${SERVICEACCOUNT}/ca.crt
curl --cacert ${CACERT} --header "Authorization: Bearer ${TOKEN}" -X GET https://kubernetes.default.svc/api/v1/namespaces/default/pods/<pod name>/log?container=<container name>

This should return the entirety of logs for the given pod.

Retrieving the logs in code

Kubernetes has a fairly extensive list of client libraries, we'll be using java for this example.  Our code snippet for retrieving the logs from the API looks something like:

        String podName = "podname";
        String containerName = "containerName";
        // Only retrieves logs logged in the last X seconds, in this case, it's 600 seconds or 10 minutes
        Integer sinceSeconds = 600;
        // Only displays the last X lines of logs, in this case, 1000
        Integer tailLines = 1000;

        try {
            ApiClient client = Config.defaultClient();
            Configuration.setDefaultApiClient(client);
            CoreV1Api api = new CoreV1Api();

            Call call = api.readNamespacedPodLogCall(podName, "default", containerName, false, null, null, "false", false, sinceSeconds, tailLines, null, null);
            Response response = call.execute();

            StringBuilder sb = new StringBuilder();

            try (InputStream in = response.body().byteStream()) {
                sb.append(new String(in.readAllBytes(), StandardCharsets.UTF_8));
            }

            response.close();

            console.log(sb.toString());
        } catch (IOException | ApiException e) {
            // TODO Auto-generated catch block
            System.out.println(e.getMessage());
            e.printStackTrace();
            return "Encountered error";
        }

The 4th parameter of readNamespacedPodLogCall can be set to true to return a non-terminating InputStream which can be used to tail the logs.  Displaying the logs is left as an exercise for the reader.