On-Premises Debug Bundle

Overview

When you hit a problem with an on-premises FOSSA deployment, support will often ask for a debug bundle: a ZIP file containing information about your FOSSA application and the Kubernetes cluster it runs in. Generating one is a cluster-operator task. Enable it in your Helm values, let it collect, then copy the result off the pod and send it in.

The debugBundle deployment has shipped with the fossa/fossa-core chart since chart version 2.6.0. You generally won't need it unless FOSSA support asks.

What the bundle collects

The bundle is an exact snapshot of your release and cluster, written to a temporary directory on the debug-bundle pod. It includes:

• Application configuration: FOSSAConfigs, FeatureFlags, and SequelizeMeta tables.
• Build data: recent Builds, and the related Tasks, Revisions, and DependencyLocks.
• Cluster state: pods (with per-container logs), services, deployments, statefulsets, daemonsets, jobs, cronjobs, config maps, events, persistent volumes and claims, and the rendered Helm release values and manifest.
• Node diagnostics: nodes and node leases, only when cluster-role access is granted (see below).
• Endpoint connectivity: reachability checks to the external hosts FOSSA depends on (for example app.fossa.com, vulns.fossa.com, pypi.org, and s3.amazonaws.com).

Because this is a faithful copy of your cluster, the bundle can contain secrets. You'll redact those before sending. See Redact and send the bundle.

Generating the bundle

1. 1

   Enable the debug bundle

   Set deployment.debugBundle.enabled: true in your fossa/fossa-core values:

   YAMLCopy

   deployment:  debugBundle:    enabled: true

   If you have permission to create cluster roles, also set createClusterRole: true. This lets the bundle collect diagnostics about your cluster's compute nodes, which helps FOSSA diagnose hardware-provisioning and performance issues:

   YAMLCopy

   deployment:  debugBundle:    enabled: true    createClusterRole: true

   Both values default to false.

   Warning

   If the bundle logs errors about being unable to retrieve information about nodes and node leases, you most likely left deployment.debugBundle.createClusterRole at its default of false. The bundle is still useful without node data; it just won't include diagnostics about your underlying compute nodes.

2. 2

   Apply the new Helm values

   Apply the values the same way you apply any update, for most operators that's helm upgrade:

   ShellCopy

   helm upgrade RELEASE_NAME fossa/fossa-core --values NEW_VALUES_FILE

   Once the upgrade completes, you should see a RELEASE_NAME-fossa-core-debug-bundle deployment with a single pod:

   ShellCopy

   $ kubectl get deploymentsNAME                                   READY   UP-TO-DATE   AVAILABLE   AGERELEASE_NAME-fossa-core-debug-bundle   1/1     1            1           24h $ kubectl get podsNAME                                                    READY   STATUS    RESTARTS   AGERELEASE_NAME-fossa-core-debug-bundle-85bdd985d9-c8hm4   1/1     Running   0          71m

   Tip

   The deployment already exists at 0 replicas. Instead of editing values, you can scale it up directly, then pick up at the next step:

   ShellCopy

   kubectl scale deployment RELEASE_NAME-fossa-core-debug-bundle --replicas=1

3. 3

   Wait for collection to finish

   Once the pod is running, follow its logs:

   ShellCopy

   kubectl logs -f -l app.kubernetes.io/component=debug-bundle

   When collection finishes, the pod prints a completion message and a ready-to-run copy command, then exits about an hour later (removing the collected data with it):

   Copy

   [INFO] Collection of data is complete, this process will exit in an hour and data will be removed.
   [INFO] Use kubectl to copy the data: kubectl cp RELEASE_NAME-fossa-core-debug-bundle-85bdd985d9-c8hm4:/tmp/debug_167969928 LOCAL_DEBUG_BUNDLE_LOCATION
   [WARN] Information collected may contain secrets, please parse output and remove secrets.

   Copy the command from your own log output, the pod name and temp directory are unique to your run, and the command includes a --namespace flag when your release sets one.

4. 4

   Copy the bundle off the pod

   Run the kubectl cp command from the log output, replacing LOCAL_DEBUG_BUNDLE_LOCATION with a path on your machine:

   ShellCopy

   kubectl cp RELEASE_NAME-fossa-core-debug-bundle-85bdd985d9-c8hm4:/tmp/debug_167969928 ./fossa-debug-bundle

   The contents of that folder are the debug bundle.

5. 5

   Redact and send the bundle

   The bundle reproduces your cluster's state and may contain any secrets present in your Kubernetes manifests. Review and redact before sending. The most common places secrets appear:

   File	What to check
   values_sh.*.json	Passwords in the saved Helm values
   manifest_sh.*.yaml	Passwords in the rendered Kubernetes manifests
   config_maps.json	Passwords in the ConfigMap dump

   Warning

   Depending on your cluster's configuration, secrets may appear elsewhere too. Treat the table above as a starting point, not a complete inventory.

   Once you've redacted it, ZIP up the folder and attach it to your support email or ticket.