Dependency Metadata Migration
Prepare your on-premises instance for the removal of the legacy dependency-metadata table by running the dependency backfill and waiting for it to complete before the follow-up upgrade.
Overview
FOSSA is retiring a legacy dependency-metadata table (DependencyLocks) and moving all dependency-graph data into a newer storage service. This improves query performance and keeps dependency data consistent across the platform.
For on-premises instances, this rolls out across two separate upgrades:
- A backfill release that copies all of your existing dependency data into the new storage.
- A later cleanup release that removes the old table.
You must run the backfill release and wait for it to finish before installing the cleanup release.
Warning
Do not install the cleanup release until you have received the backfill completion email described in this guide. Removing the table before the backfill finishes can result in the loss of dependency details for scans that had not yet been migrated.
Note
Running the backfill preserves your historical dependency data. It copies the dependencies for every project revision on your instance into the new storage before the old table is removed, so you will not lose dependency details for any of your scans, no matter how old.
The two releases
| Step | Helm chart version | What it does |
|---|---|---|
| 1. Backfill | 7.0.0 | On deploy, automatically starts a background job that copies all dependency data into the new storage. |
| 2. Cleanup | 8.0.0 | Removes the legacy DependencyLocks table. Safe to install only after the backfill has completed. |
Install them in order. The two upgrades follow the standard Upgrading FOSSA process; nothing about the upgrade mechanics changes.
Before you begin
Confirm that at least one enabled super-user account on your instance has a valid, monitored email address. The backfill completion notification is sent to super users, and that email is your signal to proceed to the cleanup release.
Running the migration
- 1
Upgrade to the backfill release
Upgrade your instance to Helm chart version
7.0.0using the standard Upgrading FOSSA process.The upgrade completes normally and does not require a maintenance window or downtime. When the new version starts, it automatically enqueues a background job that begins copying your dependency data into the new storage. The upgrade does not wait for this job to finish, so your deploy completes in the usual amount of time.
- 2
Let the backfill run
The backfill runs entirely in the background on a low-priority queue and is paced deliberately so it does not compete with normal scanning and analysis. No action is required from you during this period, and FOSSA remains fully usable.
See Timeline and impact for how long to expect this to take.
- 3
Wait for the completion email
When every dependency record has been copied successfully, FOSSA automatically emails all super-user accounts on your instance:
- Subject:
[FOSSA] Sparkle depgraph backfill complete - Body: confirmation that the dependency-graph backfill has finished and all export jobs have run.
This email is your signal that it is safe to continue. If you do not receive it, do not proceed. Contact FOSSA support instead.
- Subject:
- 4
Upgrade to the cleanup release
Once you have received the completion email, upgrade to Helm chart version
8.0.0. This release removes the legacy table. Because the data has already been migrated, this is a routine upgrade with no dependency-data impact.
Timeline and impact
How long the backfill takes
The backfill is paced to protect instance performance rather than to finish as fast as possible. It processes roughly 500,000 to 600,000 project revisions per day, so completion time scales with the size of your instance:
| Approximate project revisions | Expected completion |
|---|---|
| Up to ~250,000 | Within a day |
| ~500,000 to ~1 million | 1 to 2 days |
| ~2 million | 3 to 4 days |
| 4 million or more | Up to ~7 to 10 days |
Most instances complete within a few days. Rather than tracking these estimates, treat the completion email as the definitive signal that the backfill is done.
What you will see during the backfill
- No downtime. The application stays fully available throughout.
- No maintenance window required for the backfill itself.
- Minimal load impact. Work runs on a low-priority queue and is spread out over time, so normal scanning and analysis are not disrupted.
- No user-facing changes while the backfill runs. Dependency details continue to display as normal.
After the cleanup release
Once the legacy table is removed, your dependency data continues to display as before. Because the backfill copies every project revision on your instance into the new storage, dependency details remain available for all of your scans, regardless of age. This includes original SBOMs from SBOM imports and any revisions referenced in a release group.
If any historical scan is ever found to be missing dependency details, re-running an analysis of that project restores them.
Checking progress
The completion email is the definitive signal, and no monitoring is required. If you would like to watch progress in the meantime, the options below all give an approximate view of how much work remains.
Warning
These are progress indicators, not completion criteria. The backfill is done only when you receive the completion email. Do not proceed to the cleanup release based on these signals alone.
Job-queue depth
The backfill runs as background jobs on FOSSA's low-priority job queue. While signed in as a super user, open the following on your instance:
https://YOUR-FOSSA-HOST/api/jobs/faktoryThis returns a JSON snapshot of the background-job system. Two values track the remaining backfill work:
faktory.queues["low-priority"]: jobs that are due to run now.faktory.tasks.Scheduled.size: jobs scheduled for a future time slot. The backfill deliberately spreads its work into future slots, so early on most pending work sits here.
Both counts trend toward zero as the backfill progresses. Because other routine jobs also share these queues, read the numbers as an approximate trend rather than an exact remaining count.
Application logs
The pods that run FOSSA's background jobs log two milestones you can search for:
Enqueue complete (logged near the start; reports the totals):
Completed on-prem Sparkle depgraph sweep: enqueued <N> chunks (<M> revisions)Each chunk is up to 1,000 revisions, so this line tells you the total volume that will be processed.
Backfill complete (the same event that sends the email):
Sparkle depgraph backfill complete
For example:
kubectl -n NAMESPACE logs <core-worker-pod> | grep "Sparkle depgraph"Adjust the namespace and pod name to match your installation.
Faktory dashboard
If your deployment exposes the Faktory dashboard, you can watch the low-priority queue and the scheduled-jobs set drain there directly. The backfill appears as a batch named Sparkle depgraph backfill.
Frequently asked questions
Do I need to schedule downtime? No. Both upgrades are routine, and the backfill runs in the background with the application fully online.
Can I keep using FOSSA while the backfill runs? Yes. All functionality remains available. Scanning, analysis, and dependency views work normally.
What if the backfill is interrupted, for example by a restart during the run? The process is safe to re-run. Contact FOSSA support and we will help you re-trigger it. Re-running does not duplicate or corrupt data.
What happens if I install the cleanup release too early? Do not do this. Installing the cleanup release before the backfill completes can result in the loss of dependency details for scans that had not yet been migrated. Always wait for the completion email.
Who receives the completion email? Every enabled super-user account on your instance that has an email address configured.
Need help?
If you have any questions or concerns, or if you do not receive the completion email within the expected window for your instance size, contact your Customer Success representative or support@fossa.com.