Skip to main content
Version: 030-rc3

Updates

Getting back to revisions, these can be modified and upgraded, thus creating a new revision of the device.

Flow​

Updates will be processed differently depending on the source being remote or local.

Remote​

The diagram shows a high level concept of how remote updates work. In this case, we have a periodic routine that checks for pending updates upstream and executes the installation and transition to them one by one:

Local​

In this diagram, you can see how local updates work. In the case of this kind of update, we have a one-shot process triggered by a run revision command:

Progress​

Pantavisor keeps the update progress information for each revision in storage.

This info will also be reported to the cloud in case of the device being remote and communication available at that moment.

Pantavisor will only progress to the new revision in case of success. Otherwise, no action or rollback will ensue, depending on the point where the error is detected. This is the list of possible update states:

NEW​

Initial state for remote updates. Set by the cloud side.

SYNCING​

Device is syncing its first revision with the cloud. Set by the cloud side.

QUEUED​

Only valid for remote updates.

Pantavisor has got the state JSON of the new revision, but is performing other operations and has put it to the queue to be processed later.

Messages
Retried X of Y

DOWNLOADING​

Only valid for remote updates.

Downloading the artifacts for the new revision.

Object downloads are resumable: if a transfer is interrupted (dropped connection, timeout, or a Hub-side error), Pantavisor keeps the partial file on disk and retries with an HTTP Range request picking up from the last byte received, instead of starting the object over from scratch. Since objects are immutable and content-addressed by their sha256, any partial file left over from an earlier attempt is always safe to resume from. If the Hub does not honor the Range request (e.g. an older Hub without range support) and answers with a full 200 response instead of 206 Partial Content, Pantavisor falls back gracefully and restarts that object's download from scratch. This reuses the same retry loop as before. The step progress reports how often a download was resumed in downloads.total.total_resumes, but that is a counter, not a separate retry budget: a stuck object still only stops retrying once the update itself hits its overall retry ceiling.

Messages
Retry X of Y

INPROGRESS​

Installing or progressing to this revision. Transitions to new revisions can either require a reboot or not.

Hooks fire at key points during installation: before and after the bootloader writes the new revision (system-before-install-update / system-after-install-update), and once the revision has been committed after a successful try-boot (system-boot-done).

To finish this state, it is necessary that all status goals existing in the new revision have been achieved. Also, in the case of a remote update, Pantavisor needs to have performed communication with Pantacor Hub. If these two conditions are not met within a configurable time, Pantavisor will rollback the revision. The wait is set per group by timeout in groups.json, falling back to PV_UPDATER_GOALS_TIMEOUT (seconds, default 120) for any group that does not set it.

Messages
Update objects downloaded
Update applied
Update installed
Starting updated version
Transitioning to new revision without rebooting
Rebooting

Reboot transition​

Reboot transitions are performed based on the location of the changes belonging to the new revision update:

In this case, Pantavisor will stop all the containers and reboot the board.

Non-reboot transition​

Non-reboot transitions are performed after an update that does not contain any changes in any of the components described for the reboot transition.

In this case, Pantavisor will only stop the containers that were affected by the update and restart them with the recently installed new revision artifacts.

TESTING​

Waiting to see if the revision is stable. During this stage, Pantavisor checks if all containers are running and will rollback if any of them exits. Besides that, in the case of a remote update, it will also rollback in case Pantacor Hub communication is lost.

If any container has a stable_timeout, the commit is held even after the commit delay timer (PV_UPDATER_COMMIT_DELAY, seconds, default 25) expires, until all containers have survived their stability window. If a container with auto-recovery exhausts its max_retries during TESTING, a rollback is triggered immediately regardless of the configured backoff_policy.

Messages
Awaiting to set rollback point if update is stable
Awaiting to see if update is stable
Waiting for all containers to become stable

UPDATED​

The revision is stable, but the update did not need a board reboot, so the rollback point is not set until you force a reboot.

Messages
Update finished, revision not set as rollback point

DONE​

The revision has been fully booted and is stable. The rollback point is set.

Messages
Update finished, revision set as rollback point
Factory revision

WONTGO​

The new revision cannot be installed because of a bad state JSON, so it is aborted before getting into INPROGRESS or TESTING.

MessagesPossible causes
Update abortedLocal update cancelled by a command
Max download retries reachedThe maximum processing or download retry number was reached for a remote update
Space required X B, available Y BNot enough space in disk for remote update
Internal errorMemory allocation error or code bug
State not fully covered by signaturesThe revision has some element that is not signed
Signature validation failedAny of the revision signatures is not up to date with its covered content
Unknown errorSignature failed without a known cause
State JSON has bad formatState JSON could not be parsed

ERROR​

The new revision failed during INPROGRESS or TESTING stages. Pantavisor will try to rollback to the latest DONE revision.

MessagesPossible causes
Internal errorMemory allocation error or code bug
State not fully covered by signaturesThe revision has some element that is not signed
Signature validation failedAny of the revision signatures is not up to date with its covered content
Unknown errorSignature failed without a known cause
Object validation went wrongStored object used by the revision changed or went corrupt
Hub not reachableConnection with hub could not be achieved after a remote update
Hub communication not stableConnection with hub was established but then failed during TESTING
Stale revisionAn revision was found in hub with an index that is less than the current running revision one
Status goal not reachedStatus goal of a container could not be reached before the end of TESTING
A container could not be startedA container failed during LXC start up
Unexpected rollbackCrash or power cycle before having the chance to report any meaningful status

CANCEL​

Only valid for remote updates.

Set by the device owner from the cloud side, and honored by the device while the revision is still QUEUED or DOWNLOADING — checked every ~6 seconds by polling the device's own step with its device token; the download progress report of that tick is only sent once the poll came back without a cancel, so the device does not overwrite a cancel it has not seen yet. A cancel that lands in the short window between that poll and the progress report is still overwritten by it and has to be issued again. Any object downloads in flight are aborted, but partial objects already on disk are kept so a later re-post of the same revision can resume them instead of starting over. Once the revision reaches INPROGRESS, the device no longer looks for a cancel: the bootloader may already have been written and containers may already be starting, so stopping midway is no longer safe. A cancel set on the Hub at that point only keeps a device that lost track of the step (for example after a crash before the try was recorded) from fetching and retrying it, which is also what the older owner wontgo action does.

MessagesPossible causes
Cancelled as requested by ownerThe device owner cancelled the revision from Pantacor Hub

Reference​