Versioning (git)
The engine keeps the live mapping state: operational templates, context and model mappers, and ConceptMaps. Versioning adds history on top of it:
every change is recorded in a journal, with the text that was written, who wrote it, when and why
a per-tenant git remote that you own (GitHub, GitLab, Gitea or any git server over https) receives those changes on push, as readable commits in one canonical repository layout
any commit, tag or branch can be restored, diffed or exported
releases (tags with release notes), blame per line and reviews (pull / merge requests) are built on top
The engine stays the source of truth. It has no git working copy on disk: every git operation runs on an in-memory repository that is discarded afterwards. Nothing is hosted by openFHIR.
Versioning is off by default. Turn it on with openfhir.versioning.enabled=true
(OPENFHIR_VERSIONING_ENABLED=true). While it is off, nothing is journaled, every /versioning endpoint answers
404 with "error": "versioning-disabled", and so do GET /$export and GET /openfhir/atlas/helpers when
given a ref. Plain $export / $import and the bootstrap mirror mode work either way. Without the journal,
exports write model and context mappers as canonical YAML rather than the text that was uploaded (comments and
formatting are not kept).
Everything below is available over the engine’s REST API alone. Atlas (Atlas) offers a UI for it, but is not needed. Clients that prefer to do their own versioning can use the plain export / import APIs instead (see Recipes).
Note
FHIR IG packages (/fhir/packages) are not versioned in git. An export can embed the packages the exported
model mappers reference, and .openfhir/packages.lock always lists them.
Repository layout
Exports, imports, git and the bootstrap mirror mode all use the same layout:
.openfhir/layout.properties layoutVersion=1
.openfhir/packages.lock <name>#<version> sha256=... of the referenced FHIR packages (informational)
templates/<templateId>/template.opt
templates/<templateId>/context.yml root context mapper of the template
contexts/<name>.context.yml sub-contexts (scope: sub)
models/<name>.yml
conceptmaps/<last url segment>-<hash>.json
packages/<name>#<version>.tgz only in exports with includePackages=true, never in git
An artefact is identified by its content, never by its file name: a model mapper by metadata.name, a context
mapper by its context.template.id (a sub-context by metadata.name), an OPT by its normalized template id, a
ConceptMap by its url. File names are derived from these keys and sanitised (a name that is not a safe file name
gets a short hash appended). The importer therefore accepts any layout: this one, a flat zip, or a bootstrap
directory.
Model and context mappers are written as the text you wrote, comments and formatting included, as long as it still
matches what the engine holds. Otherwise, and for JSON bodies, a canonical YAML form is written instead. ConceptMaps
become pretty JSON with sorted keys, without the engine id and meta.lastUpdated / meta.versionId. OPTs are
written as stored, with LF line endings. Archives are deterministic: the same state always produces the same bytes.
Export and import
These two endpoints work without any git remote.
- GET /$export
The state of the tenant in the repository layout.
- Query Parameters:
scope –
all(default),template:<templateId>,model:<name>,conceptmap:<url>; repeatable and comma separated. A template covers its OPT, its context mapper, the models and sub-contexts it listsformat –
zip(default),tgzorjson(a bundle{opts[], models[], contexts[], conceptMaps[], packages[]})ref – export a commit, tag or branch of the tenant’s git remote instead of the live state (needs
versioning.r)includePackages – embed the referenced FHIR packages under
packages/(needsfhirpackage.r)includeDependencies – for template scopes, also export
extendsparents and referenced ConceptMaps (defaulttrue)
- Response Headers:
ETag – hash of the exported content; send it back in
If-None-Matchto get304while nothing changedX-OpenFHIR-Exported-At – time of the export
X-OpenFHIR-Commit – the commit exported, with
ref
- Status Codes:
200 OK – the export
304 Not Modified – unchanged since the
If-None-MatchETag
- POST /$import
Imports state atomically: either every change applies or none does. The body is a JSON bundle (as
$export?format=jsonwrites it), a zip or tar.gz in any layout, amultipart/form-dataupload of such files, or a single file.- Query Parameters:
mode –
merge(default) creates and updates;mirroralso deletes what is in scope but not imported (needs thefc.d,opt.dandconceptmap.dscopes)dryRun – only compute the plan. Defaults to
trueformirror,falseformergescope – as for
$export; artefacts outside the scope are ignoredplanHash – apply only if the plan computed now has this hash (from a reviewed dry run);
409otherwisemessage – message of the journal batch (and so of the commit)
importPackages – also store the FHIR packages under
packages/(needsfhirpackage.c)allowErrors –
trueapplies despite errors, the way imports of versioned content do (see “Git remote” below): what the engine can store is imported, what it can’t is left out with a warning and listed underleftOut. A dry run withallowErrorsplans exactly that, so itsplanHashfits the import. Defaultfalse: any error and nothing is applied
- Status Codes:
200 OK – the plan, dry run or applied (with
allowErrors, also when it lists errors)400 Bad Request – the input is invalid; the body is the plan with its
errors, nothing was applied409 Conflict –
planHashdoes not match the current state500 Internal Server Error – applying failed and was rolled back
The plan lists every artefact with an action (
CREATE,UPDATE,DELETE,UNCHANGED), pluserrors,warningsand theplanHash:{ "mode": "merge", "scope": "all", "items": [ {"kind": "model", "key": "openEHR-EHR-OBSERVATION.blood_pressure.v2", "action": "UPDATE", "path": "models/openEHR-EHR-OBSERVATION.blood_pressure.v2.yml", "entityId": "6adc95900a576ab5b27e237b"} ], "errors": [], "warnings": ["context:vital_signs references model:openEHR-EHR-CLUSTER.device.v1, which is neither imported nor in the engine; that archetype is not mapped"], "planHash": "1975e500f8d16ff004bb28e5df509d9a6190d085bc2e229c4fd020d1be6b81c7", "dryRun": false, "applied": true, "batchId": "b5d6633c-893d-4170-be59-395c08fd6fa4" }
What is validated before anything is written: the FHIRConnect schemas of model and context mappers, OPTs and ConceptMaps parse,
extendsparents exist and are not cyclic, sub-contexts exist, and natural keys are not duplicated. A missing OPT, a ConceptMap that is not imported, an archetype without a model mapper and a FHIR package frompackages.lockthat is not installed are only warnings, since the engine works without them.Existing artefacts are updated in place, so their ids (and mapping insights referring to them) stay the same. Deletes run first, then OPTs, ConceptMaps, models (
extendsparents first) and contexts (sub-contexts first). Inmirrormode with a narrower scope thanall, an artefact still referenced by something outside the import (a model shared with another template, say) is kept, with a warning.Limits: 1000 entries, 50 MB per entry, 200 MB in total. Archive entries escaping the archive root and XML with a
DOCTYPEare refused.
Note
On PostgreSQL an import is one database transaction. On MongoDB it is a multi-document transaction when the server is a replica set or sharded cluster; on a standalone server a failed import is undone by restoring what it had changed.
The journal
Every successful write through /fc/model, /fc/context, /opt, /terminology/fhir/ConceptMap,
$import, the bootstrap scan and the versioning operations below is recorded as a journal row: the artefact, the
operation (upsert or delete), the text written, the user (preferred_username of the token), a source label, the
request id, and the commit it was pushed in. A write that does not change anything is not recorded.
Writes that follow each other closely share a row: when the artefact’s latest row is not pushed yet, was written by
the same user from the same source less than coalesceMinutes (default 10) before, and carries no message or batch,
the new write replaces it. An editor that saves while you type (such as Atlas) so leaves one row per editing session,
and its commit holds the final text only; intermediate saves are not kept.
Any API client can label its writes with optional headers:
Header |
Effect |
|---|---|
|
free label of the client, e.g. |
|
writes with the same value become one commit |
|
commit message for the change; it also closes the current commit group |
- GET /versioning/revisions
Journal rows, newest first, without content. Filter with
kind(opt,model,context,conceptmap) andkey; page withlimitandbefore=<seq of the last row received>. A row submitted for review carriesreviewIdwith the review’sreviewState,reviewNumberandreviewUrl(the review is read from the provider when due, as forGET /versioning/reviews). ItscommitShais on the review branch, sopushedonly means it is on main oncereviewStateismerged.
- GET /versioning/revisions/(id)
One journal row, with its content.
- GET /versioning/diff
Two versions of an artefact and their unified diff.
fromandtoare each a journal row id, a git ref of the remote, orengine(the live state; default forto).path(orkindandkey) names the artefact unlessfromortois a journal row.
- POST /versioning/checkpoint
{"message": "..."}names the next commit: the message is set on the caller’s latest unpushed change, which closes its commit group.409when nothing is unpushed.
Journal rows older than openfhir.versioning.journal-retention-days (7) are pruned daily once git has them, or
when the tenant has no git remote at all (there is nothing to push them to, and connecting commits the full engine
state anyway). The latest row of every existing artefact and the rows of reviews are always kept, and so are unpushed
rows of a tenant with a remote. Without a remote the journal so holds about one row per artefact, plus the changes of
the last week. GET /$purge clears the journal of the tenant together with the rest of its state.
Git remote
Everything that imports versioned content into the engine (connect with repoToEngine, pull,
restore, taking the remote’s version of a conflict, reverting the engine after a review) treats errors the same
way: unlike $import, they don’t stop it. Everything the engine can store is imported, references that don’t
resolve included (a model whose extends parent is missing, say). An artefact the engine can’t store (invalid
content, a context whose sub-context isn’t there) is left out with a warning, and the engine keeps its own version of
it. The applied plan lists the errors and answers 200; a dry run with errors answers 400, as for $import.
- PUT /versioning/config
Settings of the tenant. Fields left out stay unchanged.
{ "remoteUrl": "https://github.com/acme/openfhir-mappings.git", "branch": "main", "token": "github_pat_...", "autoPush": "after-change", "coalesceMinutes": 10, "workflow": "direct" }
tokenis stored encrypted and never returned;GET /versioning/configonly shows a masked hint. An emptytokenremoves it. Storing a token needsopenfhir.versioning.secret-key.A token is never sent to a host it was not given for: moving
remoteUrlorapiBaseUrlto another host removes the stored token, unless the same request sets a new one. Setting the first remote counts as a move (a token stored before any remote is removed); an API on the remote’s own host, or the provider’s default API for it (api.github.comforgithub.com), does not.username: user for the token, if the server needs one (GitHub and GitLab do not).provider(github,gitlab,giteaornone) is detected from the remote URL unless given, andapiBaseUrlis derived from the remote and provider whenever either changes, unless given; GitHub Enterprise and self-hosted GitLab / Gitea work throughapiBaseUrl. A provider is needed for reviews and provider releases only.autoPush:never(default),after-change(a push runs shortly after the last change) orinterval(unpushed changes are pushed once their last push is older thancoalesceMinutes).workflow:direct(push to the branch) orreview(changes reach the branch through reviews only).authorEmailOverride: e-mail address used for every commit author.
While a git operation of the tenant (push, pull, connect, review, release) runs, the update answers
409with"error": "busy"; the same holds forDELETE.
- GET /versioning/config
The settings, without the token.
- DELETE /versioning/config
Removes the settings and the stored token. The journal and the remote are not touched.
- POST /versioning/config/test
Lists the branches of the remote with the stored credentials and, with a provider, checks whether the token may push.
- POST /versioning/connect
Initial setup. An empty branch gets the full engine state as its first commit. A branch that already has commits answers
409with"error": "remote-not-empty"unless adirectionis given:{"direction": "engineToRepo"}commits the engine state on top of it; files outside the engine’s directories (a README, CI configuration) are kept{"direction": "repoToEngine"}imports it into the engine, mirror by default, withdryRunandplanHashas for$import. With"mode": "merge", engine changes to artefacts the repository lacks (or holds differently) stay unpushed, so the next push adds them. Errors don’t stop it (see above)
- POST /versioning/push
Pushes the unpushed journal rows as commits. Rows of the same user, written within
coalesceMinutesof each other, become one commit, until a row carries a message; a batch is always one commit. Commit messages are the checkpoint message or a summary (Update model x; delete context y), with the trailersOpenFHIR-User,OpenFHIR-Tenant,OpenFHIR-SourceandOpenFHIR-Commit-Kind. The commit author is the user who made the change; the committer isopenFHIR.If the branch moved on since the last push, the commits are written on top of the new tip. A file that changed both on the remote and in the unpushed rows is a conflict: the push answers
409with"error": "conflict"and theconflicts. Push again with{"resolve": "keepEngine"}to overwrite the remote version, or{"resolve": "takeRemote"}to import the remote version into the engine.A push never takes other remote changes into the engine; that is a pull, with its dry run and scopes. It answers
409with"error": "pull-required"(and theremoteState) when the remote changed artefacts the engine has no unpushed changes to (ahead), when the engine’s last push is unknown (untracked) or when it is no longer on the remote (diverged), also when there is nothing to push. Foruntracked, connecting is usually the right step: a pull would delete every engine artefact the repository lacks. Remote changes outside the engine’s directories (a README) don’t count. On an empty branch (never connected, or deleted since) it answers409with"error": "connect-required": connect writes the full engine state. Automatic pushes stop with the same errors (and on conflicts), shown aslastErrorin the status, and are not tried again until the engine’s last push or the remote branch moves: someone pulls, connects, resolves the conflict, or pushes to the branch.In the review workflow the push answers
409with"error": "review-required".
- POST /versioning/pull
Brings the remote’s changes into the engine: every artefact the remote changed since the engine’s last push is imported from the tip of the branch, or deleted when the tip no longer has it. When that push is unknown (
untracked) or no longer on the remote (diverged), every artefact that differs from the tip counts.{"dryRun": true}(the default) answers theplan, the incomingcommitsand theconflicts: artefacts the remote changed that also have unpushed engine changes. Apply with{"dryRun": false, "planHash": "..."}; with conflicts,"resolve": "keepEngine"leaves them as they are (the next push overwrites the remote) and"resolve": "takeRemote"imports them, resolving their unpushed rows. Applying withoutresolvewhile there are conflicts answers409with"error": "conflict". Once applied, the tip counts as pushed, so the next push stacks onto it; this also holds when nothing had to be imported. The import is journaled with sourcesyncand counts as pushed. Applying needs the state write and delete scopes. Works in both workflows.
- GET /versioning/status
Unpushed changes, how the remote relates to the engine’s last push, open reviews, the last push and the last error (also of automatic pushes).
remoteStateisempty,inSync,ahead(remoteAheadBycommits on top of the last push),diverged(the remote no longer contains the last push: rewritten history) oruntracked(the engine has not pushed to or pulled from the branch yet, or the remote or branch setting changed since);remoteAheadis true for the last three.
- POST /versioning/restore
{"revisionId": "..."}restores one artefact to the version a journal row recorded (a delete row deletes it){"ref": "v1.2.0", "scope": ["template:Vital signs"], "mode": "merge", "dryRun": true}imports the state as of a commit, tag or branch, like$import(planHashto apply a reviewed plan).kindandkeyrestore a single artefact from the ref
Restores are journaled with source
restoreand a message naming the commit.
- GET /versioning/remote/log
Commits of the remote (
ref, default the configured branch), newest first, optionally only those that changedpath.
- GET /versioning/remote/file
A file of the remote at
ref.
Releases
A release is an annotated tag on the main branch: v<semver> for the whole state, or <templateId>/v<semver>
for one template. The tag message holds the release notes (Markdown) and the trailers OpenFHIR-Release-Scope
and, when supplied, OpenFHIR-Coverage (a JSON summary, which Atlas computes).
- GET /versioning/releases/draft
For
scope(allortemplate:<id>): the changes since the last release of that scope (added, changed, removed artefacts), checkpoint messages, restores, contributors, a suggested version (a removal bumps the major version, an addition the minor, a change the patch), a notes template, and pre-checks (unpushed changes, open reviews touching the scope).
- POST /versioning/releases
{"scope": "all", "version": "1.3.0", "notes": "## v1.3.0 ...", "bump": "release", "coverage": {"mapped": 412}}
In the direct workflow, unpushed changes are pushed first.
bumpsetsmetadata.versionof the model and context mappers changed since the last release:releasesets the release version,semver-incrementincrements each mapper’s own semantic version by the release’s level (other version strings are left alone),none(default) changes nothing. The bump is journaled with sourcereleaseand pushed.The tag is created and pushed, and with a provider a provider release is created.
The result is
released,tagged(no provider release) or, in the review workflow,pending: the unpushed changes in the release’s scope (for a template release: the template’s artefacts, as on main and in the engine) and the bump go into a “Release <tag>” review, andPOST /versioning/releases/{tag}/retry-tagcreates the tag once that review is merged (/versioning/releases/<templateId>/<tag>/retry-tagfor template releases).
- GET /versioning/releases
Releases, newest version first, with notes and coverage read from the tags.
scopenarrows them (allfor tenant-wide releases). A release is exported withGET /$export?ref=<tag>.
Blame
- GET /versioning/blame
For every line of the current text of an artefact (
kind,key): the journal row or commit that last changed it, its author, time and message. Pushed history is read from the remote (at mostopenfhir.versioning.blame-depthcommits; older lines are attributed to the oldest commit read), unpushed changes from the journal; without a remote, from the journal alone. Lines changed before the journal existed carry no attribution.
Reviews
Reviews need a provider (github, gitlab or gitea).
- POST /versioning/reviews
{"topic": "...", "title": "...", "description": "...", "rowIds": [...]}puts the given journal rows (default: all unpushed changes of the caller) on a branch<prefix>/<user>/<topic>off the main branch and opens a pull / merge request. The rows then count as submitted, not as unpushed.
- GET /versioning/reviews
Reviews, newest first. Open reviews are read from the provider when last read more than 60 seconds ago (
GET /versioning/reviews/{id}?sync=trueforces it):state(open,approved,changes_requested,merged,closed),checks(pending,success,failure,none) andmergeable.After a merge,
conflictslists the files whose content on main differs from the engine (a squash or rebase merge, or an edit made in the review).POST /versioning/reviews/{id}/reconcilewith{"resolve": "takeRemote"}imports main’s version,{"resolve": "keepEngine"}queues the engine’s version again.Once a merge (and its reconcile) leaves nothing on main the engine lacks, main’s tip is recorded as pulled, so the merge commit does not make the remote look ahead. When main also brought other changes, the status says
aheaduntil a pull.A pull treats artefacts in open reviews like unpushed changes: they are conflicts, kept by default.
A review closed without merge sets
engineDiverged: the engine still holds its changes.POST /versioning/reviews/{id}/revert-engineputs the engine back to main for those files.
POST /versioning/reviews/{id}/append adds further changes to an open review, POST /versioning/reviews/{id}/resubmit
opens a new review with the same changes once the review was closed without merge (any other state answers 409
with "error": "invalid-state"; if opening the new review fails, the changes stay with the closed one).
Helpers as of a ref
GET /openfhir/atlas/helpers?templateId=<id>&ref=<commit, tag or branch> builds the mapping helpers of a template
from the context, models and OPT of that commit, without touching the live state. Atlas uses it to compare mapping
coverage between versions.
Bootstrap mirror mode
With OPENFHIR_BOOTSTRAP_MODE=mirror (or POST /$bootstrap?mode=mirror) the bootstrap directory is imported like
$import?mode=mirror: atomically, files recognised by content, and artefacts that are not in the directory are
deleted. A deployment can mount a git checkout as the bootstrap directory and always run exactly that state. A
directory without any engine state is refused rather than mirrored. A directory with errors is imported the way
connect imports a repository that wins, leaving out only what the engine can’t store; with
OPENFHIR_BOOTSTRAP_ALLOW_ERRORS=false it changes nothing. See State configuration.
Configuration
Property (environment variable) |
Default |
Meaning |
|---|---|---|
|
|
the whole feature: the journal, the |
|
encrypts git tokens at rest (AES-GCM); required to store a token. Changing it makes stored tokens unreadable |
|
|
(any) |
only allow remotes and provider APIs on these hosts; |
|
|
refuse remotes that resolve to private, loopback or link-local addresses |
|
|
download cap of one git operation |
|
|
journal rows older than this are pruned once pushed, or when the tenant has no remote (see The journal);
|
|
|
commits read for blame |
|
|
commits read for the remote log and release drafts |
|
|
prefix of review branches |
|
|
poll open reviews on this interval; |
|
|
timeout of git and provider calls |
|
|
wait after the last change before an |
|
|
|
|
|
a directory with errors still imports what the engine can store (a mirror like a connect from the
repository); |
Remotes must be https URLs without credentials in them. openfhir.versioning.allow-http-urls and
openfhir.versioning.allow-file-urls exist for tests against a local git server only.
Note
Pushes, connects, reviews and releases read the full history of the branch into memory (reads, such as exports,
diffs and restores, only what they need), so very large repositories need a higher max-pack-mb. Git
operations of a tenant run one at a time across the cluster; a concurrent one answers 409 with
"error": "busy".
Scopes
versioning.r reads (settings, status, journal, remote log and files, diffs, blame, releases, reviews);
versioning.w writes (settings, connect, checkpoint, push, pull, releases, reviews). Operations that change the engine
state (restore, pull, repoToEngine, takeRemote, releases, reconcile, revert) also need the fc, opt and
conceptmap create / update scopes, and the delete scopes when they may delete. See Authentication.
Recipes
1. Version the state in your own git repository. No settings in the engine are needed:
curl -sf -H "Authorization: Bearer $TOKEN" "$ENGINE/\$export?format=zip" -o state.zip
rm -rf repo/templates repo/contexts repo/models repo/conceptmaps repo/.openfhir
unzip -oq state.zip -d repo
git -C repo add -A && git -C repo commit -m "openFHIR state $(date -u +%FT%TZ)" || echo "no changes"
The export is deterministic, so only real changes show up. Keep the ETag response header and send it as
If-None-Match next time to skip the download (304) while nothing changed.
2. Deploy a state from git.
(cd repo && zip -qr ../state.zip templates contexts models conceptmaps .openfhir)
PLAN=$(curl -sf -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/zip" \
--data-binary @state.zip "$ENGINE/\$import?mode=mirror&dryRun=true")
echo "$PLAN" | jq '.items[] | select(.action != "UNCHANGED")' # review
curl -sf -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/zip" -H "X-OpenFHIR-Source: ci" \
--data-binary @state.zip \
"$ENGINE/\$import?mode=mirror&dryRun=false&planHash=$(echo "$PLAN" | jq -r .planHash)"
3. GitOps. Mount a checkout of the repository as the bootstrap directory and set OPENFHIR_BOOTSTRAP_MODE=mirror.
Every start (and every POST /$bootstrap) makes the engine hold exactly that state.
4. Engine-managed git. PUT /versioning/config once, then from CI: POST /versioning/push to publish
changes, POST /versioning/releases to cut a release, GET /versioning/releases to list them and
POST /versioning/restore to roll back.
5. A GitHub Actions workflow for recipes 1 and 2:
name: openFHIR state
on:
schedule: [{cron: "*/30 * * * *"}] # snapshot the engine every 30 minutes
push:
branches: [main] # deploy what is merged
jobs:
snapshot:
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
permissions: {contents: write}
steps:
- uses: actions/checkout@v4
- name: Export
run: |
curl -sf -H "Authorization: Bearer ${{ secrets.OPENFHIR_TOKEN }}" \
"${{ vars.OPENFHIR_URL }}/\$export?format=zip" -o /tmp/state.zip
rm -rf templates contexts models conceptmaps .openfhir
unzip -oq /tmp/state.zip -d .
- name: Commit
run: |
git config user.name "openfhir-snapshot" && git config user.email "openfhir@users.noreply.github.com"
git add -A && git commit -m "openFHIR state snapshot" && git push || echo "no changes"
deploy:
if: github.event_name == 'push' && !contains(github.event.head_commit.message, 'openFHIR state snapshot')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Import
run: |
zip -qr /tmp/state.zip templates contexts models conceptmaps .openfhir
curl -sf -H "Authorization: Bearer ${{ secrets.OPENFHIR_TOKEN }}" -H "Content-Type: application/zip" \
-H "X-OpenFHIR-Source: github-actions" -H "X-OpenFHIR-Message: Deploy ${GITHUB_SHA::8}" \
--data-binary @/tmp/state.zip \
"${{ vars.OPENFHIR_URL }}/\$import?mode=mirror&dryRun=false" | jq .