Migration Format Stability

PormG's migration history is persisted in two places that a user commits to version control and applies to a production database: the on-disk migration files and the pormg_migrations tracking table. Once those exist in a real database, they cannot be rewritten by a package upgrade — so their layout is a stability contract, frozen at format version 1.

This page documents that contract: the on-disk file format, the checksum algorithm, the tracking-table schema, and the version marker that lets future format revisions migrate forward.

Format version

Every artifact PormG writes is stamped with a single integer format version, sourced from the constant PormG.Migrations.MIGRATION_FORMAT_VERSION (currently 1):

• On disk — each generated migration file carries a comment header on the line directly below module …:

  module pending_migrations
  # pormg-migration-format: 1

  It is a comment, not a const, deliberately. A plan file is parsed as data and never run (see A plan file is read as data, never executed), and the reader accepts no statement other than import or using lines and table entries, so a const would make the file unreadable. A comment also needs no parsing: the regex ^# pormg-migration-format: (\d+)\r?$ finds it by scanning lines, so a future engine can detect the format before parsing a file whose shape the version-1 reader would refuse. The optional \r? keeps the scan line-ending agnostic (a file checked out with CRLF on Windows still matches).

• In the database — the pormg_migrations.format_version column records the format of each applied record. This column is the authoritative version source (the runtime history table is the source of truth; the file comment is a secondary on-disk annotation).

A future format change bumps MIGRATION_FORMAT_VERSION and provides a documented forward migration; existing format_version = 1 rows and files remain valid and are read under the v1 rules.

On-disk migration files

Location and naming

<db_folder> is the database's configuration folder (the directory holding its connection.yml). When an applied filename would collide, a random 4-digit suffix is inserted (YYYY-MM-DD_HH-MM-SS_NNNN_migration.jl). Applying a plan also snapshots the models file alongside it as YYYY-MM-DD_HH-MM-SS_old_models.jl. This is the file the plan was generated from: the connection's own model_file, or the one named by makemigrations(db; models_file = …). A plan generated from another file records it in a # pormg-models-file: <path> comment under the format marker, followed by a # pormg-models-sha256: <hex> line with the file's SHA-256 at that moment. The path is relative to <db_folder> when the file sits under it, and written with Julia's escape_string, so a backslash appears doubled. Neither comment is part of the checksum, and a plan without them is unchanged. The same holds for the # pormg-lossy-alter: lines below them — one per lossy column change, each a tab-separated list of key=value fields with escaped values — which dry_run and migrate read back and which change nothing about the plan's SQL. One field is written by the operator, never by makemigrations: handled=pre, which says a Data (pre): step of the plan fixes that change's rows (see Data steps in a plan). A PormG from before that field ignores it, so it counts the rows and refuses the plan as it always did. migrate snapshots the named file only if it still has that digest. If the header cannot be read, the file is gone, or the file changed after makemigrations, it logs a warning and takes no snapshot. The migration itself is still applied and archived.

Every plan makemigrations writes also carries one # pormg-schema-table: <fingerprint><TAB><table> line per table its diff compared, after any of the lines above: the schema the plan was generated against, which migrate checks before it applies anything (see Shipping a plan with a release). The fingerprint is 16 hex digits, or absent, and the table name is escaped like the models-file path. These lines are not part of the checksum either. A plan without them — one generated before the check existed, or one whose lines were deleted on purpose — is applied without the check; a line that does not read back is refused as InvalidMigrationError. The fingerprint's serialization is versioned: a change to it would refuse every plan generated by an earlier PormG, so it is announced in the upgrade log.

Content structure

A migration file is a Julia module named pending_migrations. Archiving it to applied_migrations/ moves the file without rewriting it, so an applied migration keeps that module name; its timestamp is in the file name. Each table touched by the plan is one variable bound to an OrderedDict{String, String} mapping a human-readable operation description to the SQL that performs it. Order is preserved, so the dict reads as an executable changelog:

module pending_migrations
# pormg-migration-format: 1

import PormG.Migrations
import OrderedCollections: OrderedDict

# table: results
results = OrderedDict{String, String}(
"New model" =>
 """CREATE TABLE IF NOT EXISTS results (
  "resultid" BIGINT PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY,
  "driverid" BIGINT NOT NULL,
  "points" DOUBLE PRECISION NOT NULL
);""",

"Create index on driverid" =>
 """CREATE INDEX IF NOT EXISTS "results_driverid_idx" ON results ("driverid");""")

end

The outer structure is OrderedDict{Symbol, OrderedDict{String, String}} — one entry per table, keyed by table name. Operation descriptions are stable strings such as "New model", "Add field: <name>", "Rename field: <name>", "Remove field: <name>", "Create index on <name>", "Remove duplicate index <index> on <name>", and "Drop table".

Two label prefixes are reserved for hand-written data steps (#740): an entry whose label starts with Data (pre): runs before every schema statement of the plan, and one starting with Data (post): after every one, whichever table's binding holds it. makemigrations never writes them. A label that reads like one but is neither prefix exactly — any case, any spacing, a missing colon — is refused with InvalidMigrationError. A plan with no such label orders, and therefore checksums, exactly as it did before these prefixes existed. See Data Migrations.

A plan file is read as data, never executed

The plan is written in Julia syntax, but PormG never runs it. dry_run(), migrate() and discard_pending_migration() parse pending_migrations.jl and accept only the shape shown above:

• one module holding import or using lines;
• # table: comments;
• one name = OrderedDict{String, String}("label" => """sql""", …) entry per table (untyped OrderedDict(…) is accepted too).

Every label and every SQL must be a plain string literal. Anything else raises an InvalidMigrationError naming the line, and nothing in the file is evaluated. That includes $ interpolation, a function call, string concatenation, a statement between entries, a name bound twice, and a label repeated within one entry (only its last SQL would survive). If you see that error, regenerate the plan with makemigrations().

This matters because a plan carries identifiers from the live database: undeclared table and index names, foreign-key and composite-constraint names. Before this rule, the file was included. An index named x$(run(…)) then ran as code on your machine at the next dry_run(), the step meant for inspecting a plan safely. A plain default such as "R$ 0,00" also produced a file that could not be loaded.

On the writing side, every string is escaped:

• in the SQL, \ and $ always, and a " only where it could end the literal;
• labels are written with Julia's repr;
• the table name in each # table: comment is escaped, so a newline cannot end the comment.

A plan without those characters is written exactly as before. The on-disk format is still version 1, and a plan file an earlier release wrote still loads with the same content. The exception is a file containing $ interpolation: it was either unloadable or ran code, so regenerate it.

Checksum

Each applied migration stores a SHA-256 integrity checksum (lower-case hex, 64 chars):

compute_checksum(sql_content) == bytes2hex(sha256(Vector{UInt8}(sql_content)))

The input is the migration's full SQL content — every statement in the plan concatenated in execution order. mark_applied() therefore requires either the migration's sql_content (from which this checksum is computed) or an explicit checksum; it will not fabricate one, so that every recorded digest stays verifiable against real SQL. The legacy manual fallback sha256("manual:" * version * ":" * name) remains a frozen v1 primitive for reproducing digests of records created that way before the requirement was introduced. Both schemes are frozen at format version 1: re-hashing a committed v1 migration must reproduce its stored checksum exactly.

Tracking table: pormg_migrations

The history table name is fixed (pormg_migrations, not configurable) and is created automatically by migrate() or init_migrations(). Its frozen v1 columns:

The version string is generated as Dates.format(now(), "yyyymmddHHMMSSsss") — date, time, and milliseconds, giving collision-resistant ordering for rapid successive migrations.

Upgrading an existing database

format_version was added in this release. A database initialized by an earlier PormG version has a pormg_migrations table without the column. init_migrations repairs that idempotently: on PostgreSQL via ALTER TABLE … ADD COLUMN IF NOT EXISTS, and on SQLite by probing PRAGMA table_info(pormg_migrations) and adding the column only when absent. Pre-existing rows backfill to 1 through the column DEFAULT — they were genuinely written under the v1 contract.

On SQLite, applied_at used to default to datetime('now'), which writes YYYY-MM-DD HH:MM:SS rather than the canonical YYYY-MM-DDTHH:MM:SS.sss+00:00 every DateTimeField stores. A table created by an earlier release keeps that default (CREATE TABLE IF NOT EXISTS never revisits it), so PormG writes the column explicitly on every migration record, and init_migrations rewrites rows still in the old form into the canonical text — once, idempotently, with no action on your part.

Data-step table: pormg_migrations_data

run_once records its steps in a table of its own, created on first use, so the pormg_migrations columns above are untouched (#740). Its name sits under the pormg_migrations prefix that every introspection ignore list carries, so makemigrations and check never plan anything for it.

It holds one row per applied step and nothing else: a step that failed is not recorded.

What "frozen" guarantees

Within format version 1, PormG promises:

1. A committed v1 migration file remains parseable and applicable.
2. Re-running compute_checksum over a v1 migration's SQL reproduces its stored checksum.
3. The pormg_migrations column set above is stable; new columns (if ever needed) arrive only with a format_version bump and a forward migration.
4. The version-string format stays yyyymmddHHMMSSsss.

The committed fixture in test/fixtures/migration_format_v1/, exercised by test/unit/test_migration_format_v1.jl, regression-guards points 1–2.