Advanced Configuration

PormG is designed for Julia's asynchronous task scheduling. This section covers connection pooling, advisory locks, and the performance implications of the async-first design.

Connection Pooling & Async-First Design

  • Async-First API: The synchronous fetch() API is a wrapper around the asynchronous fetch_async() core. This wrapper yields to the Julia scheduler while waiting for the database, ensuring compatibility with async frameworks. See the Async & Concurrency guide for usage patterns.
  • Pooling Strategy: The default strategy is :poll, which retries at configured intervals. Use :block on PostgreSQL or MySQL to let the server-side manage the wait, bounded by timeout_ms through PostgreSQL's lock_timeout or GET_LOCK's own timeout (whole seconds, rounded up).
  • Sizing & capacity: A pool starts at pool_size connections (default 3) and grows lazily, on demand, up to pool_size × 10 under concurrent load — so the idle footprint stays small while async fan-out still gets headroom. If every connection is busy and none frees within the retry/timeout budget, acquire_connection raises a catchable PoolTimeoutError (exported by PormG). Raise pool_size in your connection.yml to add capacity for genuinely high-concurrency workloads:
dev:
  adapter: PostgreSQL
  database: 'formula1'
  # ...
  pool_size: 10   # base 10 → grows to 100 under burst

A misspelled pool key (pool:, poolsize:, pool_timout:) is reported on load with a suggestion instead of silently leaving the default in place — see Unrecognised keys.

  • Acquire timeout (pool_timeout): How long acquire_connection waits for a free connection before raising PoolTimeoutError — default 30 s. Set pool_timeout: in connection.yml (seconds; fractional allowed) to fail fast instead of blocking a request while the pool is saturated:

    dev:
      adapter: PostgreSQL
      database: 'formula1'
      pool_size: 10
      pool_timeout: 5   # give up after 5s waiting for a connection, then raise PoolTimeoutError

    An explicit per-call acquire_connection(pool; timeout_seconds=…) still overrides it, and Configuration.register_connection accepts the same pool_timeout kwarg. A value ≤ 0 falls back to the 30 s default (a "never wait" setting is ambiguous and a footgun). Absent means the historical 30 s — zero behavior change.

  • Connect failures (PoolConnectError, fail_fast_on_connect): pool_timeout covers a saturated healthy pool. A pool that can never open a connection — a wrong password, a missing role/database, an unopenable SQLite path — is a different failure: waiting pool_timeout for it to "free up" is pointless, and blaming pool_size is misleading. acquire_connection therefore classifies the driver error: a permanent one (PostgreSQL auth / missing role or database; MySQL access denied, unknown database, a host not allowed, an authentication plugin either side lacks, or a server older than 8.4; SQLite unable to open database file) fails fast with a catchable PoolConnectError (exported) that carries the underlying driver cause and a redacted connection string — remedy: fix credentials/host/database, not pool_size. Ambiguous errors (host/DNS/network, a refused connect while the server restarts — possibly a transient blip) are not fast-failed. The caller retries the connect, one attempt at a time, after a backoff of 50 ms doubling to 0.5 s, so it gets a connection within about half a second of the server coming back; if a connection is released meanwhile, the caller takes it at once. During an outage each waiting caller costs the server about five connect attempts in its first second, then two a second. A connect runs without holding the pool's lock (#1540): the caller reserves a slot, connects, then installs the connection. While one caller waits on a slow or unreachable server, every other caller can still return a connection, borrow an idle one, or give up at its own pool_timeout. Callers connect side by side, so during an outage the server sees up to one attempt per waiting caller at a time, not one for the whole pool. The reserved slot counts as in_use in pool_stats, and leak detection times it from the reservation, so a connect slower than leak_detection_threshold is reported, worded as a connect (see Leak detection below). If the deadline passes first, it raises PoolConnectError (with the last cause) rather than PoolTimeoutError. With the default PostgreSQL driver (LibPQ), recognizing a permanent failure means reading the server's message, because libpq gives a failed connect no SQLSTATE. PostgreSQL translates that message by its lc_messages setting, so on a server set to another language (pt_BR, say) a wrong password is not recognized and waits out pool_timeout like an ambiguous error. Set lc_messages = 'C' in postgresql.conf to keep the fast-fail; a per-role or per-database setting does not apply before authentication. The Postgres.jl driver reads the SQLSTATE and is unaffected. A PostgreSQL connection string that cannot even be parsed — typically a hand-written url: or register_connection string with an unquoted space inside a value, a malformed %-escape, or a NUL byte — is permanent too: it fails fast with a PoolConnectError whose cause is an InvalidConfigurationError. PormG checks the string before handing it to the driver, so the part of libpq's message that would quote your password is masked and nothing is logged by the driver itself; quote a value containing spaces (password='two words') or percent-encode it in a URL. A URL password holding an unencoded @ is refused the same way (libpq would end the password at the first @ and try the rest as a host name or port) — write it as %40, and a : as %3A. An unencoded / is refused too — libpq ends the credentials at the first / and reads the rest of the password as the database name — so write it as %2F. The check behind that refusal is "the URL's database name contains an @", so a database whose name really contains an @ cannot be reached through a URL (not even as %40, which libpq decodes first); give that one in the keyword form, dbname='…'. When the password's tail also holds a ?, libpq reads the rest as a query option instead, so in a URL only user, password, sslpassword, application_name and fallback_application_name may carry an @ (?user=me@server stays accepted), and even those are refused when an unencoded @ is followed by what looks like a host, port or path (?user=@db1/f1). Write a ? in a password as %3F, percent-encode the @ of an allowed value that trips that check, and give any other option whose value really contains an @ (a certificate path, say) in the keyword form. Set fail_fast_on_connect: false to opt out of the fast-fail and keep waiting the full pool_timeout:

    dev:
      adapter: PostgreSQL
      database: 'formula1'
      pool_size: 10
      fail_fast_on_connect: false   # default true; false = wait pool_timeout even on a bad password

    Default is true (zero-config: a misconfigured deploy fails immediately instead of hanging every request for 30 s). register_connection accepts the same fail_fast_on_connect kwarg. Transient recovery is not retried inside the pool — retry the whole operation at the application layer (the same rule as lost connections inside a transaction).

  • Idle reaping & max-lifetime (opt-in): By default the pool never shrinks after a burst and reuses connections indefinitely (a dropped connection is caught by the liveness check on the next checkout — see Dropped connections below). For long-lived services — or databases/proxies that drop idle connections — you can enable a background reaper via connection.yml (both in seconds, 0/absent = off):

    dev:
      adapter: PostgreSQL
      database: 'formula1'
      pool_size: 10
      idle_timeout: 60      # close *overflow* connections idle > 60s, trimming back toward pool_size
      max_lifetime: 1800    # retire connections older than 30 min (on return, and by the sweeper)

    Reaping is overflow-only and never drops below the base pool_size (those stay warm), and never closes an in-use connection. It closes the connection and clears its slot in place — the pool's slot layout is unchanged, so a reaped slot simply opens a fresh connection on next use. Disabled by default: unset means zero behavior change. Programmatic pools accept the same idle_timeout / max_lifetime kwargs via Configuration.register_connection. The sweeper starts at run time only: a load in a package's module body runs while the package precompiles and starts no sweeper. So these keys are safe in the shape Loading before @import_models is not enough for a package describes below, where the load in __init__ starts it. A sysimage or a juliac executable keeps no configuration from its build (see Built images read their configuration at run time), so its run-time load starts the sweeper the same way.

  • Dropped connections: Every checkout probes the connection before handing it out, and PostgreSQL's probe consumes any pending socket input before trusting the driver's status. That matters because PQstatus reports libpq's cached state: it only turns bad after an I/O attempt fails, and an idle pooled connection attempts none — so before this, a backend the server had terminated (a restart, a failover, an admin pg_terminate_backend) kept reporting a healthy connection while its FATAL: terminating connection due to administrator command sat unread in the socket buffer, and the pool went on serving it. A rejected connection is closed and its slot reopened. SQLite's probe already round-trips, so it never had the blind spot. MySQL's probe is local (isopen): a connection the server dropped silently fails its next statement as a lost connection and is renewed then. The statement runs again only when MySQL.jl reports it never ran (error 2006 or 4031), and a pool also pings a connection idle past validate_after_idle before using it (MySQL Guide).

    Whatever kills one pooled connection has usually killed all of them, so recovery is not per-slot either: when a failure is classified as a lost connection, fetch renews the connection that failed and retires every other idle one. In-use connections are left alone — their borrower still holds the handle, and will recover the same way. A failure counts as a lost connection when the server reports the backend gone, or when the driver reports the socket under the session failing: a reset, a timeout, an unreachable host, or a TLS socket closed without its goodbye, which a proxy or load balancer cutting idle connections can produce (SSL SYSCALL error: EOF detected).

    Which lost connections are retried. Renewing the pool is always safe; running the statement again is not. When the socket goes away after the server received the statement, an INSERT, UPDATE or DELETE may already have committed, and running it again would apply it twice. So fetch retries the statement once only when it provably never ran:

    FailureRenewed and sweptRetried
    The server reports the backend gone with an error code: 57P01 (an admin pg_terminate_backend, a fast shutdown), 57P02, 57P03, 57P05 (idle-session timeout), class 08 except 08007 / 08P01✅✅
    PostgreSQL: the driver refused to send because the connection was already closed✅✅
    MySQL: error 2006, the statement's write failed or the connection was already known dead✅✅
    MySQL: error 4031, the server's farewell to a connection idle past its wait_timeout, read after the statement was written to a server no longer reading✅✅
    MySQL: error 2013 (lost after the statement was written), 1927, or MySQL.jl's "connection has been closed" refusal✅— raises OperationalError
    The socket went away with no error code: server closed the connection unexpectedly, SSL SYSCALL error: EOF detected, a reset, a timeout, an unreachable host✅— raises OperationalError
    Any lost connection inside a transaction, or on a connection you passed as conn— the transaction or your code owns it—

    Both PostgreSQL drivers follow this rule, but they do not always see the same event in the same row. A backend terminated while running a statement reaches Postgres.jl as 57P01, which is retried. libpq reports the same event as the socket closing with no code, so on the LibPQ driver it raises. SQLite retries nothing: its one lost-connection signal, a disk I/O error, can come from a commit whose outcome is unknown. MySQL retries only errors 2006 and 4031. MySQL.jl 2.2 and newer raises 2006 when the statement was never sent: its write failed, or the connection was already known dead. 4031 is the farewell the server sends before it closes a connection idle past its wait_timeout, which the client reads after writing its next statement, when the server is no longer reading it. Error 2013, a connection lost after the statement was written, and every other lost connection raise, because the statement may have run (MySQL Guide). Most dead connections never reach a statement at all, because the checkout probe above retires them first. When an OperationalError does reach your code, retry only work you know is safe to repeat, such as a read or a write keyed so that a second run changes nothing.

    Reads are not retried either. The rule above looks at the failure, never at the statement, so a list, get, count or exists whose connection drops with no error code raises OperationalError like a write does. The SQL text cannot prove a statement is safe to run twice: a SELECT can call nextval() or a function with side effects, and a WITH query can insert rows. Django and SQLAlchemy draw the same line. Each checks a pooled connection before using it (Django's CONN_HEALTH_CHECKS, SQLAlchemy's pool_pre_ping, PormG's checkout probe above), and neither re-runs a statement after the connection is lost mid-query. Your code knows which reads are safe to repeat, so wrap those in with_read_retry, which runs the block again when it raises OperationalError:

    rows = with_read_retry() do                      # attempts = 1, backoff = 0 (seconds)
        M.Result.objects.filter("raceid" => 1120).list()
    end

    You vouch that the block is safe to run twice; PormG does not check. Only OperationalError is retried, except a cancelled statement, on any engine (on PostgreSQL SQLSTATE 57014: a statement_timeout, a pg_cancel_backend; on MySQL error 1317, a KILL QUERY, and 3024, max_execution_time), and each retry is logged as a warning. Inside a transaction with_read_retry raises TransactionError without running the block: retry the whole atomic block instead (see Error types). Library code that reads on an application's behalf, such as a session or task store, cannot know whether the application's own transaction is open around the call; it passes inside_transaction = :run_once, and inside a transaction the block then runs once, with no retry and no error, so a drop propagates and the application's atomic block rolls back as a whole. The default stays strict because application code that wraps a read inside its own transaction has misread what the helper can do.

    The remaining case is a connection dropped with no bytes delivered at all — a silently broken network path rather than a server that announced itself. Nothing can see that without a round trip, so it surfaces on first use as a socket failure: the pool recovers as above and the statement raises. Bound how long a connection can sit in that state with the max_lifetime reaper.

  • Idle validation (validate_after_idle, MySQL): MySQL's checkout probe is local: it sees a connection the driver already knows is closed, not one the server closed while it sat idle in the pool past its wait_timeout (8 hours by default, often less on a hosted server). So a MySQL pool pings a connection that sat idle longer than validate_after_idle seconds before handing it out (one COMPING round trip). If the ping fails, the pool opens a new connection in that slot before your statement is sent, so the statement runs on the new one and nothing raises. A failed ping counts as no statement failure, and nothing is retried. A busy connection, one returned to the pool less than `validateafteridleseconds ago, is handed out without a ping. The default is 60 seconds; keep it below the server'swaittimeout, or set0` to turn the ping off:

    dev:
      adapter: MySQL
      database: 'formula1'
      pool_size: 10
      validate_after_idle: 30   # ping a connection idle > 30s before handing it out (default 60; 0 = off)

    The ping runs without holding the pool's lock: other callers borrow and return connections while it waits, and the connection being pinged counts as in_use in pool_stats. A Ctrl-C during the ping closes that connection rather than returning it. The ping is bounded by what is left of the checkout's timeout_seconds (or pool_timeout), and never given less than one second: on a network path that drops packets without a reset (a firewall or NAT that forgot the connection), a ping with no answer by then gives up, and the pool opens a new connection in that slot as for any failed ping, instead of waiting minutes for the kernel's TCP retransmissions (#1618). The new connection is bounded by connect_timeout, which is unset by default: on a path that drops new connections too, set it, or the connect waits for the kernel's own connect timeout. If the server refuses the connection, or cannot be reached within connect_timeout, the new connection fails too: the checkout then retries the connect until its deadline and raises PoolConnectError, where without the ping the statement raised OperationalError at once.

    PostgreSQL and SQLite have no such setting, so a validate_after_idle above 0 there raises BackendCapabilityError (idle_validation) when the configuration loads. Their checkout probe already sees a connection the server closed: PostgreSQL's reads what the server sent, SQLite's runs SELECT 1 (see Dropped connections above). Neither sends a round trip, so a path that dropped silently still surfaces at the first statement there.

  • Health snapshot (pool_stats): pool_stats (exported) returns a NamedTuple for debugging saturation — pass a pool object or a connection key/path:

    using PormG
    pool_stats("db")   # => (; pool_size, size, in_use, available, ceiling, waiting)

    pool_size is the configured floor, size the slots allocated so far (== in_use + available), ceiling the maximum (pool_size × 10), and waiting the callers currently parked for a connection — a non-zero waiting with in_use == ceiling is the signature of saturation (see PoolTimeoutError). Counts are read under the pool lock for a coherent snapshot.

  • Leak detection (leak_detection_threshold, opt-in): A connection acquired but never released (e.g. a fetch_async that's never awaited) is silently lost until the pool starves. Set leak_detection_threshold (seconds, 0/absent = off) to have acquire_connection emit a single @warn — naming the slot and hold time — when a connection has been held past the threshold:

    dev:
      adapter: PostgreSQL
      database: 'formula1'
      pool_size: 10
      leak_detection_threshold: 30   # warn when a connection is held > 30s without release

    The scan runs on the next acquire_connection (no background task), so a leak is flagged as the pool comes under pressure — pointing at the offending slot just before a PoolTimeoutError. Off by default; register_connection accepts the same leak_detection_threshold kwarg.

    The warning says what the slot is doing when the scan finds it (#1637). Every wording starts with Pool connection held past leak_detection_threshold, so a log search for that finds all three:

    the slot at the scanthe warning
    leased, no statement running… — likely acquired without release; await the FetchTask or release the connection. This is the leak the setting is for.
    leased, its statement still running… while its statement is still running: a slow statement or an unresponsive server, not a leak, with running_s (how long the statement has run) beside held_s
    reserved for a connect… while connecting to the server. The same wording covers a slot whose idle connection is being pinged before it is handed out (validate_after_idle), since a failed ping is followed by a connect, and a slot handed to a waiting caller that will connect into it

    A reported lease that comes back to the pool logs an @info, Pool connection reported past leak_detection_threshold was returned to the pool, with the same slot, held_s (counted, like the warning's, from the checkout, or from the reservation for a slot reported while connecting) and threshold_s (#1643). Every way a lease hands its slot back logs it: a plain return, a connection renewed after a failed ROLLBACK and then returned, an overflow connection retired on return (max_lifetime), a slot reported while connecting whose caller later returns the connection, and a reservation given back because its connect failed or the acquire was interrupted. So a warning with no such line after it for its slot is the leak. Two endings log no such line, because the connection did not come back: a connection discarded after a failure, and the connections close_pool! closes.

    A statement still running past the threshold is reported but is not a leak: the statement is slow, or the server is paused, half-open or failing over, and the connection comes back when it ends. It is still a @warn, once per lease like the others, because a statement that runs that long is worth seeing. Once per lease means that if the holder of that same lease then never returns it, no second warning names the leak; the threshold was chosen for leaks, so set it above your slowest expected statement if the slow-statement warnings are noise.

    To tell the states apart, PormG records when each statement starts and clears the record when it ends, only on a pool with leak detection on: two timestamps and a few short lock acquisitions per statement, one of them the pool's own lock (on the LibPQ driver also a small task per statement that clears the record once the statement ends). A pool without leak detection pays one flag check per statement while no pool in the process has it on, and a lookup in the pool registry otherwise.

  • Thread Safety: PormG uses ReentrantLock for pool management.

  • Failed-rollback self-healing: If a transaction's ROLLBACK itself fails (e.g. the connection died mid-transaction), the pool never returns that connection as-is. It is renewed in its slot (PostgreSQL: LibPQ.reset!; MySQL: a new connection, since MySQL.jl has no reset; SQLite: a fresh handle, with the old one closed so it releases the database file write-lock) or — if renewal also fails — closed and its slot cleared so the next borrower opens a fresh connection.

  • Process exit: PormG closes every registered pool from an atexit hook, with close_pool!(…; drain_seconds = 0): at exit no borrower is coming back, so it does not wait. A connection still checked out at that moment is closed under its borrower, whose next statement raises, and the hook logs one @warn naming the in_use count.

    • A transaction still open at exit is rolled back by the server, when it sees the connection close; nothing it wrote is committed. PostgreSQL may log it as unexpected EOF on client connection with an open transaction, always on the Postgres.jl driver, which closes its socket without the protocol's Terminate message.
    • The hook does not roll back for you. On Julia 1.13 the task that owns a transaction is still running when the hook runs, so a ROLLBACK sent from the hook could land between two of that task's statements, and the next one would commit on its own. Closing is the safe end. For an orderly one, stop the workload before the process exits (a web server's drain, or waiting on your own tasks), so every run_in_transaction block commits or rolls back itself.
    • A SIGTERM exit can hang on Julia 1.13 while statements are in flight, most often on the LibPQ driver: the exiting thread waits in whichever atexit hook first waits on another task. The cause is in Julia's termination path (JuliaLang/julia#63988), not in PormG; removing PormG's hook moves the hang to the next hook. JuliaLang/julia#64425 removes it in every case measured. Until a Julia release carries that change, have the process supervisor bound its wait and send SIGKILL after a grace period (a Kubernetes terminationGracePeriodSeconds, systemd's TimeoutStopSec).

Credentials never leave through show or JSON.json

A connection pool holds the connection string it was built from, and for PostgreSQL and MySQL that string is the credential. Configuration.Settings holds the pool, and PormG.config holds the settings — so any of the three is one JSON.json or one REPL show away from a password, in exactly the places application state gets serialized without much thought: a debug endpoint, a health check that dumps config, an error reporter attaching context.

PormG bounds all three. The two routes deliberately promise different things:

what it emits
JSON.json(pool){"pormg_connection":"PostgreSQL"} — no connection string, in any form
JSON.json(settings)a pormg_settings object: app_env, db_def_folder, model_file, time_zone, django_prefix, change_db, change_data, implicit, and the backend name. Never the pool's DSN, never db_config_settings
JSON.json(PormG.config)the same, once per configured key
show(pool) — nested, e.g. inside a VectorPool(PostgreSQL, 10 slots) — no connection string
the REPL card, when you type the valuethe above plus the connection string, redacted
julia> pool = PormG.config["db"].connections
Pool(PostgreSQL, 10 slots)
  dsn: host='127.0.0.1' port='5432' password=**** dbname='formula1' user=****

Why JSON is stricter than the display. A card is read by a human who asked for it, at a terminal, and "which database is this pointed at?" is the only reason to type a pool — so the card answers it, redacted. A JSON document travels: to a debug endpoint, a log aggregator, an error reporter, a third party. Redaction is a denylist, and a denylist is one unfamiliar DSN dialect away from emitting a password, which is an acceptable risk at a terminal and not on a wire.

If you do want the connection string in a document, ask for it explicitly:

PormG.Configuration.redact_secret(pool.connection_string)

redact_secret is the single owner of the redaction rule and the same one every log line and exception uses. It covers both dialects PormG accepts — the libpq keyword form (password=…, user=…, quoted values included) and the URL form (postgres://user:password@host/db → postgres://****:****@host/db).

db_config_settings is never emitted at all, by either route. It is the raw parsed YAML block for your environment, so it can hold a password: key, a url: carrying an entire DSN, or paths to TLS private keys — and being an arbitrary user-supplied dictionary, there is nothing to redact it with. Read it directly if you need it; PormG will not put it in a document for you.


Advisory Locks

Use advisory locks to ensure long-running tasks (migrations, seeds, imports) do not run in parallel across processes.

using PormG, LibPQ   # advisory locks need PostgreSQL or MySQL

# Wrap multiple operations in an advisory lock
PormG.run_in_transaction("db") do
    with_advisory_lock(settings, "my_job_name") do
        # Long-running task
        M.Result.objects.create("year" => 2025, "name" => "New Race")
        bulk_insert(M.Result.objects, results_df)
    end
end

Key Technical Details

  • Hashing: Keys are hashed using MD5 to provide a 64-bit bigint identifier on PostgreSQL, and a 38-character lock name (pormg: and an MD5, the database name hashed in) on MySQL.
  • Cleanup: PostgreSQL and MySQL release the lock automatically if the session drops.
  • SQLite Limitation: SQLite does not support advisory locks; the helper is a no-op on that backend — the body runs unprotected and the wait/timeout keyword arguments are ignored. It warns once per lock key; pass on_missing_lock = :ignore to accept the no-op silently, or on_missing_lock = :error to raise BackendCapabilityError instead of running unprotected. See Advisory Locks.

The Boot-Time Hazard

@import_models eventually calls Models.set_models(...), and a package's __init__ calls it directly (the run-time bind). Either way, it binds every model to a connect_key by matching its models folder against the configurations already loaded. Everything below is about that match: what it does when it finds nothing, and what it does when it finds more than one thing.

When nothing matches, PormG refuses

If no loaded configuration matches the models folder, set_models (and so @import_models and @models_module) raises InvalidConfigurationError. The message names the folder, the configurations that are loaded, and the call to add:

PormG.Configuration.load("db"; root = APP_ROOT, env = "prod")   # before @import_models

Before #1226, PormG loaded the folder itself and logged a @warn. That load decided two things you did not state:

  • The environment. ENV["PORMG_ENV"] when it is set, else the file's default_env:. In a server that has not selected its environment yet, that is usually dev, which in many deployments points at production. Since #1228, that fallback past PORMG_ENV is a REPL's only, for every load (see Environment Selection).
  • The key. The folder's path, not a name your application chose.

The warning went to a boot log nobody reads, and the queries ran. Now the registration fails instead, so the wrong-database case cannot happen silently.

The same rule holds on the first Model.objects of a restored package image. The self-heal there (ensure_model_initialized) binds the model to a configuration that is already loaded, or raises the same error. It never loads one. The __init__ an @models_module block defines binds only when the configuration is already loaded, and otherwise does nothing. It runs before your package's own __init__, so refusing there would make using fail. @import_models adds no __init__ (#1607): the first query rebinds from the key it recorded.

The REPL keeps the convenience. In an interactive session (isinteractive()), a direct @import_models or set_models with nothing loaded loads the folder anyway, with the old warning (#1248). A service, a test or a script is not interactive and gets the refusal; it can still ask for the guess one call at a time:

PormG.Models.set_models(MyModels, "db"; implicit = true)

A test or a script that only renders SQL and never runs a query needs neither: register the key with Configuration.register_offline("db"; adapter = :postgres) before importing the models. It binds them to a connection that renders SQL and never connects. A load or implicit = true there would open a real pool. The recipe is on Testing Your App.

There is no process-wide switch: whether a registration refuses never depends on what some other code in the process turned on earlier. The self-heal and the @models_module __init__ above never load, not even in a REPL — using SomePackage at a REPL is an interactive session too.

An implicit load marks the entry as implicit: PormG.Configuration.status(key).implicit reads it back. The rules below go by that mark, not by the shape of the key.

When several configurations match, the explicit key wins

Applications normally load configuration by short key (Configuration.load("db")) and import models by relative path (@import_models "../db/models.jl"). Those two agree on the resolved absolute path only when the working directory lines up, so PormG also matches on the folder's final component.

Matches are ranked — an exact path beats a folder-name match. Within a rank, a key you loaded explicitly beats one set_models minted implicitly, because the implicit entry is the one whose environment came from PORMG_ENV/default_env: rather than from your application. PormG records that on the entry rather than inferring it from the key's spelling, so a relative implicit key (set_models(mod, "db")) still loses to an absolute key you loaded yourself.

Only when that still leaves several candidates is the binding genuinely ambiguous — two configured folders that end in the same name (db and vendor_app/db), say. PormG then warns, lists every candidate, and picks the lexicographically first so that at least the same configuration resolves the same way on every boot. Give the folders distinct final components and the ambiguity disappears.

One boot for development and production

A package binds its models at run time: the module body includes the models file, and __init__ loads the configuration and calls set_models(models, "db"; source = MODELS_FILE). The full snippet is under The Bootstrap Sequence. Precompiling then reads no connection.yml, so the image holds no connection detail, and every process binds in its own session. A Revise reload is the only difference between development and production, and it is decided at run time: with Revise loaded, source registers it; without Revise, source does nothing (#1487).

Using @import_models in development and the run-time bind in production looks tempting, but it does not work:

  • @import_models in a package body runs at precompile time, in development too. The precompile worker is never interactive, so the folder has to be loaded right there in the module body (next section). The reload it registers is skipped there, and the restored session never runs that code again. So in a package, @import_models gives no reload even in development.
  • A module-body choice is frozen into the cache. An if on the environment in the module body runs once, in the precompile worker. Changing RACECONTROL_ENV or PORMG_ENV later does not invalidate the cache, so the package silently keeps the other shape.
  • Tests and services run the production shape. They are not interactive, so a development-only shape is a path that no test exercises. #1226 surfaced this way: the documented boot broke only outside a REPL.

Django draws the same line. models.py is the same file in development and production, importing it never reads DATABASES, and runserver's autoreload is a choice made at run time.

Loading before @import_models is not enough for a package

This section is for a package that keeps @import_models in its module body. The boot above avoids the problem it describes.

Module-body configuration does not survive precompilation

The obvious fix — call Configuration.load(path; env = ...) above @import_models — is correct but incomplete for a precompiled package. Module-body code runs at precompile time, and PormG's config is a global inside PormG, so those entries do not survive into the session. At runtime config starts empty again, and the first Model.objects access raises InvalidConfigurationError because nothing has loaded the folder (see above).

Load the configuration in both places: in the module body, so the right connect_key is baked into the image, and again from your module's __init__, so it is right in the running session. Route both through one helper so they cannot drift apart. Repeating it costs nothing: a load of a folder already loaded with the same environment keeps the existing pool. The module-body load opens no connection either: pools connect on first use, and the extensions: probe and the connection sweeper wait for the run-time load, so the machine that precompiles the package does not need the database.

module RaceControl

using PormG

const DB_DIRS  = ["db", "db_telemetry"]
const APP_ROOT = normpath(joinpath(@__DIR__, ".."))

# `root` keeps the keys short ("db", "db_telemetry") while the folders resolve under APP_ROOT, so
# the precompile worker, `__init__` and the server all register the same keys, whatever their `pwd()`.
_load_configs() =
    PormG.Configuration.load_many(DB_DIRS; root = APP_ROOT, env = get(ENV, "RACECONTROL_ENV", "dev"))

_load_configs()                       # precompile: lets @import_models bind, and record the key
PormG.@import_models "../db/models.jl" models

__init__() = _load_configs()          # runtime: the image's configuration did not survive

end

Each half does a different job, which is why dropping either one breaks something. The first Model.objects access runs the self-heal (ensure_model_initialized), so the state right after using and the state your queries actually run against are two different columns:

Where the configuration is loadedAfter using, before any .objectsAfter the first .objects access
Module body and __init__short key, config populated ✔unchanged ✔
Module body onlyshort key, config emptyInvalidConfigurationError: the key the models were imported under is not loaded, and the message names the load(key; root, env) call to add
__init__ onlythe precompile run fails: @import_models in the module body raises InvalidConfigurationError—
Neitheras "__init__ only"—

The module body runs at precompile time, so it is what lets @import_models bind the models at all; __init__ runs in the session, so it is what puts the connection in config where a query can find it. Neither substitutes for the other.

What the first .objects binds to (#1227). When @import_models binds the models it records, in the models module's model set (its MODEL_SET, #1607), the key the folder was loaded under and the root it was loaded relative to. That record is part of the image. On the first .objects of a restored session the self-heal reads it back and binds to that key — the one your __init__ loads — or refuses, naming load(key; root, env). It does not look through config for some other configuration that happens to match the folder, so a session that loaded the same folder under a different key, or from a different working directory, cannot quietly change which entry the models use. The module-body load is therefore what makes the import succeed and fixes the key; the __init__ load is the one the running session depends on. "Module body only" is the row to watch: it looks right after using and fails on the first query. Since #1226 it fails with an error that says what is missing. Before, it quietly ran against a database you did not choose.

Verify the binding instead of assuming it

connect_key is the field that decides which database a model's queries reach, so read it back rather than inferring it from the configuration you meant to load:

julia> using RaceControl
julia> RaceControl.models.Driver.objects       # the access that re-derives the key
julia> RaceControl.models.Driver.connect_key   # want the short key, not an absolute path
"db"

The implicit load runs only in a REPL or when a call passed implicit = true. When it did, PormG.Configuration.status(RaceControl.models.Driver.connect_key).implicit is true, and the environment came from PORMG_ENV/default_env:, not from whatever your application selected.