Advanced Configuration
PormG is designed for Julia's asynchronous task scheduling. This section covers connection pooling, advisory locks, and the performance implications of the async-first design.
Connection Pooling & Async-First Design
- Async-First API: The synchronous
fetch()API is a wrapper around the asynchronousfetch_async()core. This wrapper yields to the Julia scheduler while waiting for the database, ensuring compatibility with async frameworks. See the Async & Concurrency guide for usage patterns. - Pooling Strategy: The default strategy is
:poll, which retries at configured intervals. Use:blockon PostgreSQL or MySQL to let the server-side manage the wait, bounded bytimeout_msthrough PostgreSQL'slock_timeoutorGET_LOCK's own timeout (whole seconds, rounded up). - Sizing & capacity: A pool starts at
pool_sizeconnections (default3) and grows lazily, on demand, up topool_size × 10under concurrent load — so the idle footprint stays small while async fan-out still gets headroom. If every connection is busy and none frees within the retry/timeout budget,acquire_connectionraises a catchablePoolTimeoutError(exported by PormG). Raisepool_sizein yourconnection.ymlto add capacity for genuinely high-concurrency workloads:
dev:
adapter: PostgreSQL
database: 'formula1'
# ...
pool_size: 10 # base 10 → grows to 100 under burstA misspelled pool key (pool:, poolsize:, pool_timout:) is reported on load with a suggestion instead of silently leaving the default in place — see Unrecognised keys.
Acquire timeout (
pool_timeout): How longacquire_connectionwaits for a free connection before raisingPoolTimeoutError— default 30 s. Setpool_timeout:inconnection.yml(seconds; fractional allowed) to fail fast instead of blocking a request while the pool is saturated:dev: adapter: PostgreSQL database: 'formula1' pool_size: 10 pool_timeout: 5 # give up after 5s waiting for a connection, then raise PoolTimeoutErrorAn explicit per-call
acquire_connection(pool; timeout_seconds=…)still overrides it, andConfiguration.register_connectionaccepts the samepool_timeoutkwarg. A value≤ 0falls back to the 30 s default (a "never wait" setting is ambiguous and a footgun). Absent means the historical 30 s — zero behavior change.Connect failures (
PoolConnectError,fail_fast_on_connect):pool_timeoutcovers a saturated healthy pool. A pool that can never open a connection — a wrong password, a missing role/database, an unopenable SQLite path — is a different failure: waitingpool_timeoutfor it to "free up" is pointless, and blamingpool_sizeis misleading.acquire_connectiontherefore classifies the driver error: a permanent one (PostgreSQL auth / missing role or database; MySQL access denied, unknown database, a host not allowed, an authentication plugin either side lacks, or a server older than 8.4; SQLiteunable to open database file) fails fast with a catchablePoolConnectError(exported) that carries the underlying driver cause and a redacted connection string — remedy: fix credentials/host/database, notpool_size. Ambiguous errors (host/DNS/network, a refused connect while the server restarts — possibly a transient blip) are not fast-failed. The caller retries the connect, one attempt at a time, after a backoff of 50 ms doubling to 0.5 s, so it gets a connection within about half a second of the server coming back; if a connection is released meanwhile, the caller takes it at once. During an outage each waiting caller costs the server about five connect attempts in its first second, then two a second. A connect runs without holding the pool's lock (#1540): the caller reserves a slot, connects, then installs the connection. While one caller waits on a slow or unreachable server, every other caller can still return a connection, borrow an idle one, or give up at its ownpool_timeout. Callers connect side by side, so during an outage the server sees up to one attempt per waiting caller at a time, not one for the whole pool. The reserved slot counts asin_useinpool_stats, and leak detection times it from the reservation, so a connect slower thanleak_detection_thresholdis reported, worded as a connect (see Leak detection below). If the deadline passes first, it raisesPoolConnectError(with the last cause) rather thanPoolTimeoutError. With the default PostgreSQL driver (LibPQ), recognizing a permanent failure means reading the server's message, because libpq gives a failed connect no SQLSTATE. PostgreSQL translates that message by itslc_messagessetting, so on a server set to another language (pt_BR, say) a wrong password is not recognized and waits outpool_timeoutlike an ambiguous error. Setlc_messages = 'C'inpostgresql.confto keep the fast-fail; a per-role or per-database setting does not apply before authentication. ThePostgres.jldriver reads the SQLSTATE and is unaffected. A PostgreSQL connection string that cannot even be parsed — typically a hand-writtenurl:orregister_connectionstring with an unquoted space inside a value, a malformed%-escape, or a NUL byte — is permanent too: it fails fast with aPoolConnectErrorwhosecauseis anInvalidConfigurationError. PormG checks the string before handing it to the driver, so the part of libpq's message that would quote your password is masked and nothing is logged by the driver itself; quote a value containing spaces (password='two words') or percent-encode it in a URL. A URL password holding an unencoded@is refused the same way (libpq would end the password at the first@and try the rest as a host name or port) — write it as%40, and a:as%3A. An unencoded/is refused too — libpq ends the credentials at the first/and reads the rest of the password as the database name — so write it as%2F. The check behind that refusal is "the URL's database name contains an@", so a database whose name really contains an@cannot be reached through a URL (not even as%40, which libpq decodes first); give that one in the keyword form,dbname='…'. When the password's tail also holds a?, libpq reads the rest as a query option instead, so in a URL onlyuser,password,sslpassword,application_nameandfallback_application_namemay carry an@(?user=me@serverstays accepted), and even those are refused when an unencoded@is followed by what looks like a host, port or path (?user=@db1/f1). Write a?in a password as%3F, percent-encode the@of an allowed value that trips that check, and give any other option whose value really contains an@(a certificate path, say) in the keyword form. Setfail_fast_on_connect: falseto opt out of the fast-fail and keep waiting the fullpool_timeout:dev: adapter: PostgreSQL database: 'formula1' pool_size: 10 fail_fast_on_connect: false # default true; false = wait pool_timeout even on a bad passwordDefault is
true(zero-config: a misconfigured deploy fails immediately instead of hanging every request for 30 s).register_connectionaccepts the samefail_fast_on_connectkwarg. Transient recovery is not retried inside the pool — retry the whole operation at the application layer (the same rule as lost connections inside a transaction).Idle reaping & max-lifetime (opt-in): By default the pool never shrinks after a burst and reuses connections indefinitely (a dropped connection is caught by the liveness check on the next checkout — see Dropped connections below). For long-lived services — or databases/proxies that drop idle connections — you can enable a background reaper via
connection.yml(both in seconds,0/absent = off):dev: adapter: PostgreSQL database: 'formula1' pool_size: 10 idle_timeout: 60 # close *overflow* connections idle > 60s, trimming back toward pool_size max_lifetime: 1800 # retire connections older than 30 min (on return, and by the sweeper)Reaping is overflow-only and never drops below the base
pool_size(those stay warm), and never closes an in-use connection. It closes the connection and clears its slot in place — the pool's slot layout is unchanged, so a reaped slot simply opens a fresh connection on next use. Disabled by default: unset means zero behavior change. Programmatic pools accept the sameidle_timeout/max_lifetimekwargs viaConfiguration.register_connection. The sweeper starts at run time only: a load in a package's module body runs while the package precompiles and starts no sweeper. So these keys are safe in the shape Loading before@import_modelsis not enough for a package describes below, where theloadin__init__starts it. A sysimage or ajuliacexecutable keeps no configuration from its build (see Built images read their configuration at run time), so its run-timeloadstarts the sweeper the same way.Dropped connections: Every checkout probes the connection before handing it out, and PostgreSQL's probe consumes any pending socket input before trusting the driver's status. That matters because
PQstatusreports libpq's cached state: it only turns bad after an I/O attempt fails, and an idle pooled connection attempts none — so before this, a backend the server had terminated (a restart, a failover, an adminpg_terminate_backend) kept reporting a healthy connection while itsFATAL: terminating connection due to administrator commandsat unread in the socket buffer, and the pool went on serving it. A rejected connection is closed and its slot reopened. SQLite's probe already round-trips, so it never had the blind spot. MySQL's probe is local (isopen): a connection the server dropped silently fails its next statement as a lost connection and is renewed then. The statement runs again only when MySQL.jl reports it never ran (error 2006 or 4031), and a pool also pings a connection idle pastvalidate_after_idlebefore using it (MySQL Guide).Whatever kills one pooled connection has usually killed all of them, so recovery is not per-slot either: when a failure is classified as a lost connection,
fetchrenews the connection that failed and retires every other idle one. In-use connections are left alone — their borrower still holds the handle, and will recover the same way. A failure counts as a lost connection when the server reports the backend gone, or when the driver reports the socket under the session failing: a reset, a timeout, an unreachable host, or a TLS socket closed without its goodbye, which a proxy or load balancer cutting idle connections can produce (SSL SYSCALL error: EOF detected).Which lost connections are retried. Renewing the pool is always safe; running the statement again is not. When the socket goes away after the server received the statement, an
INSERT,UPDATEorDELETEmay already have committed, and running it again would apply it twice. Sofetchretries the statement once only when it provably never ran:Failure Renewed and swept Retried The server reports the backend gone with an error code: 57P01(an adminpg_terminate_backend, a fast shutdown),57P02,57P03,57P05(idle-session timeout), class08except08007/08P01✅ ✅ PostgreSQL: the driver refused to send because the connection was already closed ✅ ✅ MySQL: error 2006, the statement's write failed or the connection was already known dead ✅ ✅ MySQL: error 4031, the server's farewell to a connection idle past its wait_timeout, read after the statement was written to a server no longer reading✅ ✅ MySQL: error 2013 (lost after the statement was written), 1927, or MySQL.jl's "connection has been closed" refusal ✅ — raises OperationalErrorThe socket went away with no error code: server closed the connection unexpectedly,SSL SYSCALL error: EOF detected, a reset, a timeout, an unreachable host✅ — raises OperationalErrorAny lost connection inside a transaction, or on a connection you passed as conn— the transaction or your code owns it — Both PostgreSQL drivers follow this rule, but they do not always see the same event in the same row. A backend terminated while running a statement reaches Postgres.jl as
57P01, which is retried. libpq reports the same event as the socket closing with no code, so on the LibPQ driver it raises. SQLite retries nothing: its one lost-connection signal, a disk I/O error, can come from a commit whose outcome is unknown. MySQL retries only errors 2006 and 4031. MySQL.jl 2.2 and newer raises 2006 when the statement was never sent: its write failed, or the connection was already known dead. 4031 is the farewell the server sends before it closes a connection idle past itswait_timeout, which the client reads after writing its next statement, when the server is no longer reading it. Error 2013, a connection lost after the statement was written, and every other lost connection raise, because the statement may have run (MySQL Guide). Most dead connections never reach a statement at all, because the checkout probe above retires them first. When anOperationalErrordoes reach your code, retry only work you know is safe to repeat, such as a read or a write keyed so that a second run changes nothing.Reads are not retried either. The rule above looks at the failure, never at the statement, so a
list,get,countorexistswhose connection drops with no error code raisesOperationalErrorlike a write does. The SQL text cannot prove a statement is safe to run twice: aSELECTcan callnextval()or a function with side effects, and aWITHquery can insert rows. Django and SQLAlchemy draw the same line. Each checks a pooled connection before using it (Django'sCONN_HEALTH_CHECKS, SQLAlchemy'spool_pre_ping, PormG's checkout probe above), and neither re-runs a statement after the connection is lost mid-query. Your code knows which reads are safe to repeat, so wrap those inwith_read_retry, which runs the block again when it raisesOperationalError:rows = with_read_retry() do # attempts = 1, backoff = 0 (seconds) M.Result.objects.filter("raceid" => 1120).list() endYou vouch that the block is safe to run twice; PormG does not check. Only
OperationalErroris retried, except a cancelled statement, on any engine (on PostgreSQL SQLSTATE57014: astatement_timeout, apg_cancel_backend; on MySQL error 1317, aKILL QUERY, and 3024,max_execution_time), and each retry is logged as a warning. Inside a transactionwith_read_retryraisesTransactionErrorwithout running the block: retry the wholeatomicblock instead (see Error types). Library code that reads on an application's behalf, such as a session or task store, cannot know whether the application's own transaction is open around the call; it passesinside_transaction = :run_once, and inside a transaction the block then runs once, with no retry and no error, so a drop propagates and the application'satomicblock rolls back as a whole. The default stays strict because application code that wraps a read inside its own transaction has misread what the helper can do.The remaining case is a connection dropped with no bytes delivered at all — a silently broken network path rather than a server that announced itself. Nothing can see that without a round trip, so it surfaces on first use as a socket failure: the pool recovers as above and the statement raises. Bound how long a connection can sit in that state with the
max_lifetimereaper.Idle validation (
validate_after_idle, MySQL): MySQL's checkout probe is local: it sees a connection the driver already knows is closed, not one the server closed while it sat idle in the pool past itswait_timeout(8 hours by default, often less on a hosted server). So a MySQL pool pings a connection that sat idle longer thanvalidate_after_idleseconds before handing it out (one COMPING round trip). If the ping fails, the pool opens a new connection in that slot before your statement is sent, so the statement runs on the new one and nothing raises. A failed ping counts as no statement failure, and nothing is retried. A busy connection, one returned to the pool less than `validateafteridleseconds ago, is handed out without a ping. The default is 60 seconds; keep it below the server'swaittimeout, or set0` to turn the ping off:dev: adapter: MySQL database: 'formula1' pool_size: 10 validate_after_idle: 30 # ping a connection idle > 30s before handing it out (default 60; 0 = off)The ping runs without holding the pool's lock: other callers borrow and return connections while it waits, and the connection being pinged counts as
in_useinpool_stats. A Ctrl-C during the ping closes that connection rather than returning it. The ping is bounded by what is left of the checkout'stimeout_seconds(orpool_timeout), and never given less than one second: on a network path that drops packets without a reset (a firewall or NAT that forgot the connection), a ping with no answer by then gives up, and the pool opens a new connection in that slot as for any failed ping, instead of waiting minutes for the kernel's TCP retransmissions (#1618). The new connection is bounded byconnect_timeout, which is unset by default: on a path that drops new connections too, set it, or the connect waits for the kernel's own connect timeout. If the server refuses the connection, or cannot be reached withinconnect_timeout, the new connection fails too: the checkout then retries the connect until its deadline and raisesPoolConnectError, where without the ping the statement raisedOperationalErrorat once.PostgreSQL and SQLite have no such setting, so a
validate_after_idleabove0there raisesBackendCapabilityError(idle_validation) when the configuration loads. Their checkout probe already sees a connection the server closed: PostgreSQL's reads what the server sent, SQLite's runsSELECT 1(see Dropped connections above). Neither sends a round trip, so a path that dropped silently still surfaces at the first statement there.Health snapshot (
pool_stats):pool_stats(exported) returns aNamedTuplefor debugging saturation — pass a pool object or a connection key/path:using PormG pool_stats("db") # => (; pool_size, size, in_use, available, ceiling, waiting)pool_sizeis the configured floor,sizethe slots allocated so far (== in_use + available),ceilingthe maximum (pool_size × 10), andwaitingthe callers currently parked for a connection — a non-zerowaitingwithin_use == ceilingis the signature of saturation (seePoolTimeoutError). Counts are read under the pool lock for a coherent snapshot.Leak detection (
leak_detection_threshold, opt-in): A connection acquired but never released (e.g. afetch_asyncthat's never awaited) is silently lost until the pool starves. Setleak_detection_threshold(seconds,0/absent = off) to haveacquire_connectionemit a single@warn— naming the slot and hold time — when a connection has been held past the threshold:dev: adapter: PostgreSQL database: 'formula1' pool_size: 10 leak_detection_threshold: 30 # warn when a connection is held > 30s without releaseThe scan runs on the next
acquire_connection(no background task), so a leak is flagged as the pool comes under pressure — pointing at the offending slot just before aPoolTimeoutError. Off by default;register_connectionaccepts the sameleak_detection_thresholdkwarg.The warning says what the slot is doing when the scan finds it (#1637). Every wording starts with
Pool connection held past leak_detection_threshold, so a log search for that finds all three:the slot at the scan the warning leased, no statement running … — likely acquired without release; await the FetchTask or release the connection. This is the leak the setting is for.leased, its statement still running … while its statement is still running: a slow statement or an unresponsive server, not a leak, withrunning_s(how long the statement has run) besideheld_sreserved for a connect … while connecting to the server. The same wording covers a slot whose idle connection is being pinged before it is handed out (validate_after_idle), since a failed ping is followed by a connect, and a slot handed to a waiting caller that will connect into itA reported lease that comes back to the pool logs an
@info,Pool connection reported past leak_detection_threshold was returned to the pool, with the sameslot,held_s(counted, like the warning's, from the checkout, or from the reservation for a slot reported while connecting) andthreshold_s(#1643). Every way a lease hands its slot back logs it: a plain return, a connection renewed after a failedROLLBACKand then returned, an overflow connection retired on return (max_lifetime), a slot reported while connecting whose caller later returns the connection, and a reservation given back because its connect failed or the acquire was interrupted. So a warning with no such line after it for its slot is the leak. Two endings log no such line, because the connection did not come back: a connection discarded after a failure, and the connectionsclose_pool!closes.A statement still running past the threshold is reported but is not a leak: the statement is slow, or the server is paused, half-open or failing over, and the connection comes back when it ends. It is still a
@warn, once per lease like the others, because a statement that runs that long is worth seeing. Once per lease means that if the holder of that same lease then never returns it, no second warning names the leak; the threshold was chosen for leaks, so set it above your slowest expected statement if the slow-statement warnings are noise.To tell the states apart, PormG records when each statement starts and clears the record when it ends, only on a pool with leak detection on: two timestamps and a few short lock acquisitions per statement, one of them the pool's own lock (on the LibPQ driver also a small task per statement that clears the record once the statement ends). A pool without leak detection pays one flag check per statement while no pool in the process has it on, and a lookup in the pool registry otherwise.
Thread Safety: PormG uses
ReentrantLockfor pool management.Failed-rollback self-healing: If a transaction's
ROLLBACKitself fails (e.g. the connection died mid-transaction), the pool never returns that connection as-is. It is renewed in its slot (PostgreSQL:LibPQ.reset!; MySQL: a new connection, since MySQL.jl has no reset; SQLite: a fresh handle, with the old one closed so it releases the database file write-lock) or — if renewal also fails — closed and its slot cleared so the next borrower opens a fresh connection.Process exit: PormG closes every registered pool from an
atexithook, withclose_pool!(…; drain_seconds = 0): at exit no borrower is coming back, so it does not wait. A connection still checked out at that moment is closed under its borrower, whose next statement raises, and the hook logs one@warnnaming thein_usecount.- A transaction still open at exit is rolled back by the server, when it sees the connection close; nothing it wrote is committed. PostgreSQL may log it as
unexpected EOF on client connection with an open transaction, always on the Postgres.jl driver, which closes its socket without the protocol's Terminate message. - The hook does not roll back for you. On Julia 1.13 the task that owns a transaction is still running when the hook runs, so a
ROLLBACKsent from the hook could land between two of that task's statements, and the next one would commit on its own. Closing is the safe end. For an orderly one, stop the workload before the process exits (a web server's drain, or waiting on your own tasks), so everyrun_in_transactionblock commits or rolls back itself. - A SIGTERM exit can hang on Julia 1.13 while statements are in flight, most often on the LibPQ driver: the exiting thread waits in whichever
atexithook first waits on another task. The cause is in Julia's termination path (JuliaLang/julia#63988), not in PormG; removing PormG's hook moves the hang to the next hook. JuliaLang/julia#64425 removes it in every case measured. Until a Julia release carries that change, have the process supervisor bound its wait and send SIGKILL after a grace period (a KubernetesterminationGracePeriodSeconds, systemd'sTimeoutStopSec).
- A transaction still open at exit is rolled back by the server, when it sees the connection close; nothing it wrote is committed. PostgreSQL may log it as
Credentials never leave through show or JSON.json
A connection pool holds the connection string it was built from, and for PostgreSQL and MySQL that string is the credential. Configuration.Settings holds the pool, and PormG.config holds the settings — so any of the three is one JSON.json or one REPL show away from a password, in exactly the places application state gets serialized without much thought: a debug endpoint, a health check that dumps config, an error reporter attaching context.
PormG bounds all three. The two routes deliberately promise different things:
| what it emits | |
|---|---|
JSON.json(pool) | {"pormg_connection":"PostgreSQL"} — no connection string, in any form |
JSON.json(settings) | a pormg_settings object: app_env, db_def_folder, model_file, time_zone, django_prefix, change_db, change_data, implicit, and the backend name. Never the pool's DSN, never db_config_settings |
JSON.json(PormG.config) | the same, once per configured key |
show(pool) — nested, e.g. inside a Vector | Pool(PostgreSQL, 10 slots) — no connection string |
| the REPL card, when you type the value | the above plus the connection string, redacted |
julia> pool = PormG.config["db"].connections
Pool(PostgreSQL, 10 slots)
dsn: host='127.0.0.1' port='5432' password=**** dbname='formula1' user=****Why JSON is stricter than the display. A card is read by a human who asked for it, at a terminal, and "which database is this pointed at?" is the only reason to type a pool — so the card answers it, redacted. A JSON document travels: to a debug endpoint, a log aggregator, an error reporter, a third party. Redaction is a denylist, and a denylist is one unfamiliar DSN dialect away from emitting a password, which is an acceptable risk at a terminal and not on a wire.
If you do want the connection string in a document, ask for it explicitly:
PormG.Configuration.redact_secret(pool.connection_string)redact_secret is the single owner of the redaction rule and the same one every log line and exception uses. It covers both dialects PormG accepts — the libpq keyword form (password=…, user=…, quoted values included) and the URL form (postgres://user:password@host/db → postgres://****:****@host/db).
db_config_settings is never emitted at all, by either route. It is the raw parsed YAML block for your environment, so it can hold a password: key, a url: carrying an entire DSN, or paths to TLS private keys — and being an arbitrary user-supplied dictionary, there is nothing to redact it with. Read it directly if you need it; PormG will not put it in a document for you.
Advisory Locks
Use advisory locks to ensure long-running tasks (migrations, seeds, imports) do not run in parallel across processes.
using PormG, LibPQ # advisory locks need PostgreSQL or MySQL
# Wrap multiple operations in an advisory lock
PormG.run_in_transaction("db") do
with_advisory_lock(settings, "my_job_name") do
# Long-running task
M.Result.objects.create("year" => 2025, "name" => "New Race")
bulk_insert(M.Result.objects, results_df)
end
endKey Technical Details
- Hashing: Keys are hashed using MD5 to provide a 64-bit bigint identifier on PostgreSQL, and a 38-character lock name (
pormg:and an MD5, the database name hashed in) on MySQL. - Cleanup: PostgreSQL and MySQL release the lock automatically if the session drops.
- SQLite Limitation: SQLite does not support advisory locks; the helper is a no-op on that backend — the body runs unprotected and the wait/timeout keyword arguments are ignored. It warns once per lock key; pass
on_missing_lock = :ignoreto accept the no-op silently, oron_missing_lock = :errorto raiseBackendCapabilityErrorinstead of running unprotected. See Advisory Locks.
The Boot-Time Hazard
@import_models eventually calls Models.set_models(...), and a package's __init__ calls it directly (the run-time bind). Either way, it binds every model to a connect_key by matching its models folder against the configurations already loaded. Everything below is about that match: what it does when it finds nothing, and what it does when it finds more than one thing.
When nothing matches, PormG refuses
If no loaded configuration matches the models folder, set_models (and so @import_models and @models_module) raises InvalidConfigurationError. The message names the folder, the configurations that are loaded, and the call to add:
PormG.Configuration.load("db"; root = APP_ROOT, env = "prod") # before @import_modelsBefore #1226, PormG loaded the folder itself and logged a @warn. That load decided two things you did not state:
- The environment.
ENV["PORMG_ENV"]when it is set, else the file'sdefault_env:. In a server that has not selected its environment yet, that is usuallydev, which in many deployments points at production. Since #1228, that fallback pastPORMG_ENVis a REPL's only, for everyload(see Environment Selection). - The key. The folder's path, not a name your application chose.
The warning went to a boot log nobody reads, and the queries ran. Now the registration fails instead, so the wrong-database case cannot happen silently.
The same rule holds on the first Model.objects of a restored package image. The self-heal there (ensure_model_initialized) binds the model to a configuration that is already loaded, or raises the same error. It never loads one. The __init__ an @models_module block defines binds only when the configuration is already loaded, and otherwise does nothing. It runs before your package's own __init__, so refusing there would make using fail. @import_models adds no __init__ (#1607): the first query rebinds from the key it recorded.
The REPL keeps the convenience. In an interactive session (isinteractive()), a direct @import_models or set_models with nothing loaded loads the folder anyway, with the old warning (#1248). A service, a test or a script is not interactive and gets the refusal; it can still ask for the guess one call at a time:
PormG.Models.set_models(MyModels, "db"; implicit = true)A test or a script that only renders SQL and never runs a query needs neither: register the key with Configuration.register_offline("db"; adapter = :postgres) before importing the models. It binds them to a connection that renders SQL and never connects. A load or implicit = true there would open a real pool. The recipe is on Testing Your App.
There is no process-wide switch: whether a registration refuses never depends on what some other code in the process turned on earlier. The self-heal and the @models_module __init__ above never load, not even in a REPL — using SomePackage at a REPL is an interactive session too.
An implicit load marks the entry as implicit: PormG.Configuration.status(key).implicit reads it back. The rules below go by that mark, not by the shape of the key.
When several configurations match, the explicit key wins
Applications normally load configuration by short key (Configuration.load("db")) and import models by relative path (@import_models "../db/models.jl"). Those two agree on the resolved absolute path only when the working directory lines up, so PormG also matches on the folder's final component.
Matches are ranked — an exact path beats a folder-name match. Within a rank, a key you loaded explicitly beats one set_models minted implicitly, because the implicit entry is the one whose environment came from PORMG_ENV/default_env: rather than from your application. PormG records that on the entry rather than inferring it from the key's spelling, so a relative implicit key (set_models(mod, "db")) still loses to an absolute key you loaded yourself.
Only when that still leaves several candidates is the binding genuinely ambiguous — two configured folders that end in the same name (db and vendor_app/db), say. PormG then warns, lists every candidate, and picks the lexicographically first so that at least the same configuration resolves the same way on every boot. Give the folders distinct final components and the ambiguity disappears.
One boot for development and production
A package binds its models at run time: the module body includes the models file, and __init__ loads the configuration and calls set_models(models, "db"; source = MODELS_FILE). The full snippet is under The Bootstrap Sequence. Precompiling then reads no connection.yml, so the image holds no connection detail, and every process binds in its own session. A Revise reload is the only difference between development and production, and it is decided at run time: with Revise loaded, source registers it; without Revise, source does nothing (#1487).
Using @import_models in development and the run-time bind in production looks tempting, but it does not work:
@import_modelsin a package body runs at precompile time, in development too. The precompile worker is never interactive, so the folder has to be loaded right there in the module body (next section). The reload it registers is skipped there, and the restored session never runs that code again. So in a package,@import_modelsgives no reload even in development.- A module-body choice is frozen into the cache. An
ifon the environment in the module body runs once, in the precompile worker. ChangingRACECONTROL_ENVorPORMG_ENVlater does not invalidate the cache, so the package silently keeps the other shape. - Tests and services run the production shape. They are not interactive, so a development-only shape is a path that no test exercises. #1226 surfaced this way: the documented boot broke only outside a REPL.
Django draws the same line. models.py is the same file in development and production, importing it never reads DATABASES, and runserver's autoreload is a choice made at run time.
Loading before @import_models is not enough for a package
This section is for a package that keeps @import_models in its module body. The boot above avoids the problem it describes.
The obvious fix — call Configuration.load(path; env = ...) above @import_models — is correct but incomplete for a precompiled package. Module-body code runs at precompile time, and PormG's config is a global inside PormG, so those entries do not survive into the session. At runtime config starts empty again, and the first Model.objects access raises InvalidConfigurationError because nothing has loaded the folder (see above).
Load the configuration in both places: in the module body, so the right connect_key is baked into the image, and again from your module's __init__, so it is right in the running session. Route both through one helper so they cannot drift apart. Repeating it costs nothing: a load of a folder already loaded with the same environment keeps the existing pool. The module-body load opens no connection either: pools connect on first use, and the extensions: probe and the connection sweeper wait for the run-time load, so the machine that precompiles the package does not need the database.
module RaceControl
using PormG
const DB_DIRS = ["db", "db_telemetry"]
const APP_ROOT = normpath(joinpath(@__DIR__, ".."))
# `root` keeps the keys short ("db", "db_telemetry") while the folders resolve under APP_ROOT, so
# the precompile worker, `__init__` and the server all register the same keys, whatever their `pwd()`.
_load_configs() =
PormG.Configuration.load_many(DB_DIRS; root = APP_ROOT, env = get(ENV, "RACECONTROL_ENV", "dev"))
_load_configs() # precompile: lets @import_models bind, and record the key
PormG.@import_models "../db/models.jl" models
__init__() = _load_configs() # runtime: the image's configuration did not survive
endEach half does a different job, which is why dropping either one breaks something. The first Model.objects access runs the self-heal (ensure_model_initialized), so the state right after using and the state your queries actually run against are two different columns:
| Where the configuration is loaded | After using, before any .objects | After the first .objects access |
|---|---|---|
Module body and __init__ | short key, config populated ✔ | unchanged ✔ |
| Module body only | short key, config empty | InvalidConfigurationError: the key the models were imported under is not loaded, and the message names the load(key; root, env) call to add |
__init__ only | the precompile run fails: @import_models in the module body raises InvalidConfigurationError | — |
| Neither | as "__init__ only" | — |
The module body runs at precompile time, so it is what lets @import_models bind the models at all; __init__ runs in the session, so it is what puts the connection in config where a query can find it. Neither substitutes for the other.
What the first .objects binds to (#1227). When @import_models binds the models it records, in the models module's model set (its MODEL_SET, #1607), the key the folder was loaded under and the root it was loaded relative to. That record is part of the image. On the first .objects of a restored session the self-heal reads it back and binds to that key — the one your __init__ loads — or refuses, naming load(key; root, env). It does not look through config for some other configuration that happens to match the folder, so a session that loaded the same folder under a different key, or from a different working directory, cannot quietly change which entry the models use. The module-body load is therefore what makes the import succeed and fixes the key; the __init__ load is the one the running session depends on. "Module body only" is the row to watch: it looks right after using and fails on the first query. Since #1226 it fails with an error that says what is missing. Before, it quietly ran against a database you did not choose.
Verify the binding instead of assuming it
connect_key is the field that decides which database a model's queries reach, so read it back rather than inferring it from the configuration you meant to load:
julia> using RaceControl
julia> RaceControl.models.Driver.objects # the access that re-derives the key
julia> RaceControl.models.Driver.connect_key # want the short key, not an absolute path
"db"The implicit load runs only in a REPL or when a call passed implicit = true. When it did, PormG.Configuration.status(RaceControl.models.Driver.connect_key).implicit is true, and the environment came from PORMG_ENV/default_env:, not from whatever your application selected.