PormG.jl Documentation
PormG.jl is a Django-inspired ORM for Julia, built with an async-first architecture for high-concurrency web frameworks. It brings the expressive power of Django's query builder to Julia while leveraging Julia's performance and type system.
Why PormG?
| Feature | Description |
|---|---|
| Django-Style Query Builder | Familiar syntax using filter, values, order_by, and chainable methods. Join traversal with __ notation. |
| Async-First Execution | Non-blocking I/O via LibPQ.async_execute. Synchronous helpers are thin wrappers — the event loop is never blocked. |
| Cross-Database Support | PostgreSQL (LibPQ.jl) and SQLite (SQLite.jl) with automatic connection pooling and dialect adaptation. |
| Multi-Database & Multi-Tenancy | Switch databases at runtime with .db("tenant_id"). Lazy connection resolution via resolver functions. |
| F-Expressions & Aggregations | Column arithmetic, field-to-field comparisons, Count, Sum, Avg, Max, Min with automatic GROUP BY / HAVING. |
| Migrations | State-based schema reconciliation with makemigrations / migrate, destructive-operation guards, and a history table. |
| Transactions | run_in_transaction with async context propagation, savepoint support, and automatic rollback on error. |
| Advisory Locks | Distributed coordination via with_advisory_lock for safe concurrent processes. |
Installation
PormG is currently in early development and is not yet registered in the Julia General Registry. Install the development version using the Julia package manager:
using Pkg
Pkg.add(url="https://github.com/PingoLee/PormG.jl")Or develop locally:
using Pkg
Pkg.develop(url="https://github.com/PingoLee/PormG.jl")PormG is pre-1.0, so the API still changes when a change makes it better. Every breaking change ships with a migration entry: what changed, a grep that finds the affected call sites in your code, and the concrete before → after edit.
Run PormG.upgrade_guide(from = v"<your pinned version>") to see only what applies to your version — usually nothing. See Upgrading PormG.
Please report any issues on the GitHub repository.
Install a database driver
PormG does not pull in a SQL driver automatically — LibPQ (PostgreSQL) and SQLite are weak dependencies, so each app installs and loads the one it uses as a direct dependency:
using Pkg
Pkg.add("LibPQ") # PostgreSQL backend
Pkg.add("SQLite") # SQLite backendLoad the driver alongside PormG (using PormG, LibPQ or using PormG, SQLite). A bare using PormG loads the ORM but no backend, so the first query raises a clear error telling you which driver to load:
PormG: the PostgreSQL backend requires LibPQ. Run `using LibPQ` (or `using PormG, LibPQ`) so the PostgreSQL extension loads.Quick Start
1. Initialize Your Project
using PormG
PormG.setup() # Interactive setup for database and modelsThis creates a db/ folder with a connection.yml template and a models.jl skeleton.
2. Configure the Database Connection
Open db/connection.yml and configure your database:
default_env: dev
dev:
adapter: PostgreSQL
database: your_database_name
host: 'localhost'
username: your_username
password: your_password
port: 5432
config:
change_db: true # create the database if it doesn't exist
change_data: true # allow data modifications
time_zone: 'America/Sao_Paulo'For SQLite, use:
default_env: dev
dev:
adapter: SQLite
database: 'my_app.db'
config:
change_db: true # allow schema migrations
change_data: true # allow data mutationsImplicit Defaults & The Omitted config: Gotcha: If the config: block is omitted or left empty under an environment, PormG applies strict safety-first defaults:
change_dbdefaults tofalse: Database migrations and schema modifications (makemigrations/migrate) are disabled.change_datadefaults tofalse: All data modifications (creates, updates, deletes) are disabled at the query layer.time_zonedefaults to"UTC": Default database timezone.
To perform migrations or modify data, you must explicitly define config: and set these settings to true in your active environment.
3. Define Your Models
Create db/models.jl with your model definitions.
PormG preserves the case you declare field names with: driverid stays driverid (column "driverid"), and field lookups are case-sensitive — query a field by the exact case you declared it. The recommended house style is lowercase snake_case (PormG's own models use it), so define new fields that way; reserve mixed-case declarations for faithfully mapping existing mixed-case/uppercase columns you don't control.
module models
import PormG.Models
Driver = Models.Model("drivers",
driverid = Models.IDField(),
forename = Models.CharField(max_length=50),
surname = Models.CharField(max_length=50),
nationality = Models.CharField(max_length=50),
dob = Models.DateField(null=true),
)
Circuit = Models.Model("circuits",
circuitid = Models.IDField(),
name = Models.CharField(max_length=100),
location = Models.CharField(max_length=100),
country = Models.CharField(max_length=100),
)
Race = Models.Model("races",
raceid = Models.IDField(),
name = Models.CharField(max_length=255),
year = Models.IntegerField(),
date = Models.DateField(),
circuitid = Models.ForeignKey(Circuit, pk_field="circuitid", on_delete="CASCADE"),
)
Constructor = Models.Model("constructors",
constructorid = Models.IDField(),
name = Models.CharField(max_length=50),
nationality = Models.CharField(max_length=50),
)
Result = Models.Model("results",
resultid = Models.IDField(),
raceid = Models.ForeignKey(Race, pk_field="raceid", on_delete="CASCADE"),
driverid = Models.ForeignKey(Driver, pk_field="driverid", on_delete="RESTRICT"),
constructorid = Models.ForeignKey(Constructor, pk_field="constructorid", on_delete="RESTRICT"),
positionorder = Models.IntegerField(),
points = Models.FloatField(),
)
end4. Load Configuration and Import Models
using PormG, LibPQ, DataFrames # LibPQ → PostgreSQL; use SQLite instead if your adapter is SQLite
# Load configuration — must happen BEFORE importing models
PormG.Configuration.load("db")
# Import models with hot-reload support (Revise.jl)
PormG.@import_models "db/models.jl" models
import .models as M5. Create and Apply Migrations
PormG.Migrations.makemigrations("db") # Analyze models and generate migration
PormG.Migrations.migrate("db") # Apply migration to database6. Create Records
# Single record creation
M.Driver.objects.create(
"forename" => "Ayrton",
"surname" => "Senna",
"nationality" => "Brazilian"
)
# Bulk insert for multiple records
bulk_data = DataFrame([
Dict("forename" => "Alain", "surname" => "Prost", "nationality" => "French"),
Dict("forename" => "Nelson", "surname" => "Piquet", "nationality" => "Brazilian"),
])
# We call bulk_insert on the .objects handler to respect the ORM boundary
bulk_insert(M.Driver.objects, bulk_data)7. Query Your Data
using PormG: F
using PormG.Functions: Count
# Simple filter and list
drivers = M.Driver.objects.filter("nationality" => "Brazilian").order_by("surname").list()
# Chainable methods with DataFrame output (query fields in the case they were declared;
# the F1 models use lowercase, so the paths below are lowercase)
df = M.Result.objects.filter(
"driverid__nationality" => "Brazilian",
"positionorder" => 1
).values(
"driverid__forename",
"driverid__surname",
"raceid__year",
).order_by("-raceid__year") |> DataFrame
# Aggregations
df = M.Result.objects.filter(
"positionorder" => 1
).values(
"constructorid__name",
"wins" => Count("resultid")
).order_by("-wins") |> DataFrameJulia Method Chain Syntax Gotcha: Multi-line method chains must use trailing-dot syntax (the . must be at the end of the line) or stay completely inline. Placing the dot at the start of the next line (leading-dot syntax) will cause a Julia ParseError.
# ✓ CORRECT: Trailing dot or inline
df = M.Driver.objects.
filter("nationality" => "Brazilian").
list()
# ✗ INCORRECT (ParseError): Leading dot
df = M.Driver.objects
.filter("nationality" => "Brazilian")
.list()8. Update and Delete
# Update matching records
M.Driver.objects.filter("nationality" => "Brazilian").update("nationality" => "Brazil")
# Atomic update with F-expression (no read-modify-write race)
M.Result.objects.filter("resultid" => 1).update("points" => F("points") + 10)
# Delete matching records
M.Driver.objects.filter("surname" => "TestDriver").delete()Django Data-Type & Compatibility Contract
PormG is designed to be fully wire-format compatible with tables managed by Django. It guarantees identical column serialization and mutation semantics across both PostgreSQL and SQLite backends:
- TIMESTAMPTZ Contract: Naive
DateTimevalues are treated as UTC, while timezone-awareZonedDateTimevalues preserve the exact UTC instant across backends. (SQLite stores ISO 8601 strings and normalizes on read). - DateField Truncation: Temporal inputs like
DateTimeorZonedDateTimeare automatically truncated to their calendar date portion when saved into aDateField, matching Django's silent truncation behavior instead of throwing errors. - DecimalField Precision: Underpinned by
NUMERICtypes,DecimalField(max_digits, decimal_places)prevents float drift in round-trips (e.g.99.99is retrieved precisely as99.99inDecimals.Decimalformat). - Auto Temporal Fields:
auto_now_add=true: Automatically populated with the transaction's timezone-aware timestamp uponINSERT, then frozen during subsequent updates.auto_now=true: Automatically populated uponINSERTand refreshed with the system timestamp on everyUPDATE.
Architecture Overview
PormG is structured into five distinct, decoupled layers, ensuring a clean separation of concerns, high-performance execution, and cross-database dialect compatibility:
| Layer | Responsibility | Key Components & Files |
|---|---|---|
| 1. Application | High-level user interface. Developers define models and write fluent queries. | M.Driver.objects.filter(...)<br>• Models.jl |
| 2. Query Builder | Fluent chaining API. Resolves chaining, table joins (__ notation), and subqueries into an abstract AST. | filter, values, order_by, cjoin<br>• QueryBuilder.jl |
| 3. Dialect Adapter | Compiles the abstract query builder AST into vendor-specific, parameterized SQL strings. | PostgreSQL vs. SQLite translation<br>• Dialect.jl |
| 4. Connection Pool | Async-first execution layer. Manages active connections and non-blocking I/O without blocking the Julia event loop. | LibPQ.async_execute, SQLite.execute<br>• ConnectionPool.jl |
| 5. Config & Tenants | Multi-database and multi-tenancy registry. Loads configurations and maps active tenant resolvers. | load, load_many, resolver<br>• Configuration.jl |
Documentation Guide
This documentation is organized into the following sections:
| Section | Description |
|---|---|
| Configuration | Database connections, environments, multi-tenancy, and health checks. |
| Models | Defining models, @import_models, hot-reloading, and naming conventions. |
| Fields | Comprehensive field type reference: text, numeric, date, boolean, relationships. |
| Migrations | makemigrations, migrate, dry_run, history table, destructive guards. |
| Writing | |
| Overview | Async-first write philosophy and performance comparison. |
| Creating Records | Single-record create() patterns. |
| Updating Records | update() with filters and F-expressions. |
| Deleting Records | Safe record deletion with cascading. |
| Bulk Operations | bulk_insert, bulk_copy, and bulk_update. |
| Transactions | run_in_transaction, async context propagation, savepoints. |
| Reading | |
| Overview | Query execution, output formats, and query styles. |
| Values and Joins | Column selection, __ join traversal, aliases. |
| Custom Joins | .cjoin() for runtime join conditions and .on() for ON-clause predicates. |
| Filters and Aggregates | Lookup operators, grouping, and HAVING. |
| Functions and Dates | SQL functions and date-oriented querying. |
| Subqueries and CTEs | IN subqueries, .with(...) CTEs, and the CTE(name, path) column reference. |
| Field Expressions | F() expressions, column arithmetic, aggregate ratios. |
| Q Objects | Complex boolean logic with Q, Qor, and NOT. |
| Import from Django | Migrating models and data from Django projects. |
| Async & Concurrency | Task-based concurrent queries, fan-out patterns, fetch_async, pool interplay. |
| Advisory Locks | Distributed locking with with_advisory_lock. |
| Contributing | Development workflow, @pormg_debug breakpoints, and testing conventions. |
| API Reference | Full auto-generated API reference and exported function catalog. |
Database Example: Formula 1
The examples throughout this documentation use the Formula 1 World Championship dataset (Kaggle). This provides a real-world schema with multiple related tables (Driver, Race, Circuit, Constructor, Result, Status), making it ideal for demonstrating joins, aggregations, and complex queries.
# Example: Find all Brazilian race winners with circuit information
df = M.Result.objects.filter(
"driverid__nationality" => "Brazilian",
"positionorder" => 1,
).values(
"driverid__forename",
"driverid__surname",
"raceid__name",
"raceid__circuitid__name",
"raceid__year",
).order_by("-raceid__year") |> DataFrameContributing
Contributions to PormG are welcome! Please see the Contributing & Debugging page for the development workflow, testing conventions, and how to use @pormg_debug breakpoints.
License
PormG is licensed under the MIT License. See the LICENSE file for details.