SQLAlchemy / Chapter 2 of 4

Advanced queries and async sessions

Write SQLAlchemy bulk updates, subqueries, CTEs, and window functions. Use AsyncSession safely and connect database sessions to FastAPI.

8 topicsUpdated
10 / 27

INSERT, UPDATE, DELETE, RETURNING, executemany, and upsert#

Use object changes for individual domain objects. Use DML statements for set-based changes, high-volume inserts, RETURNING, or when loading each object first is unnecessary.

PythonDML through Session
from sqlalchemy import delete, insert, update

with SessionLocal.begin() as session:
    new_user = session.scalar(
        insert(User).values(name="Ada", email="ada@example.com")
        .returning(User)
    )
    changed = session.execute(
        update(User).where(User.is_active.is_(False))
        .values(is_active=True).returning(User.id, User.email)
    )
    session.execute(delete(User).where(User.created_at < cutoff))
SQLEquivalent SQL
INSERT INTO user_account (name, email, is_active)
VALUES ('Ada', 'ada@example.com', true) RETURNING *;

UPDATE user_account SET is_active = true
WHERE is_active = false RETURNING id, email;

DELETE FROM user_account WHERE created_at < :cutoff;
PythonBulk parameters and PostgreSQL upsert
from sqlalchemy.dialects.postgresql import insert as pg_insert

session.execute(insert(User), [
    {"name": "Ada", "email": "ada@example.com"},
    {"name": "Lin", "email": "lin@example.com"},
])

stmt = pg_insert(User).values(name="Ada", email="ada@example.com")
stmt = stmt.on_conflict_do_update(
    index_elements=[User.email],
    set_={"name": stmt.excluded.name},
).returning(User.id, User.name)
row = session.execute(stmt).one()
SQLPostgreSQL SQL
INSERT INTO user_account (name, email) VALUES
  ('Ada', 'ada@example.com'), ('Lin', 'lin@example.com');

INSERT INTO user_account (name, email)
VALUES ('Ada', 'ada@example.com')
ON CONFLICT (email) DO UPDATE SET name = EXCLUDED.name
RETURNING id, name;
Bulk DML bypasses normal instance behavior.
It does not call Python setters or per-instance ORM events. If the Session already holds matching objects, choose an appropriate synchronize_session option or use a separate short Session.
11 / 27

Functions, CASE, casts, binds, text, aggregates, and operators#

Columns, literal values, operators, and database functions form composable expression trees. SQLAlchemy compiles the tree for the current dialect and binds values separately from SQL text.

PythonCalculated columns and aggregate
from sqlalchemy import case, cast, func, literal, String

count_addresses = func.count(Address.id)
status = case(
    (User.is_active, literal("active")),
    else_=literal("disabled"),
).label("status")

stmt = (
    select(User.id, func.upper(User.name).label("name_upper"),
           count_addresses.label("address_count"), status)
    .outerjoin(User.addresses)
    .group_by(User.id, User.name, User.is_active)
    .having(count_addresses >= 2)
)
typed = select(cast(User.id, String).label("id_as_text"))
SQLEquivalent SQL
SELECT u.id, upper(u.name) AS name_upper,
       count(a.id) AS address_count,
       CASE WHEN u.is_active THEN 'active' ELSE 'disabled' END AS status
FROM user_account u LEFT JOIN address a ON u.id = a.user_id
GROUP BY u.id, u.name, u.is_active
HAVING count(a.id) >= 2;

SELECT CAST(id AS VARCHAR) AS id_as_text FROM user_account;
func.*Database function namespace: count, lower, vendor functions.
case()Conditional CASE WHEN expression.
cast()Ask database to convert a value.
type_coerce()Tell SQLAlchemy a type without necessarily emitting CAST.
literal(), bindparam()Bound SQL value / named reusable parameter.
text()Exact SQL with named binds; use deliberately.
Debug only.
print(stmt) shows SQL placeholders. stmt.compile(engine, compile_kwargs={"literal_binds": True}) inlines values for local inspection; do not log secrets or use it to construct production SQL.
12 / 27

Aliases, subqueries, CTEs, EXISTS, unions, windows, and LATERAL#

Build a select, then turn it into a named/selectable object. This keeps complex SQL readable and prevents unsafe hand-assembled SQL strings.

PythonSubquery, CTE, EXISTS, UNION, window
from sqlalchemy import exists, union_all

counts = (
    select(Address.user_id, func.count().label("n"))
    .group_by(Address.user_id).subquery()
)
with_two = select(User.name, counts.c.n).join(
    counts, counts.c.user_id == User.id
).where(counts.c.n >= 2)

active_users = select(User.id, User.name).where(User.is_active).cte("active_users")
has_address = select(User.name).where(
    exists(select(Address.id).where(Address.user_id == User.id))
)
names = union_all(select(User.name.label("value")),
                  select(Address.email.label("value")))
rank = func.row_number().over(
    partition_by=Address.user_id, order_by=Address.id.desc()
).label("rank")
SQLEquivalent SQL
SELECT u.name, c.n FROM user_account u JOIN
  (SELECT user_id, count(*) n FROM address GROUP BY user_id) c
ON c.user_id = u.id WHERE c.n >= 2;

WITH active_users AS (
  SELECT id, name FROM user_account WHERE is_active = true
) SELECT * FROM active_users;

SELECT name FROM user_account u WHERE EXISTS
  (SELECT 1 FROM address a WHERE a.user_id = u.id);

SELECT name AS value FROM user_account
UNION ALL SELECT email AS value FROM address;

SELECT row_number() OVER (PARTITION BY user_id ORDER BY id DESC) FROM address;

Correlated counts and top row per group

A correlated subquery reads a value from the outer row. Use scalar_subquery() to select one value per outer row, such as the number of each user’s addresses. A window function ranks rows within each group; filtering the rank in an outer SELECT picks the newest child of each parent. A window function is not valid directly in WHERE on many databases, hence the subquery.

PythonSQLAlchemy 2.x
from sqlalchemy.orm import aliased

address_count = (select(func.count(Address.id))
                 .where(Address.user_id == User.id)
                 .correlate(User).scalar_subquery())
stmt = select(User.name, address_count.label("address_count"))

ranked = (select(Address,
                 func.row_number().over(
                     partition_by=Address.user_id,
                     order_by=Address.id.desc()).label("rn"))
          .subquery())
newest = aliased(Address, ranked)
stmt = select(newest).where(ranked.c.rn == 1)
latest_addresses = session.scalars(stmt).all()
SQLEquivalent SQL
SELECT u.name, (SELECT count(*) FROM address AS a
                WHERE a.user_id = u.id) AS address_count
FROM user_account AS u;

SELECT r.id, r.user_id, r.email FROM (
  SELECT a.*, row_number() OVER
    (PARTITION BY a.user_id ORDER BY a.id DESC) AS rn
  FROM address AS a
) AS r WHERE r.rn = 1;

Recursive CTEs and lateral subqueries

A recursive CTE repeatedly follows links, useful for a tree of descendants. Its base query starts at the root; the recursive query joins children to the rows found so far. Prevent cycles in data or add a depth/cycle guard. LATERAL evaluates a subquery once per outer row and is supported by PostgreSQL; other databases differ. This example returns each user’s newest address.

PythonSQLAlchemy 2.x
from sqlalchemy import literal, true

descendants = (select(Node.id, Node.parent_id, literal(0).label("depth"))
               .where(Node.id == root_id).cte("descendants", recursive=True))
n = Node.__table__.alias("n")
descendants = descendants.union_all(
    select(n.c.id, n.c.parent_id, descendants.c.depth + 1)
    .join(descendants, n.c.parent_id == descendants.c.id)
    .where(descendants.c.depth < 20)
)
stmt = select(descendants.c.id).select_from(descendants)

newest = (select(Address.email)
          .where(Address.user_id == User.id)
          .order_by(Address.id.desc()).limit(1).lateral("newest"))
stmt = select(User.name, newest.c.email).join(newest, true())
SQLEquivalent SQL
WITH RECURSIVE descendants(id, parent_id, depth) AS (
  SELECT id, parent_id, 0 FROM node WHERE id = :root_id
  UNION ALL
  SELECT n.id, n.parent_id, d.depth + 1 FROM node AS n
  JOIN descendants AS d ON n.parent_id = d.id WHERE d.depth < 20
) SELECT id FROM descendants;

SELECT u.name, newest.email FROM user_account AS u
JOIN LATERAL (
  SELECT a.email FROM address AS a WHERE a.user_id = u.id
  ORDER BY a.id DESC LIMIT 1
) AS newest ON true;
FeatureBuilderPurpose
Aliasaliased(User) / table.alias()Same table/selectable under a second name, often self-join.
Subquerystmt.subquery()Nested SELECT in FROM or join.
Scalar subquerystmt.scalar_subquery()Nested query yielding one value in an expression.
CTEstmt.cte("name")Named WITH name AS (...).
Recursive CTEcte(recursive=True) + union_all()Tree/hierarchy/graph traversal.
Set operationsunion, union_all, intersect, except_Combine compatible result sets.
Windowfunc.rank().over(...)Rank/running totals without collapsing rows.
LATERALselect(...).lateral()Subquery can refer to prior FROM values; backend support varies.
13 / 27

Metadata, constraints, indexes, DDL, identity, sequences, and computed values#

Metadata is SQLAlchemy’s table catalog. It can create tables in tests and development. Production changes should use Alembic migrations so every environment follows the same schema history.

PythonConstraints and index
from sqlalchemy import CheckConstraint, Computed, Index, UniqueConstraint

class Product(Base):
    __tablename__ = "product"
    __table_args__ = (
        UniqueConstraint("tenant_id", "sku", name="uq_product_tenant_sku"),
        CheckConstraint("price_cents >= 0", name="ck_product_nonnegative_price"),
        Index("ix_product_tenant_active", "tenant_id", "is_active"),
    )
    id: Mapped[int] = mapped_column(primary_key=True)
    tenant_id: Mapped[int]
    sku: Mapped[str] = mapped_column(String(64))
    price_cents: Mapped[int]
    quantity: Mapped[int]
    line_total: Mapped[int] = mapped_column(Computed("price_cents * quantity"))
    is_active: Mapped[bool] = mapped_column(default=True)
SQLEquivalent SQL
CREATE TABLE product (
  id SERIAL PRIMARY KEY, tenant_id INTEGER NOT NULL, sku VARCHAR(64) NOT NULL,
  price_cents INTEGER NOT NULL, quantity INTEGER NOT NULL,
  line_total INTEGER GENERATED ALWAYS AS (price_cents * quantity) STORED,
  is_active BOOLEAN NOT NULL,
  CONSTRAINT uq_product_tenant_sku UNIQUE (tenant_id, sku),
  CONSTRAINT ck_product_nonnegative_price CHECK (price_cents >= 0)
);
CREATE INDEX ix_product_tenant_active ON product (tenant_id, is_active);
KeysPrimaryKeyConstraint, ForeignKey, ForeignKeyConstraint.
RulesUniqueConstraint, CheckConstraint.
IndexesIndex, multi-column, expression, partial/dialect options.
Generated valuesIdentity, Sequence, Computed, server defaults.
DDLcreate_all, drop_all, CreateTable, DDL events.
Naming conventionPredictable constraint names make migrations reliable.
create_all() does not migrate.
It creates missing objects; it does not safely rename a column, alter a constraint, or move existing data. Use Alembic for production changes.
14 / 27

Column types, JSON, UUID, Enum, mutable values, and custom types#

A type converts values between Python and the database. Type hints infer many defaults; explicit types give precision, length, special behavior, and database-specific power.

PythonPortable common types
from datetime import date, datetime
from decimal import Decimal
from enum import Enum
from uuid import UUID, uuid4
from sqlalchemy import JSON, Date, DateTime, Enum as SAEnum, Numeric, Uuid

class OrderState(str, Enum):
    pending = "pending"
    paid = "paid"

class Order(Base):
    __tablename__ = "orders"
    id: Mapped[UUID] = mapped_column(Uuid, primary_key=True, default=uuid4)
    state: Mapped[OrderState] = mapped_column(SAEnum(OrderState))
    amount: Mapped[Decimal] = mapped_column(Numeric(12, 2))
    metadata_: Mapped[dict] = mapped_column("metadata", JSON, default=dict)
    ordered_on: Mapped[date] = mapped_column(Date)
    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
SQLRepresentative PostgreSQL SQL
CREATE TYPE orderstate AS ENUM ('pending', 'paid');
CREATE TABLE orders (
  id UUID PRIMARY KEY, state orderstate NOT NULL,
  amount NUMERIC(12, 2) NOT NULL, metadata JSON NOT NULL,
  ordered_on DATE NOT NULL, created_at TIMESTAMPTZ NOT NULL
);
PythonMutable JSON and custom conversion
from sqlalchemy.ext.mutable import MutableDict
from sqlalchemy.types import String, TypeDecorator

settings: Mapped[dict] = mapped_column(MutableDict.as_mutable(JSON), default=dict)
profile.settings["theme"] = "dark"  # Session detects in-place edit

class LowercaseEmail(TypeDecorator[str]):
    impl = String(255)
    cache_ok = True
    def process_bind_param(self, value, dialect):
        return value.lower().strip() if value else value

email: Mapped[str] = mapped_column(LowercaseEmail())
SQLWhat database sees
UPDATE profile SET settings = '{"theme":"dark"}' WHERE id = :id;
INSERT INTO contact (email) VALUES ('ada@example.com');
-- TypeDecorator converts at Python/database boundary.
FamilyUseWatch for
Integer/String/Text/BooleanEveryday valuesLength and text behavior can vary by dialect.
Numeric(p,s)Money/exact decimalsUse Decimal, not float.
Date/Time/DateTimeTime valuesFor global timestamp use timezone-aware UTC values.
JSONFlexible structured dataValidate schema; query/index API varies by database.
Uuid/EnumPortable identity/finite statesNative backend types may need migrations to evolve.
Dialect typesPostgreSQL JSONB/ARRAY/range, etc.Intentional portability trade-off.
15 / 27

Reflection, inspection, deferred reflection, and Automap#

Reflection reads an existing database schema into SQLAlchemy table metadata. It helps with legacy systems, administrative tools, and migrations. Explicit typed models are usually clearer for new product code.

PythonReflect and inspect
from sqlalchemy import MetaData, Table, inspect, select

metadata = MetaData()
legacy_users = Table("legacy_users", metadata, autoload_with=engine)
rows = engine.connect().execute(
    select(legacy_users.c.id, legacy_users.c.email)
)

i = inspect(engine)
print(i.get_table_names())
print(i.get_columns("legacy_users"))
print(i.get_foreign_keys("legacy_users"))
print(i.get_indexes("legacy_users"))
SQLRepresentative introspection SQL
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_name = 'legacy_users';

SELECT id, email FROM legacy_users;
PythonAutomap mapped classes
from sqlalchemy.ext.automap import automap_base

AutomapBase = automap_base()
AutomapBase.prepare(autoload_with=engine)
LegacyUser = AutomapBase.classes.legacy_users

with Session(engine) as session:
    users = session.scalars(select(LegacyUser).limit(10)).all()
SQLSQL
SELECT * FROM legacy_users LIMIT 10;
inspect() is a public runtime discovery API.
It can inspect engine/connection/session behavior, schema details, mappers, mapped objects, attribute histories, and relationship state.
16 / 27

AsyncEngine, AsyncSession, awaitable attributes, and async safety#

Async SQLAlchemy requires an async-capable driver. Its API mirrors synchronous SQLAlchemy, but any work that can talk to the database is awaited. One AsyncSession belongs to one concurrent task.

PythonAsync setup and transaction
from sqlalchemy import select
from sqlalchemy.ext.asyncio import (
    async_sessionmaker, create_async_engine,
)

engine = create_async_engine(
    "postgresql+asyncpg://app:secret@localhost/shop",
    pool_pre_ping=True,
)
AsyncSessionLocal = async_sessionmaker(engine, expire_on_commit=False)

async def find_users() -> list[User]:
    async with AsyncSessionLocal() as session:
        result = await session.scalars(
            select(User).where(User.is_active)
        )
        return result.all()

async def create_user() -> User:
    async with AsyncSessionLocal.begin() as session:
        user = User(name="Ada", email="ada@example.com")
        session.add(user)
    return user
SQLSQL is unchanged
SELECT * FROM user_account WHERE is_active = true;
BEGIN;
INSERT INTO user_account (name, email, is_active)
VALUES ('Ada', 'ada@example.com', true);
COMMIT;
PythonAvoid hidden I/O
from sqlalchemy.ext.asyncio import AsyncAttrs
from sqlalchemy.orm import selectinload

class Base(AsyncAttrs, DeclarativeBase):
    pass

# Best: eager load response data
users = (await session.scalars(
    select(User).options(selectinload(User.addresses))
)).all()

# Explicit lazy I/O, if relationship was not preloaded:
addresses = await user.awaitable_attrs.addresses

# For strict APIs, configure relationships with lazy="raise".
SQLRepresentative SQL
SELECT * FROM user_account;
SELECT * FROM address WHERE user_id IN (1, 2, 3);
-- await schedules Python; SQL is normal SQL.
await session.execute()General async result.
await session.scalars()Entity / scalar result.
await conn.run_sync(fn)Run sync SQLAlchemy helper through async connection.
AsyncAttrsExplicit awaitable mapped attributes.
async_scoped_sessionContext-local async Session helper; dependencies are often clearer.
engine.dispose()Close async pool on shutdown.
Never share an AsyncSession across concurrent tasks.
Do not pass one Session into two asyncio.gather() branches. Each task needs its own transaction/session.
17 / 27

FastAPI with SQLAlchemy#

Make session scope explicit with a dependency. Every request gets a short-lived AsyncSession, which closes even on errors. Endpoint/service code decides the transaction boundary; lower-level helpers normally should not commit independently.

PythonComplete async FastAPI pattern
from collections.abc import AsyncGenerator
from fastapi import Depends, FastAPI, HTTPException, status
from pydantic import BaseModel, ConfigDict, EmailStr
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession

app = FastAPI()

class UserCreate(BaseModel):
    name: str
    email: EmailStr

class UserOut(BaseModel):
    model_config = ConfigDict(from_attributes=True)
    id: int
    name: str
    email: str

async def get_session() -> AsyncGenerator[AsyncSession, None]:
    async with AsyncSessionLocal() as session:
        yield session

@app.post("/users", response_model=UserOut, status_code=status.HTTP_201_CREATED)
async def create_user(data: UserCreate, session: AsyncSession = Depends(get_session)):
    user = User(name=data.name, email=str(data.email))
    session.add(user)
    try:
        await session.commit()
    except Exception:
        await session.rollback()
        raise
    await session.refresh(user)
    return user

@app.get("/users/{user_id}", response_model=UserOut)
async def get_user(user_id: int, session: AsyncSession = Depends(get_session)):
    user = await session.get(User, user_id)
    if user is None:
        raise HTTPException(404, "User not found")
    return user
SQLEndpoint SQL
-- POST /users
BEGIN;
INSERT INTO user_account (name, email, is_active)
VALUES (:name, :email, true);
COMMIT;
SELECT id, name, email FROM user_account WHERE id = :new_id;

-- GET /users/{id}
SELECT * FROM user_account WHERE id = :user_id;
PythonFastAPI lifespan, local schema only
from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Production: run Alembic before deployment instead.
    async with engine.begin() as conn:
        await conn.run_sync(Base.metadata.create_all)
    yield
    await engine.dispose()

app = FastAPI(lifespan=lifespan)
PythonAPI rules
# 1. One session per request.
# 2. Eager-load any relationships response serialization needs.
# 3. Return DTOs/Pydantic models, not an open Session.
# 4. Translate expected IntegrityError to a clear HTTP error.
# 5. Do not run sync database I/O inside async endpoints.
Commit at the service boundary.
If an endpoint creates an order, payment record, and audit log, one outer transaction should decide whether all three become permanent. Helpers may add() or flush() without committing.