A temporal EAV entity store in Postgres
Part 2 — The abstract repository
Part 1 defined the storage: six tables, half-open time windows, and a behavior matrix for every update. This part is about the code that makes the storage usable — one abstract repository class that owns all the machinery, so that a concrete domain type is nothing but a family id and a field list.
Goal
Write the hard parts exactly once. Transactions, the close-and-open temporal dance, relation handling, validators, batch reads, pagination, history — all of it lives in one abstract class. A new domain type ("product", "category", "promo campaign") subclasses it, declares its fields as metadata, and gets a complete CRUD interface for free: create, point-in-time read (one, list, history), temporal update, and soft delete — identical in shape across every domain type, without writing a single query. Consumers speak CRUD in domain terms; only the inside of the class knows the storage is EAV.
The stack
- PostgreSQL — the six tables from Part 1; the only stateful thing in the design.
- TypeORM — entity mapping, and more importantly
DataSource/QueryRunner/EntityManagerfor explicit transactions, plus an escape hatch for raw CTE queries on the read path. - NestJS — dependency injection: repositories are bound to string tokens behind interfaces, so consumers never name a concrete class.
- TypeScript generics — the abstract class is parameterized over four types (DTO, Entity, View, History), which is what lets one implementation return correctly-typed results for every domain.
- lodash —
groupBy/mapValuespivot the flat raw SQL rows back into per-entity, per-field structures on the batch read path.
The shape
One class, four type parameters — the four shapes a domain object takes on its way through the system. The DTO comes in from the API; the View goes out with one current value per field; the History goes out with the full value timeline per field.
Class signature — four generics, one machine
@Injectable()
export abstract class SmentityRepository<
EntityDTO, // create payload (API in)
Entity extends BaseSmentity, // flat domain object
EntityView extends BaseSmentity, // read model: current value per field
EntityHistory extends BaseSmentity, // read model: value timeline per field
> implements IGenericSmentityRepository<EntityDTO, Entity, EntityView, EntityHistory>
{
// ...
}
Minimal fields
The entire state of the class. The first four are the contract a subclass fulfills; the rest is injected plumbing shared by every domain type.
| Field | Who sets it | Role |
|---|---|---|
| familyId / familyName | subclass | Which family this repository owns. Scopes every query the base class ever runs — a repository can never see another family's rows. |
| fieldsMeta | subclass | The virtual schema (Part 1's term): one entry per field — storage name, type, validators, optional default and DTO mapping. This list drives create, update, read assembly and history. |
| entityValidators | subclass | Cross-field checks that need the whole object (single-field checks belong to the field's own validators). |
| _createClass* (×4) | subclass → constructor | The four concrete class constructors matching the generics. TypeScript types vanish at runtime, so the base class needs real constructors to new up DTOs, Views and Histories. |
| dataSource | injected | Creates QueryRunners — every write runs in an explicit transaction, every batch read on one connection. |
| 6 × Repository<…> | injected | TypeORM repositories for the six tables. Inside a transaction the base class re-derives them from the EntityManager so all statements share the transaction. |
IFieldMeta — one virtual schema entry
interface IFieldMeta {
field: string; // storage name: 'name', 'price', 'parentCategory', ...
type: string; // TYPE_STR | TYPE_INT | TYPE_NUM | TYPE_BOOL
// | TYPE_REL_UUID_CHILD_OF | TYPE_REL_UUID_PARENT_OF
// | TYPE_TAG_SET
validators: FieldValidator[]; // async checks, run inside the write tx
default?: unknown; // used when the DTO omits the field
dtoField?: string; // DTO property name, when it differs from `field`
}
interface FieldValidator {
name: string; // shows up in the 422 when the check fails
exec: (queryRunner: QueryRunner, value: unknown) => Promise<boolean>;
}
Minimal methods
Split by visibility — public vs private. Strict visibility isn't a
popular discipline in TypeScript codebases, but here the split is the design: the
public surface speaks CRUD in domain terms, the private one speaks EAV. Nothing temporal or
EAV-shaped may leak upward — a consumer should be unable to tell the storage isn't a plain
table. The public set below is also deliberately deduplicated to the minimal full-CRUD
contract: one method per operation (e.g. a single paginated list read — a separate
findAll is the same read with the paging left out, so it doesn't earn a slot;
same for batch variants of close).
One more rule: every method accepts an optional tx — a
transaction handle — reads included. Passed one, the method joins the caller's transaction;
omitted, it opens its own (writes) or borrows a connection (reads). For now tx is
derived from TypeORM's EntityManager — inside the class it is unwrapped back to an
EntityManager/QueryRunner pair — but the parameter is named for what
it means, not what it currently is. This single rule is what lets a service compose creates,
updates and relation changes across several repositories into one atomic commit.
Public — the CRUD surface
| Method | CRUD | What it does |
|---|---|---|
| create(dto, tx?) | C | One transaction: run the virtual schema validators, open the entity row and one attr/rel row per declared field. Joins the caller's tx when given; opens its own otherwise. |
| findByIdView(uuid, timestamp?, tx?) | R — one | Point-in-time read of one entity → View: current value per field at timestamp (default now), with the inactive-entity clamp applied. |
| findAllPaginated(page, search?, tx?) | R — list | Family-scoped listing, hydrated through the batch read engine. The one list read: paging is a parameter, not a second method. |
| entityHistory(uuid, tx?) | R — audit | The value timeline: a before/after diff per field per change instant, straight from the temporal windows. |
| update(entity, changes, tx?) | U | The temporal update: walks fieldsMeta, dispatches each changed field to the right private primitive inside the caller's tx (or its own), re-reads and returns the fresh View. (In the codebase this is tEntityFieldsUpdate — the CRUD name it deserves is update.) |
| close(uuid, timestamp, tx?) | D | Soft delete, Part 1 style: sets the entity's active_to; history stays readable. A batch variant is the same operation over a list, not a second concept. |
Private — the EAV machinery
| Method | What it does |
|---|---|
| tFieldUpdate(oldField, newField, fieldMeta, tx) | The scalar temporal primitive: close row A, open row B. The entire scalar half of Part 1's behavior matrix lives here. |
| tRelUpdate(oldRel, newRel, fieldMeta, entityUuid, tx) | The relation primitive: close the current edge, open the new one — "move, don't add", the null no-op, and root handling. |
| findByIdsView(uuids, timestamp?, tx?) | The batch read engine every public read delegates to: the raw CTE queries from Part 1 (attributes + both relation directions), pivoted with lodash into Views; detects inactive entities and re-reads them at active_to − ε. |
| _getBatchFieldQuery / _getRelationBatchFieldQuery | The raw SQL builders — the exact CTEs shown in Part 1, parameterized by family and timestamp. |
| runValidators(queryRunner, validators, value) | Executes the virtual schema inside the write transaction; a failed check throws 422 and rolls everything back. |
| smEntityToView / ToDTO / ToHistory | The three transformers between storage rows and the typed shapes, driven by fieldsMeta and the _createClass* constructors. |
Honest footnote: in the running codebase several of these internals are technically
public (batch reads are called directly by services, the primitives by tests).
The table above is the contract as designed — if you build this fresh, enforce the visibility
the keywords claim.
The optimization: handwritten SQL on the read path
The write path stays on the ORM — writes are per-entity and transaction-shaped, exactly what an ORM is good at. The read path does not: assembling an EAV entity is a pivot, and pivots are where ORMs go to die. The abstract class actually contains three generations of the read path, and the fossil record is instructive:
| Generation | How it read | Why it lost |
|---|---|---|
| 1 · per-field ORM loop | For each entry in fieldsMeta: one query for the field's attr rows, one per relation direction. Clean, obvious, metadata-driven. |
An entity with F fields costs ~1 + F queries; a page of 50 entities costs thousands of round-trips. This is the N+1 caveat from Part 1, institutionalized. |
| 2 · one big join | Query-builder join of entity × attrs × attr-names × relations in a single ORM query. | One round-trip, but the result set explodes multiplicatively (attrs × edges per entity), the window filters must be repeated on every joined alias, and the ORM hydrates thousands of duplicate objects to build one view. |
| 3 · batch CTE + pivot | Three handwritten queries for the whole batch of uuids, flat rows out, pivoted in application code. The current engine. | — |
The arithmetic that settled it: reading one entity with 8 fields went from ~17 queries to 3; a page of 50 entities went from ~850 to 4 (one count + the three batch queries). Same information, same database — the only thing that changed is who does the pivot.
Batch search — how findByIdsView actually works
The batch engine, step by step
- Guard the input. Every uuid is regex-validated and deduplicated before touching SQL — the TypeORM undefined-in-where lesson (Part 1's caveats), applied at the engine's front door since every read funnels through here.
- Three queries for N entities. One CTE fetches all attribute values for the whole uuid array (
WHERE e.uuid = ANY($1::uuid[])), two more fetch relations — one per edge direction. All three are family-scoped and window-filtered inside the SQL, and ordered (ORDER BY e_uuid, ev_active_from DESC) so the newest version of anything comes first. - Pivot in application code. The flat rows come back as one array; lodash
groupByby entity uuid, thenmapValues+groupByby field name rebuilds the per-entity, per-field structure. CPU-cheap, allocation-light — the expensive part (filtering millions of rows down to the active set) stayed in Postgres where the indexes are. - Handle the missing. Uuids that produced no rows are checked against the entity table: genuinely absent → skipped; present but inactive → re-read at
active_to − ε(the clamp from Part 1), inside the same engine. - Assemble Views.
fieldsMetadrives the final mapping — first active value per scalar, first active edge per relation,undefinedfor never-set — through the_createClass*constructors, so the output is typed per family.
The shape of the SQL and the pivot (full queries in Part 1)
-- one round-trip for the whole batch: $1 = uuid[], $2/$3 = timestamp, $4 = family
WITH targets AS (
SELECT e.uuid, e.active_from, e.active_to
FROM sm_entity e
WHERE e.uuid = ANY($1::uuid[])
),
entity_data AS (
SELECT t.uuid AS e_uuid,
ef.name AS ef_attr_name, ef.type AS ef_attr_type,
ev.val_str, ev.val_int, ev.val_num, ev.val_date,
ev.active_from AS ev_active_from, ev.active_to AS ev_active_to
FROM targets t
JOIN sm_entity_attr ev ON ev.fk_sm_entity_uuid = t.uuid
JOIN sm_entity_attr_name ef ON ef.uuid = ev.fk_sm_entity_attr_name_uuid
WHERE /* entity + attr [from, to) window filters, family scope */
ORDER BY e_uuid, ev_active_from DESC
)
SELECT * FROM entity_data;
-- (+ two sibling queries for child-of / parent-of edges)
// the pivot — flat rows back into entities, in one pass
const byEntity = groupBy(rawRows, 'e_uuid');
const grouped = mapValues(byEntity, rows => ({
uuid: rows[0].e_uuid,
...mapValues(groupBy(rows, 'ef_attr_name'),
fieldRows => fieldRows.map(toAttrView)), // newest first
}));
Two design points make this an abstract-class optimization rather than a one-off fast
query. First, the SQL is metadata-parameterized — family id, timestamps and the
uuid array are the only inputs, so every domain type gets the fast path without writing any SQL
of its own. Second, it is the single funnel: findByIdView is a
batch of one, and findAllPaginated pages bare entity ids first and hydrates the
page through the same engine — one implementation to index for, one to profile, one to get
right. The known residue of generation 1 (Part 1's pagination N+1 caveat) is the roadmap item;
the engine to fix it with already exists.
A minimal concrete repository
This is the payoff — the entire cost of a new domain type. Everything else in this article is inherited.
ProductRepository — a complete domain type
@Injectable()
export class ProductRepository
extends SmentityRepository<CreateProductDTO, Product, ProductView, ProductHistory>
implements IProductRepository
{
constructor(
@InjectDataSource() dataSource: DataSource,
@InjectRepository(Smentity) smentityRepository: Repository<Smentity>,
@InjectRepository(SmentityAttr) smentityAttrRepository: Repository<SmentityAttr>,
@InjectRepository(SmentityAttrName) smentityAttrNameRepository: Repository<SmentityAttrName>,
@InjectRepository(SmentityFamily) smentityFamilyRepository: Repository<SmentityFamily>,
@InjectRepository(SmentityRel) smEntityRelRepository: Repository<SmentityRel>,
@InjectRepository(SmCustomerEntity) smCustomerEntity: Repository<SmCustomerEntity>,
) {
super(CreateProductDTO, Product, ProductView, ProductHistory,
dataSource, smentityRepository, smentityAttrRepository,
smentityAttrNameRepository, smentityFamilyRepository,
smEntityRelRepository, smCustomerEntity);
this.familyId = SM_ENTITY_PRODUCT_FAMILY; // e.g. 2
this.familyName = 'product';
this.fieldsMeta = [
{
field: 'name', type: TYPE_STR,
validators: [{
name: 'Validate `name`',
exec: async (_queryRunner, value: string) => value?.trim().length > 0,
}],
},
{ field: 'sku', type: TYPE_STR, validators: [] },
{ field: 'price', type: TYPE_NUM, validators: [positive] },
{ field: 'order', type: TYPE_INT, validators: [] },
{
field: 'parentCategory', type: TYPE_REL_UUID_CHILD_OF,
validators: [categoryExistsInFamily], // async, runs in the write tx
},
];
}
}
Wiring it into NestJS
Consumers depend on an interface and a string token, never on the class. Swapping an implementation — or mocking one in tests — is a one-line provider change.
Module wiring — tokens, interfaces, providers
// constants
export const SM_PRODUCT_REPOSITORY = 'SmProductRepository';
// module
const productRepositoryProvider: ClassProvider = {
provide: SM_PRODUCT_REPOSITORY,
useClass: ProductRepository,
};
@Module({
imports: [TypeOrmModule.forFeature([
Smentity, SmentityAttr, SmentityAttrName,
SmentityFamily, SmentityRel, SmCustomerEntity,
])],
providers: [productRepositoryProvider /*, productServiceProvider, ... */],
exports: [productRepositoryProvider],
})
export class SmentityModule {}
// consumer (a service holding the business rules)
@Injectable()
export class ProductService {
constructor(
@Inject(SM_PRODUCT_REPOSITORY)
private readonly products: IProductRepository,
) {}
}
Where a write actually goes
What each method actually does
The algorithms behind the CRUD surface, one expandable walkthrough per method — every step
driven by fieldsMeta, every write obeying Part 1's behavior matrix.
create(dto, tx?) — one transaction, six steps
- Resolve the transaction: join the caller's
txif one was passed, otherwise open its ownQueryRunnertransaction. - Resolve the family row; create it on first use (lazy, cached for 2s).
- For each declared field: pick the DTO value (or default), run its validators — a failed validator throws 422 and rolls back everything.
- Resolve each field's attr-name row; create on first use behind the unique index.
- Build the graph in memory — entity row, one typed attr row per scalar, one edge row per relation, the customer binding — and save it in one cascade.
- Run entity-level validators, commit, and return the freshly-read View.
findByIdView(uuid, timestamp?, tx?) — read one at a moment
- Join the caller's
txor borrow a connection; defaulttimestampto now. - Guard the uuid with a regex before it reaches a query — the TypeORM undefined-in-where lesson from Part 1's caveats.
- Run the batch attribute CTE and both relation CTEs at
timestamp(this isfindByIdsViewwith a single uuid — one engine, no special case). - Pivot the flat rows: group by entity uuid, then by field name (lodash
groupBy/mapValues). - Nothing came back? Check the entity row itself — if it is inactive, re-read at
active_to − εto render its last state. - Assemble the View through
fieldsMeta: one current value per scalar, first active edge per relation,undefinedfor fields with no active row.
findAllPaginated(page, search?, tx?) — read a page
- Count and page the bare entity rows only — family-scoped, ordered by
active_from, offset/limit from the page options. No attribute joins yet. - Build the page metadata (total, page count) from the count.
- Hydrate just the page's uuids through the batch read engine — one CTE pass for the whole page, not one query per row (the N+1 caveat, enforced structurally).
- Return page metadata + hydrated Views.
entityHistory(uuid, tx?) — read the audit timeline
- Collect every
DISTINCT active_from/active_tofrom the entity's attribute rows and from every edge touching it. - Union the timestamps into a set and sort chronologically — each one is an instant where something changed.
- For each instant
T, for each declared field: before = the row closed at exactlyT, after = the row opened at exactlyT. - Pair them into a diff
{field, before, after}; skip fields with neither. No diffing logic — the temporal model stores changes natively. - Return the instants newest-first: the entity's full audit trail, straight from storage.
update(entity, changes, tx?) — the temporal update
- Resolve the transaction: join the caller's
txor open one — the whole update is atomic, never per-field. - Walk
fieldsMetaand dispatch each field: scalars →tFieldUpdate(close the old row, open the new one at a single transition instant — the scalar half of the matrix), relations →tRelUpdate(close the current edge, open the new one; "move, don't add"; null is a no-op), tag sets → attach/detach by set difference. - Run each field's validators before its write; any failure anywhere rolls back every field — no partially-updated entities.
- Re-read the entity at now through the batch engine, inside the same transaction.
- Commit and return the fresh View.
close(uuid, timestamp, tx?) — soft delete
- Resolve the transaction: join the caller's
txor open one. - Load the entity row and set
active_to = timestamp(default now). That is the entire write. - Attribute rows and edges are deliberately untouched — history stays intact, and point-in-time reads before
timestampstill work; reads after it clamp toactive_to − ε. - Commit. The batch variant is the same loop over a list of uuids inside one transaction.
Caveats
Part 1's caveats were about the storage semantics; these are about the class that implements them — each one paid for in the code.
- Generate the transaction timestamp once, on the backend. The entity's
active_fromand every field'sactive_frommust be the same instant. Let the database default each row (CURRENT_TIMESTAMP) and they drift apart by fractions of a millisecond per row — the entity becomes visible at a moment when its fields don't exist yet, and the atomic close-and-open handover stops being atomic.createandupdatecompute one timestamp at the top and stamp every row they touch with it; the column defaults exist as a fallback, never as the mechanism. - Re-derive repositories from the transaction. The injected
Repositoryinstances are bound to the default connection. Inside a transaction you must get fresh ones from theEntityManager(entityManager.getRepository(Smentity)) — a query on the injected instance silently escapes the transaction and commits on its own. - Commit only the transaction you opened. A method that received a caller's
txmust never commit, roll back or release it — that right belongs to whoever opened it. ThequeryRunner?.commitTransaction()optional-chaining guard in every write path is the visible trace of this rule: the runner is only non-null when the method created it. - Runtime constructors must match the generics. TypeScript types erase at runtime, so the class news up DTOs, Views and Histories from the four constructor arguments. A mismatch between the generic list and the constructor list compiles cleanly and fails only at runtime — keep them in the same order, always.
- Guard every dynamic where clause.
findOneBy({ uuid: undefined })drops the predicate and returns the table's first row (typeorm#9316) — and in a shared-table design, "the first row" is some other family's entity. Null-check every lookup built from input before it reaches the query builder. - Validators run inside the write transaction. That is what makes them consistent — and also what makes them dangerous: a slow validator holds the transaction (and its locks) open. Keep validator queries cheap; and note that a validator that throws is caught, logged and treated as a plain failure — so the real cause lives in the logs, not in the 422.
- Lazy creation plus cache equals races. Family and attr-name rows are find-or-create with a 2-second query cache; two concurrent first-writes can both miss and both insert. The unique index makes the loser fail loudly — add the retry, or pre-seed the names in a migration (Part 1's checklist).
- Wrap field-level errors with context. A failure deep in the EAV machinery surfaces as an anonymous constraint violation. The update path re-wraps every field error with the entity uuid and field name (
Field update error entity: …, name(str)) before rethrowing — without that, production debugging in a generic store is guesswork.
The abstract class is the design's fixed cost, paid once. Every domain type after it is a constructor, two constants and a field list.
Before you implement: the decision checklist
Same rule as Part 1's storage checklist: none of these have a universally right answer, all of them have a wrong one — deciding by accident. Settle each before the first subclass exists.
- Abstract class, or a real layer. First and foremost. Inheritance: concrete repositories extend the machinery — one
extends, onesuper()call, and every behavior is just there. Composition: a standalone EAV engine injected into thin per-family repositories — cleaner boundaries, easier to test in isolation, but every method becomes delegation plumbing and the four-generic typing gets awkward at the seam. (my choice — abstract class: the entire goal is that a domain type is a constructor, two constants and a field list; a delegation layer would double that cost for boundary purity nobody asked for) - What
txactually is. AnEntityManager, aQueryRunner, or an opaque wrapper you control? Whatever you pick, every method signature carries it — changing later touches everything. (my choice — derived from TypeORM'sEntityManagerfor now, but namedtxso the contract survives a future unwrap) - Generics or loose records. Four type parameters plus four runtime constructors is real ceremony — the alternative is untyped records and casting at every call site. (my choice — the generics: the ceremony is paid once per family, the type safety is collected on every call)
- Where the virtual schema lives.
fieldsMetain the subclass constructor, or externalized to config / a database table? External means runtime-editable fields; in-code means every field change is a reviewed, versioned commit. (my choice — in code: the virtual schema is the API contract, and contracts go through PRs) - Timestamp authority. Who mints the transaction instant — the backend or the database defaults? This is caveat #1; it only looks cosmetic until the first drift bug. (my choice — backend, one timestamp per transaction, stamped on every row it touches)
- One read engine or per-method queries. Every read path delegating to a single batch engine keeps the CTEs in one place and makes N+1 structurally hard; per-method queries are simpler to write and diverge immediately. (my choice — one engine:
findByIdViewisfindByIdsViewwith one uuid, no special cases) - Visibility discipline. Will you actually enforce the public-CRUD / private-EAV split, or let internals leak because TypeScript makes leaking easy? (my choice — designed as a strict split; honestly enforced only partially in the running code, which is exactly why it's on this list)
- DI binding. Concrete classes, or string tokens behind interfaces? Tokens cost a constants file and buy mockability and swappable implementations. (my choice — tokens + interfaces everywhere; no consumer names a concrete repository)
- Subclass override policy. When may a family override a base method instead of just declaring metadata? Unlimited overriding quietly forks the machinery per family. (my choice — metadata first; overriding is the exception and each override must justify itself in review)
- Error surfacing. What a failed validator returns to the API (and what it leaves in the logs), and how deep EAV errors get wrapped with entity/field context. (my choice — validators throw 422 with the validator's name; every field-level error is re-wrapped with entity uuid + field before it climbs)
Part 3
The layer above — services owning business rules and cross-entity transactions, the validation ladder, and controllers that expose point-in-time reads — is covered in Part 3 — Services and the API edge.