Andrei Vogulkin
← All writing
Research notes

A temporal EAV entity store in Postgres
Part 2 — The abstract repository

Part 1 defined the storage: six tables, half-open time windows, and a behavior matrix for every update. This part is about the code that makes the storage usable — one abstract repository class that owns all the machinery, so that a concrete domain type is nothing but a family id and a field list.

Goal

Write the hard parts exactly once. Transactions, the close-and-open temporal dance, relation handling, validators, batch reads, pagination, history — all of it lives in one abstract class. A new domain type ("product", "category", "promo campaign") subclasses it, declares its fields as metadata, and gets a complete CRUD interface for free: create, point-in-time read (one, list, history), temporal update, and soft delete — identical in shape across every domain type, without writing a single query. Consumers speak CRUD in domain terms; only the inside of the class knows the storage is EAV.

The stack

  • PostgreSQL — the six tables from Part 1; the only stateful thing in the design.
  • TypeORM — entity mapping, and more importantly DataSource / QueryRunner / EntityManager for explicit transactions, plus an escape hatch for raw CTE queries on the read path.
  • NestJS — dependency injection: repositories are bound to string tokens behind interfaces, so consumers never name a concrete class.
  • TypeScript generics — the abstract class is parameterized over four types (DTO, Entity, View, History), which is what lets one implementation return correctly-typed results for every domain.
  • lodash — groupBy / mapValues pivot the flat raw SQL rows back into per-entity, per-field structures on the batch read path.

The shape

One class, four type parameters — the four shapes a domain object takes on its way through the system. The DTO comes in from the API; the View goes out with one current value per field; the History goes out with the full value timeline per field.

Class signature — four generics, one machine
@Injectable()
export abstract class SmentityRepository<
  EntityDTO,                          // create payload (API in)
  Entity extends BaseSmentity,        // flat domain object
  EntityView extends BaseSmentity,    // read model: current value per field
  EntityHistory extends BaseSmentity, // read model: value timeline per field
> implements IGenericSmentityRepository<EntityDTO, Entity, EntityView, EntityHistory>
{
  // ...
}

Minimal fields

The entire state of the class. The first four are the contract a subclass fulfills; the rest is injected plumbing shared by every domain type.

FieldWho sets itRole
familyId / familyName subclass Which family this repository owns. Scopes every query the base class ever runs — a repository can never see another family's rows.
fieldsMeta subclass The virtual schema (Part 1's term): one entry per field — storage name, type, validators, optional default and DTO mapping. This list drives create, update, read assembly and history.
entityValidators subclass Cross-field checks that need the whole object (single-field checks belong to the field's own validators).
_createClass* (×4) subclass → constructor The four concrete class constructors matching the generics. TypeScript types vanish at runtime, so the base class needs real constructors to new up DTOs, Views and Histories.
dataSource injected Creates QueryRunners — every write runs in an explicit transaction, every batch read on one connection.
6 × Repository<…> injected TypeORM repositories for the six tables. Inside a transaction the base class re-derives them from the EntityManager so all statements share the transaction.
IFieldMeta — one virtual schema entry
interface IFieldMeta {
  field: string;      // storage name: 'name', 'price', 'parentCategory', ...
  type: string;       // TYPE_STR | TYPE_INT | TYPE_NUM | TYPE_BOOL
                      //   | TYPE_REL_UUID_CHILD_OF | TYPE_REL_UUID_PARENT_OF
                      //   | TYPE_TAG_SET
  validators: FieldValidator[];  // async checks, run inside the write tx
  default?: unknown;  // used when the DTO omits the field
  dtoField?: string;  // DTO property name, when it differs from `field`
}

interface FieldValidator {
  name: string;       // shows up in the 422 when the check fails
  exec: (queryRunner: QueryRunner, value: unknown) => Promise<boolean>;
}

Minimal methods

Split by visibility — public vs private. Strict visibility isn't a popular discipline in TypeScript codebases, but here the split is the design: the public surface speaks CRUD in domain terms, the private one speaks EAV. Nothing temporal or EAV-shaped may leak upward — a consumer should be unable to tell the storage isn't a plain table. The public set below is also deliberately deduplicated to the minimal full-CRUD contract: one method per operation (e.g. a single paginated list read — a separate findAll is the same read with the paging left out, so it doesn't earn a slot; same for batch variants of close).

One more rule: every method accepts an optional tx — a transaction handle — reads included. Passed one, the method joins the caller's transaction; omitted, it opens its own (writes) or borrows a connection (reads). For now tx is derived from TypeORM's EntityManager — inside the class it is unwrapped back to an EntityManager/QueryRunner pair — but the parameter is named for what it means, not what it currently is. This single rule is what lets a service compose creates, updates and relation changes across several repositories into one atomic commit.

Public — the CRUD surface

MethodCRUDWhat it does
create(dto, tx?) C One transaction: run the virtual schema validators, open the entity row and one attr/rel row per declared field. Joins the caller's tx when given; opens its own otherwise.
findByIdView(uuid, timestamp?, tx?) R — one Point-in-time read of one entity → View: current value per field at timestamp (default now), with the inactive-entity clamp applied.
findAllPaginated(page, search?, tx?) R — list Family-scoped listing, hydrated through the batch read engine. The one list read: paging is a parameter, not a second method.
entityHistory(uuid, tx?) R — audit The value timeline: a before/after diff per field per change instant, straight from the temporal windows.
update(entity, changes, tx?) U The temporal update: walks fieldsMeta, dispatches each changed field to the right private primitive inside the caller's tx (or its own), re-reads and returns the fresh View. (In the codebase this is tEntityFieldsUpdate — the CRUD name it deserves is update.)
close(uuid, timestamp, tx?) D Soft delete, Part 1 style: sets the entity's active_to; history stays readable. A batch variant is the same operation over a list, not a second concept.

Private — the EAV machinery

MethodWhat it does
tFieldUpdate(oldField, newField, fieldMeta, tx) The scalar temporal primitive: close row A, open row B. The entire scalar half of Part 1's behavior matrix lives here.
tRelUpdate(oldRel, newRel, fieldMeta, entityUuid, tx) The relation primitive: close the current edge, open the new one — "move, don't add", the null no-op, and root handling.
findByIdsView(uuids, timestamp?, tx?) The batch read engine every public read delegates to: the raw CTE queries from Part 1 (attributes + both relation directions), pivoted with lodash into Views; detects inactive entities and re-reads them at active_to − ε.
_getBatchFieldQuery / _getRelationBatchFieldQuery The raw SQL builders — the exact CTEs shown in Part 1, parameterized by family and timestamp.
runValidators(queryRunner, validators, value) Executes the virtual schema inside the write transaction; a failed check throws 422 and rolls everything back.
smEntityToView / ToDTO / ToHistory The three transformers between storage rows and the typed shapes, driven by fieldsMeta and the _createClass* constructors.

Honest footnote: in the running codebase several of these internals are technically public (batch reads are called directly by services, the primitives by tests). The table above is the contract as designed — if you build this fresh, enforce the visibility the keywords claim.

The optimization: handwritten SQL on the read path

The write path stays on the ORM — writes are per-entity and transaction-shaped, exactly what an ORM is good at. The read path does not: assembling an EAV entity is a pivot, and pivots are where ORMs go to die. The abstract class actually contains three generations of the read path, and the fossil record is instructive:

GenerationHow it readWhy it lost
1 · per-field ORM loop For each entry in fieldsMeta: one query for the field's attr rows, one per relation direction. Clean, obvious, metadata-driven. An entity with F fields costs ~1 + F queries; a page of 50 entities costs thousands of round-trips. This is the N+1 caveat from Part 1, institutionalized.
2 · one big join Query-builder join of entity × attrs × attr-names × relations in a single ORM query. One round-trip, but the result set explodes multiplicatively (attrs × edges per entity), the window filters must be repeated on every joined alias, and the ORM hydrates thousands of duplicate objects to build one view.
3 · batch CTE + pivot Three handwritten queries for the whole batch of uuids, flat rows out, pivoted in application code. The current engine. —

The arithmetic that settled it: reading one entity with 8 fields went from ~17 queries to 3; a page of 50 entities went from ~850 to 4 (one count + the three batch queries). Same information, same database — the only thing that changed is who does the pivot.

Batch search — how findByIdsView actually works

The batch engine, step by step
  1. Guard the input. Every uuid is regex-validated and deduplicated before touching SQL — the TypeORM undefined-in-where lesson (Part 1's caveats), applied at the engine's front door since every read funnels through here.
  2. Three queries for N entities. One CTE fetches all attribute values for the whole uuid array (WHERE e.uuid = ANY($1::uuid[])), two more fetch relations — one per edge direction. All three are family-scoped and window-filtered inside the SQL, and ordered (ORDER BY e_uuid, ev_active_from DESC) so the newest version of anything comes first.
  3. Pivot in application code. The flat rows come back as one array; lodash groupBy by entity uuid, then mapValues + groupBy by field name rebuilds the per-entity, per-field structure. CPU-cheap, allocation-light — the expensive part (filtering millions of rows down to the active set) stayed in Postgres where the indexes are.
  4. Handle the missing. Uuids that produced no rows are checked against the entity table: genuinely absent → skipped; present but inactive → re-read at active_to − ε (the clamp from Part 1), inside the same engine.
  5. Assemble Views. fieldsMeta drives the final mapping — first active value per scalar, first active edge per relation, undefined for never-set — through the _createClass* constructors, so the output is typed per family.
The shape of the SQL and the pivot (full queries in Part 1)
-- one round-trip for the whole batch: $1 = uuid[], $2/$3 = timestamp, $4 = family
WITH targets AS (
  SELECT e.uuid, e.active_from, e.active_to
  FROM sm_entity e
  WHERE e.uuid = ANY($1::uuid[])
),
entity_data AS (
  SELECT t.uuid AS e_uuid,
         ef.name AS ef_attr_name, ef.type AS ef_attr_type,
         ev.val_str, ev.val_int, ev.val_num, ev.val_date,
         ev.active_from AS ev_active_from, ev.active_to AS ev_active_to
  FROM targets t
  JOIN sm_entity_attr ev      ON ev.fk_sm_entity_uuid = t.uuid
  JOIN sm_entity_attr_name ef ON ef.uuid = ev.fk_sm_entity_attr_name_uuid
  WHERE /* entity + attr [from, to) window filters, family scope */
  ORDER BY e_uuid, ev_active_from DESC
)
SELECT * FROM entity_data;
-- (+ two sibling queries for child-of / parent-of edges)

// the pivot — flat rows back into entities, in one pass
const byEntity = groupBy(rawRows, 'e_uuid');
const grouped  = mapValues(byEntity, rows => ({
  uuid: rows[0].e_uuid,
  ...mapValues(groupBy(rows, 'ef_attr_name'),
               fieldRows => fieldRows.map(toAttrView)),  // newest first
}));

Two design points make this an abstract-class optimization rather than a one-off fast query. First, the SQL is metadata-parameterized — family id, timestamps and the uuid array are the only inputs, so every domain type gets the fast path without writing any SQL of its own. Second, it is the single funnel: findByIdView is a batch of one, and findAllPaginated pages bare entity ids first and hydrates the page through the same engine — one implementation to index for, one to profile, one to get right. The known residue of generation 1 (Part 1's pagination N+1 caveat) is the roadmap item; the engine to fix it with already exists.

A minimal concrete repository

This is the payoff — the entire cost of a new domain type. Everything else in this article is inherited.

ProductRepository — a complete domain type
@Injectable()
export class ProductRepository
  extends SmentityRepository<CreateProductDTO, Product, ProductView, ProductHistory>
  implements IProductRepository
{
  constructor(
    @InjectDataSource() dataSource: DataSource,
    @InjectRepository(Smentity) smentityRepository: Repository<Smentity>,
    @InjectRepository(SmentityAttr) smentityAttrRepository: Repository<SmentityAttr>,
    @InjectRepository(SmentityAttrName) smentityAttrNameRepository: Repository<SmentityAttrName>,
    @InjectRepository(SmentityFamily) smentityFamilyRepository: Repository<SmentityFamily>,
    @InjectRepository(SmentityRel) smEntityRelRepository: Repository<SmentityRel>,
    @InjectRepository(SmCustomerEntity) smCustomerEntity: Repository<SmCustomerEntity>,
  ) {
    super(CreateProductDTO, Product, ProductView, ProductHistory,
      dataSource, smentityRepository, smentityAttrRepository,
      smentityAttrNameRepository, smentityFamilyRepository,
      smEntityRelRepository, smCustomerEntity);

    this.familyId = SM_ENTITY_PRODUCT_FAMILY;   // e.g. 2
    this.familyName = 'product';
    this.fieldsMeta = [
      {
        field: 'name', type: TYPE_STR,
        validators: [{
          name: 'Validate `name`',
          exec: async (_queryRunner, value: string) => value?.trim().length > 0,
        }],
      },
      { field: 'sku',   type: TYPE_STR, validators: [] },
      { field: 'price', type: TYPE_NUM, validators: [positive] },
      { field: 'order', type: TYPE_INT, validators: [] },
      {
        field: 'parentCategory', type: TYPE_REL_UUID_CHILD_OF,
        validators: [categoryExistsInFamily],   // async, runs in the write tx
      },
    ];
  }
}

Wiring it into NestJS

Consumers depend on an interface and a string token, never on the class. Swapping an implementation — or mocking one in tests — is a one-line provider change.

Module wiring — tokens, interfaces, providers
// constants
export const SM_PRODUCT_REPOSITORY = 'SmProductRepository';

// module
const productRepositoryProvider: ClassProvider = {
  provide: SM_PRODUCT_REPOSITORY,
  useClass: ProductRepository,
};

@Module({
  imports: [TypeOrmModule.forFeature([
    Smentity, SmentityAttr, SmentityAttrName,
    SmentityFamily, SmentityRel, SmCustomerEntity,
  ])],
  providers: [productRepositoryProvider /*, productServiceProvider, ... */],
  exports: [productRepositoryProvider],
})
export class SmentityModule {}

// consumer (a service holding the business rules)
@Injectable()
export class ProductService {
  constructor(
    @Inject(SM_PRODUCT_REPOSITORY)
    private readonly products: IProductRepository,
  ) {}
}

Where a write actually goes

ProductService business rules, transactions IProductRepository token: SM_PRODUCT_REPOSITORY ProductRepository familyId + fieldsMeta only SmentityRepository abstract: all the machinery TypeORM QueryRunner / raw SQL PostgreSQL the six tables @Inject(token) DI resolves extends

What each method actually does

The algorithms behind the CRUD surface, one expandable walkthrough per method — every step driven by fieldsMeta, every write obeying Part 1's behavior matrix.

create(dto, tx?) — one transaction, six steps
  1. Resolve the transaction: join the caller's tx if one was passed, otherwise open its own QueryRunner transaction.
  2. Resolve the family row; create it on first use (lazy, cached for 2s).
  3. For each declared field: pick the DTO value (or default), run its validators — a failed validator throws 422 and rolls back everything.
  4. Resolve each field's attr-name row; create on first use behind the unique index.
  5. Build the graph in memory — entity row, one typed attr row per scalar, one edge row per relation, the customer binding — and save it in one cascade.
  6. Run entity-level validators, commit, and return the freshly-read View.
findByIdView(uuid, timestamp?, tx?) — read one at a moment
  1. Join the caller's tx or borrow a connection; default timestamp to now.
  2. Guard the uuid with a regex before it reaches a query — the TypeORM undefined-in-where lesson from Part 1's caveats.
  3. Run the batch attribute CTE and both relation CTEs at timestamp (this is findByIdsView with a single uuid — one engine, no special case).
  4. Pivot the flat rows: group by entity uuid, then by field name (lodash groupBy/mapValues).
  5. Nothing came back? Check the entity row itself — if it is inactive, re-read at active_to − ε to render its last state.
  6. Assemble the View through fieldsMeta: one current value per scalar, first active edge per relation, undefined for fields with no active row.
findAllPaginated(page, search?, tx?) — read a page
  1. Count and page the bare entity rows only — family-scoped, ordered by active_from, offset/limit from the page options. No attribute joins yet.
  2. Build the page metadata (total, page count) from the count.
  3. Hydrate just the page's uuids through the batch read engine — one CTE pass for the whole page, not one query per row (the N+1 caveat, enforced structurally).
  4. Return page metadata + hydrated Views.
entityHistory(uuid, tx?) — read the audit timeline
  1. Collect every DISTINCT active_from / active_to from the entity's attribute rows and from every edge touching it.
  2. Union the timestamps into a set and sort chronologically — each one is an instant where something changed.
  3. For each instant T, for each declared field: before = the row closed at exactly T, after = the row opened at exactly T.
  4. Pair them into a diff {field, before, after}; skip fields with neither. No diffing logic — the temporal model stores changes natively.
  5. Return the instants newest-first: the entity's full audit trail, straight from storage.
update(entity, changes, tx?) — the temporal update
  1. Resolve the transaction: join the caller's tx or open one — the whole update is atomic, never per-field.
  2. Walk fieldsMeta and dispatch each field: scalars → tFieldUpdate (close the old row, open the new one at a single transition instant — the scalar half of the matrix), relations → tRelUpdate (close the current edge, open the new one; "move, don't add"; null is a no-op), tag sets → attach/detach by set difference.
  3. Run each field's validators before its write; any failure anywhere rolls back every field — no partially-updated entities.
  4. Re-read the entity at now through the batch engine, inside the same transaction.
  5. Commit and return the fresh View.
close(uuid, timestamp, tx?) — soft delete
  1. Resolve the transaction: join the caller's tx or open one.
  2. Load the entity row and set active_to = timestamp (default now). That is the entire write.
  3. Attribute rows and edges are deliberately untouched — history stays intact, and point-in-time reads before timestamp still work; reads after it clamp to active_to − ε.
  4. Commit. The batch variant is the same loop over a list of uuids inside one transaction.

Caveats

Part 1's caveats were about the storage semantics; these are about the class that implements them — each one paid for in the code.

  • Generate the transaction timestamp once, on the backend. The entity's active_from and every field's active_from must be the same instant. Let the database default each row (CURRENT_TIMESTAMP) and they drift apart by fractions of a millisecond per row — the entity becomes visible at a moment when its fields don't exist yet, and the atomic close-and-open handover stops being atomic. create and update compute one timestamp at the top and stamp every row they touch with it; the column defaults exist as a fallback, never as the mechanism.
  • Re-derive repositories from the transaction. The injected Repository instances are bound to the default connection. Inside a transaction you must get fresh ones from the EntityManager (entityManager.getRepository(Smentity)) — a query on the injected instance silently escapes the transaction and commits on its own.
  • Commit only the transaction you opened. A method that received a caller's tx must never commit, roll back or release it — that right belongs to whoever opened it. The queryRunner?.commitTransaction() optional-chaining guard in every write path is the visible trace of this rule: the runner is only non-null when the method created it.
  • Runtime constructors must match the generics. TypeScript types erase at runtime, so the class news up DTOs, Views and Histories from the four constructor arguments. A mismatch between the generic list and the constructor list compiles cleanly and fails only at runtime — keep them in the same order, always.
  • Guard every dynamic where clause. findOneBy({ uuid: undefined }) drops the predicate and returns the table's first row (typeorm#9316) — and in a shared-table design, "the first row" is some other family's entity. Null-check every lookup built from input before it reaches the query builder.
  • Validators run inside the write transaction. That is what makes them consistent — and also what makes them dangerous: a slow validator holds the transaction (and its locks) open. Keep validator queries cheap; and note that a validator that throws is caught, logged and treated as a plain failure — so the real cause lives in the logs, not in the 422.
  • Lazy creation plus cache equals races. Family and attr-name rows are find-or-create with a 2-second query cache; two concurrent first-writes can both miss and both insert. The unique index makes the loser fail loudly — add the retry, or pre-seed the names in a migration (Part 1's checklist).
  • Wrap field-level errors with context. A failure deep in the EAV machinery surfaces as an anonymous constraint violation. The update path re-wraps every field error with the entity uuid and field name (Field update error entity: …, name(str)) before rethrowing — without that, production debugging in a generic store is guesswork.
The abstract class is the design's fixed cost, paid once. Every domain type after it is a constructor, two constants and a field list.

Before you implement: the decision checklist

Same rule as Part 1's storage checklist: none of these have a universally right answer, all of them have a wrong one — deciding by accident. Settle each before the first subclass exists.

  • Abstract class, or a real layer. First and foremost. Inheritance: concrete repositories extend the machinery — one extends, one super() call, and every behavior is just there. Composition: a standalone EAV engine injected into thin per-family repositories — cleaner boundaries, easier to test in isolation, but every method becomes delegation plumbing and the four-generic typing gets awkward at the seam. (my choice — abstract class: the entire goal is that a domain type is a constructor, two constants and a field list; a delegation layer would double that cost for boundary purity nobody asked for)
  • What tx actually is. An EntityManager, a QueryRunner, or an opaque wrapper you control? Whatever you pick, every method signature carries it — changing later touches everything. (my choice — derived from TypeORM's EntityManager for now, but named tx so the contract survives a future unwrap)
  • Generics or loose records. Four type parameters plus four runtime constructors is real ceremony — the alternative is untyped records and casting at every call site. (my choice — the generics: the ceremony is paid once per family, the type safety is collected on every call)
  • Where the virtual schema lives. fieldsMeta in the subclass constructor, or externalized to config / a database table? External means runtime-editable fields; in-code means every field change is a reviewed, versioned commit. (my choice — in code: the virtual schema is the API contract, and contracts go through PRs)
  • Timestamp authority. Who mints the transaction instant — the backend or the database defaults? This is caveat #1; it only looks cosmetic until the first drift bug. (my choice — backend, one timestamp per transaction, stamped on every row it touches)
  • One read engine or per-method queries. Every read path delegating to a single batch engine keeps the CTEs in one place and makes N+1 structurally hard; per-method queries are simpler to write and diverge immediately. (my choice — one engine: findByIdView is findByIdsView with one uuid, no special cases)
  • Visibility discipline. Will you actually enforce the public-CRUD / private-EAV split, or let internals leak because TypeScript makes leaking easy? (my choice — designed as a strict split; honestly enforced only partially in the running code, which is exactly why it's on this list)
  • DI binding. Concrete classes, or string tokens behind interfaces? Tokens cost a constants file and buy mockability and swappable implementations. (my choice — tokens + interfaces everywhere; no consumer names a concrete repository)
  • Subclass override policy. When may a family override a base method instead of just declaring metadata? Unlimited overriding quietly forks the machinery per family. (my choice — metadata first; overriding is the exception and each override must justify itself in review)
  • Error surfacing. What a failed validator returns to the API (and what it leaves in the logs), and how deep EAV errors get wrapped with entity/field context. (my choice — validators throw 422 with the validator's name; every field-level error is re-wrapped with entity uuid + field before it climbs)

Part 3

The layer above — services owning business rules and cross-entity transactions, the validation ladder, and controllers that expose point-in-time reads — is covered in Part 3 — Services and the API edge.

← Part 1 — Storage concept Work with me →