Chris M.Reviewed
Under the Bonnet: Waiting, and Where Data Lives
This is part three of the AI and Software Architecture Translation Guide, the six part series on how modern AI systems are actually assembled. Part one and part two decided what a task needs and which model tier, if any, should handle it. Part three covers two things underneath that decision: how a system behaves while it waits on a slow step, and how it should actually store the data it is working with.
Waiting is not the same as working
A construction system spends most of its life waiting: for a database, for file storage, for an external API, for an AI service to respond. Node.js, the runtime behind a large share of modern web and API backends, is particularly suited to this because its event driven design lets an application keep working while those waits play out, rather than standing still until one finishes.
A caveat worth carrying, because earlier material on this subject overstated it: this is not literally one thread performing thousands of jobs at once. The runtime also relies on operating system facilities and background worker mechanisms to do real parallel work. The useful idea is not the thread count. It is that waiting is not the same as working, and a well designed system does not stand still during a wait.
A poor workflow is a manager who gives one instruction and then stands beside the trade until it is finished. A non-blocking workflow is a project manager who issues several independent tasks, records who owns each one, and responds as each one reports back. The second manager is not working harder. They have stopped treating waiting as work.
Picture fifty site managers submitting inspections within the same minute. Each submission is accepted immediately. Behind that acceptance, the system stores photographs, writes the database record, sends the text to an AI service, generates thumbnails and updates a dashboard as separate operations. Nothing on site is held up by an AI call that takes four seconds.
What to ask a supplier. What happens to a submission if the AI service is slow or unavailable? The right answer is that the record is captured regardless, and the AI step retries or queues behind it. The wrong answer is that the user sees an error and loses their entry. On a site with poor signal, that one design decision determines whether people actually use the system.
Where the data itself should live
Relational databases such as PostgreSQL organise data into defined tables and relationships. Document databases allow more flexible, document shaped records. The real distinction is not rigid against flexible. Both require design. The difference is where the design effort sits.
Think of relational data as a controlled project register: package, company, drawing number, status, date. Every row carries the same fields, so the register can be trusted for reporting, ledgers and payment. A JSONB column, a feature of PostgreSQL that stores structured data inside a single field, works as a structured appendix attached to each record, where information specific to a package can vary without redesigning the whole register.
A fire door, an air handling unit and a lightning protection test point have almost nothing in common as data. A door has a fire rating and an ironmongery set. An air handling unit has a flow rate and a filter grade. A test point has a resistance reading and a test date. Force all of that into one rigid table and you end up with hundreds of mostly empty columns. Abandon structure entirely and you lose the ability to run a payment application or a compliance report against it. PostgreSQL's JSONB type lets a system keep identifiers, money and status in strict columns, where they can be validated and joined, while package specific technical properties sit in a flexible field that is still queryable in ordinary SQL. That single pattern removes most of the pressure to choose a side.
| Put it in a controlled column when | Put it in JSONB when |
|---|---|
| It appears on every record, is reported on, joined to, or carries money, dates or accountability | It varies by package, trade or product type, and would otherwise create a mostly empty column |
A warning sign worth watching for. If a JSONB field starts appearing in every payment report, it has earned a column. Promote it, rather than leaving a load-bearing field buried in an unstructured appendix.
An asset register keeps asset ID, location, package and status as controlled columns, while a JSONB field stores variable properties: fire rating for a door, flow rate for an air handling unit, warranty details for specialist equipment. The commercial team runs its reporting off the columns. The technical team searches the appendix. Neither blocks the other, because the two live in the same row without competing for the same structure.
That same JSONB pattern reappears when a system needs to store the evidence behind an AI generated answer, which is covered directly in how a system finds the right record and cites it, the next part in this series. For the underlying documentation on the type itself, see PostgreSQL's own reference on JSON types.
Sources
- 1.AI Metric Ltd, AI and Software Architecture Translation Guide, edition v3.1PrimaryAccessed
- 2.PostgreSQL Global Development Group, PostgreSQL documentation: JSON typesAuthorityAccessed
- 3.OpenJS Foundation, Node.js documentation: the event loop, timers and process.nextTick()PrimaryAccessed
Published , last reviewed . This guide explains general principles and is not legal, contractual or safety advice. The position on any project depends on the contract signed and the facts of that project.