A blog post published May 29 by the creator of Obelisk, an open-source durable workflow engine, landed on Hacker News the following morning with brisk developer engagement — and the argument it makes is direct: for a large class of workflow systems, a SQLite file and a Litestream backup to S3-compatible object storage is all the infrastructure teams actually need, and the managed queues and brokers most engineers reach for are over-engineered solutions to a problem that the wrong tool is solving.
The post, titled "SQLite Is All You Need for Durable Workflows," opens by acknowledging a parallel argument from DBOS — that if a team already trusts its database, it does not need a separate orchestration tier. The Obelisk author agrees with the direction but pushes the logic one level further: Postgres is not always necessary either. For many bursty, experimental, or tenant-isolated workloads, the entire execution log fits comfortably in a file on disk.
Separating Durable State from Durable Infrastructure
The blog post's core insight is a distinction that gets lost in most discussions of workflow reliability: durable execution does not require durable infrastructure. The part that must survive a crash is the workflow state — the log of what steps have completed, what results came back, and where execution left off. The compute that processes those steps can be cheap and disposable.
This matters because most workflow systems conflate the two. Teams reach for Amazon SQS, RabbitMQ, or Kafka not because their state model demands it, but because the conventional stack does. Obelisk's position is that a SQLite database gives teams ACID guarantees and a persistent execution log for the price of a local file — no network hop, no extra control plane, no new operational surface area.
Obelisk itself is a pre-release, open-source, WebAssembly-based workflow engine written in Rust. It ships as a single binary that executes deterministic workflows, activities, and webhook endpoints, persisting each step in an execution log using either SQLite or PostgreSQL. There are no brokers, no sidecars, and no YAML pipelines. Every call, sleep, and result is written to the log. If the engine crashes mid-workflow, it resumes from the last completed step on restart.
The framework enforces a strict separation between workflows — which must be purely deterministic and are compiled to a WebAssembly target with no access to networking or the filesystem — and activities, which handle all side effects such as external API calls and database writes. That separation is what makes replay safe: deterministic workflows can be reconstructed from their log without re-running side effects.
How Litestream Closes the Portability Gap
The obvious objection to any SQLite-as-production-backend argument is the single-file failure mode: if the host goes down and the file is gone, so is the state. Obelisk addresses this directly through Litestream, an open-source tool created by Ben Johnson and maintained under Fly.io that asynchronously streams SQLite changes to S3-compatible object storage. Teams running Obelisk can back up a SQLite execution log to standard object storage and restore it on a new host after a failure.
The blog post is candid about the limitation: Litestream replication is asynchronous, meaning a restore can miss writes committed locally but not yet copied to object storage before the host failed. That is an acceptable trade-off for experimentation-oriented or AI agent workloads, the author argues, but it is not equivalent to a highly available shared database.
That trade-off came up immediately in the Hacker News discussion that followed. Developer gwking reported abandoning Litestream's 0.5.x series entirely after encountering runaway disk usage that would have caused production downtime, switching instead to nightly backups using SQLite's built-in rsync tooling. Developer golem14 described a separate bug in Litestream releases 5.9 and later that generates roughly 10 gigabytes of daily replication traffic for a database with fewer than 10 kilobytes of actual data. Litestream's own change log and issue tracker are the right places to verify which versions are affected before adopting it in any production environment.
AI Agent Workflow Orchestration: Where This Architecture Fits
The blog post makes a pointed case for AI agent workflows as the natural sweet spot for the SQLite-plus-Litestream model. AI agents tend to be bursty and experimental, and they benefit from the kind of fault isolation that a fleet of small, independent servers — each with its own SQLite file and S3 backup — provides more naturally than a single large shared database. The same execution log file that keeps state safe can double as a debuggable artifact: the identical file works for local replay, post-mortem investigation, and understanding exactly what a running agent did and why.
The Hacker News thread surfaced independent corroboration of this pattern. One developer described converging on the same approach independently — building a directed acyclic graph of steps, persisting each step's output to SQLite, then asking a language model to re-run only the steps that changed. Another noted that language models interact more efficiently with SQLite than with large markdown files or JSON blobs, because a targeted SQL query returns only the rows an agent needs rather than forcing the model to scan an entire document.
Developer shukantpal, testing a separate agent harness, reported reaching 7,500 concurrent sessions on a single virtual CPU using SQLite before Postgres ran out of connections entirely — a data point that lines up with Obelisk's argument that SQLite's in-process architecture eliminates the overhead of inter-process communication.
Open-Source Workflow Engine Alternatives: Where Temporal Fits
The most substantive pushback in the developer thread came from a commenter recommending Temporal as a more mature alternative. Temporal is a production-grade durable execution platform that grew out of the Cadence project at Uber. At its Replay 2026 conference, Temporal announced serverless workers, standalone activities, and new integrations with OpenAI's Agents SDK and Google's Agent Development Kit, positioning it squarely as the enterprise-grade option for AI agent orchestration at scale.
A second commenter, posting as levkk, offered a blunter criticism: SQLite is an embedded database designed for single-process use, not for managing concurrent write workloads, and conflating its simplicity with production-readiness in shared-access scenarios reflects inexperience with the operational demands of multi-tenant systems.
The Obelisk author does not dispute the scaling ceiling. The blog post explicitly recommends Postgres when higher availability, broader shared scalability, or asynchronous replication to object storage is not the right durability model. The practical guidance is narrower: most workflow systems do not need Postgres-grade infrastructure on day one, and teams that start with more infrastructure than their state requires spend operational budget they did not need to spend.
What "Pre-Release" Means for Teams Evaluating Obelisk
Obelisk carries an explicit pre-release label on its GitHub repository, with a warning that the command-line interface, gRPC API, WIT interface definitions, and database schema are all subject to change. Teams evaluating it today should treat it as an experimentation and staging platform, not a production dependency. The project is licensed under the GNU Affero General Public License version 3, which requires that source code be made available to users who interact with the software over a network — a licensing consideration for teams building commercial services on top of it.
The project has been in active development since at least April 2025, when it first appeared on Hacker News, and a benchmark post the author published in October 2025 — measuring Obelisk against WindMill, a competing open-source workflow engine — demonstrated a technically rigorous approach to the engineering tradeoffs in SQLite's single-writer model. The full blog post and project documentation are at obeli.sk, with source code on GitHub.
Frequently Asked Questions
What is a durable workflow engine?
A durable workflow engine executes multi-step processes — sequences of function calls, retries, and waits — in a way that survives crashes and restarts. Each step's result is persisted to a log, so if the engine stops unexpectedly it resumes from the last recorded step rather than starting over. Common examples include Temporal, DBOS, and Obelisk.
Is SQLite suitable for production workflow systems?
SQLite works well for workflow state when write concurrency is low and each workflow instance is isolated — conditions that match AI agent workloads and single-tenant pipelines. It is not suitable for high-concurrency shared databases with many simultaneous writers. Obelisk supports both SQLite and PostgreSQL, and its author recommends switching to Postgres when availability or scalability requirements exceed what SQLite and Litestream can provide.
How does Litestream back up SQLite databases?
Litestream is an open-source tool that streams incremental changes from a local SQLite database to S3-compatible object storage. It replicates the SQLite write-ahead log asynchronously, meaning restores are possible after host failures but may miss the most recent writes. Users should verify the current release's stability before production adoption, as recent versions have reported replication bugs in community testing.
What is the difference between Obelisk and Temporal?
Temporal is a mature, production-grade durable execution platform with enterprise features including multi-region replication and SDKs for multiple languages, used by companies including OpenAI, Replit, and Cloudflare. Obelisk is a pre-release, open-source WebAssembly workflow engine built for simplicity and AI agent use cases where each execution instance is small and isolated. Temporal is the right choice for high-scale, distributed, multi-tenant production systems; Obelisk suits teams that want minimal infrastructure for experimental or tenant-isolated workloads.
ⓒ 2026 TECHTIMES.com All rights reserved. Do not reproduce without permission.
