The watch process that used up every file descriptor
•5 min read
Starting the dev watcher over and over across a long session, and never killing the old ones, filled the machine's file descriptor table. The thing that fell over first was Docker, which had not done anything wrong.
Starting a file watcher repeatedly across one long working session, without killing the previous ones, will exhaust the machine's file descriptors. In this project the command was the package dev script, which runs tsx in watch mode, started again each time a service needed restarting for a check. By the end of the session 30,687 of the 30,720 available descriptors were in use, and the process that failed was the Docker daemon — which had done nothing wrong and simply asked for a descriptor after they had all gone.
30,687 of 30,720Open file descriptors in use when the Docker daemon stopped working during a long development session
Why a watcher is the process that does this
A file watcher works by asking the operating system to tell it when specific paths change, and on several platforms that means holding an open handle for each watched path. A TypeScript service in a monorepo watches its own source, the packages it imports and their type declarations, which is thousands of paths before anyone has written a line. One watcher is unremarkable. Ten abandoned watchers are ten copies of that set, all still held, all invisible in a terminal that has scrolled away.
The leak is not a bug in the watcher. Each process was doing exactly what it was told for as long as it was alive, and none of them were told to stop. What went wrong is a habit: in a session driven by a scripted or assisted workflow, starting a dev server is one line and stopping it is a line nobody remembers to write.
Why the failure appears somewhere unrelated
Descriptor exhaustion is a resource problem, so the process that reports it is whichever one asks next, not the one that consumed them. That is what makes it confusing to diagnose: a container runtime, an editor's language server or a package install starts failing with errors about opening files, and nothing in that program's logs points at the dozen watchers sitting behind it. The errno for the condition is EMFILE, and its message is usually rendered as too many open files.
Two limits matter and they behave differently. There is a per-process limit, which is what a single greedy program hits, and a system-wide one, which is what a pile of small consumers hits together. A watcher restarted twenty times is the second case, so raising a per-process limit fixes nothing while the abandoned processes are still running.
Resource exhaustion on a shared machine has a family resemblance to a separate failure on this project, where
Every transactional path failed against a perfectly healthy local MongoDB. The database was fine. It just was not a replica set, and transactions do not work without one.
Market4 turns one release note into a changelog page, a blog post, a mail-out and a week of social posts — and then tells you which of them brought anyone back.
took every other application on that box down with it. Both are the same shape: a development-time convenience run on a machine that had other jobs, with the bill arriving somewhere the work was not happening.
Which command to run for which job
Watch mode is for a person editing files, not for a script that starts a service, makes a request and moves on. The distinction is easy to encode once it has been named, and it removes most of the exposure without changing anybody's editing workflow.
What each way of starting a service holds open.
What you run
Watches the file tree
What it costs to leave running
The dev script, running tsx watch
Yes
A set of open descriptors, once per watcher started
The plain start script
No
One process and its own connections
A test run
No
Nothing once it has exited
A container
No
Whatever the runtime holds, until stopped
The habits that stop it recurring
Descriptor exhaustion from watchers is preventable by habit rather than by tooling, and the habits are small enough to keep. The list below is what this project adopted after the session that produced the number above.
Use the non-watch start command for smoke tests and scripted checks, and keep watch mode for interactive editing.
Kill what you started in the same step that started it, so the stop is written at the same time as the start rather than remembered later.
When something unrelated starts failing on file access, count descriptors before blaming the tool that reported the error.
In a session that runs for hours, sweep for stray processes periodically instead of at the end, because the end is where the failure already happened.
Counting is the step people skip, and it is the cheap one. The soft per-process limit is visible with ulimit -n, a single process's open files can be listed with lsof against its process id, and comparing those two numbers answers within a minute whether the problem is one greedy program or a crowd of forgotten ones.
What causes too many open files during development?
Usually a file watcher, or several of them. A watcher holds an open handle for each path it is watching, and a TypeScript project in a monorepo watches thousands of paths. Restarting the dev command without stopping the previous run leaves the old watcher alive with all of its handles, so the count climbs with each restart until something — often an unrelated program — fails with EMFILE.
Why did Docker fail when the problem was a watch process?
Because file descriptors are a shared resource and the failure lands on whichever process asks for one after they have run out. The watchers consumed them quietly over hours; the container runtime needed one, could not get it, and reported the error. Nothing in the runtime's logs points back at the real consumer, which is why the first diagnostic step is counting descriptors rather than reading the failing program's output.
Should you raise the file descriptor limit?
Not as the first move. Raising a per-process limit does nothing when the consumption is spread across many processes, and raising the system limit buys time while leaving the leak in place. Find and kill the abandoned processes first, then decide whether the limit is genuinely too low for the work. A limit raised to accommodate a leak hides the next occurrence rather than preventing it.
When is watch mode the wrong tool?
Whenever the process is started by a script rather than by a person. A smoke test starts a service, makes a request and finishes, and the watcher's ability to restart on a file change is worth nothing in that sequence while its open handles cost something real. Use the plain start command there. Watch mode earns its cost only when somebody is editing the files it is watching.
Publishing a post sends two independent announcements. One IndexNow request reaches Bing, Yandex, Seznam, Naver and Yep. Google takes no part in IndexNow, so it is told by re-submitting the sitemap.
We split one long numbered prompt into two. Every step after the cut changed number, and the sentences that navigate by those numbers went on pointing at whatever now wore them.
Build diaryPromptsTesting
The watch process that used up every file descriptor