Local task cache

qk cache path
qk run web:build
qk run web:build
qk run web:build --skip-cache
qk cache prune --max-size 1GB

With a cacheable target and unchanged inputs, the second run restores outputs and replays logs. Run history explains why a key changed.

Targets with cache: true are cached locally. Inside a Git repository the cache lives by default in <git common dir>/qk/cache/v1, so every linked worktree of the repository shares one cache. What belongs to one worktree (file digests, records of the outputs it holds, restores in progress) lives in its own git directory, $(git rev-parse --git-dir)/qk, so nothing of qk’s appears in the working tree. Outside Git both live in .qk at the workspace root, the cache in .qk/cache/v1. qk cache path prints the cache’s location. --skip-cache (also --skip-nx-cache and --skipNxCache) bypasses result-cache reads and writes. Otherwise cacheable executions still record their keys and bounded stdout/stderr in run history, so repeated forced runs can reveal flaky outcomes.

What changes the key

A task’s key covers its ID, forwarded arguments, resolved target definition, declared inputs, the package manager, qk’s version and the platform. As in Nx, a dependency task counts only through the outputs the task reads: files its dependentTasksOutputFiles inputs select from the dependency’s declared outputs, or that its filesets select. A run that reproduces those outputs leaves the task cached however the dependency’s own inputs changed, and a task that reads none of them is keyed without them. A task that reads a dependency’s outputs without declaring it can therefore be restored stale; declare such reads as inputs. Files are keyed by workspace-relative path, content and mode. Branch names and checkout locations are not part of the key, so identical sources in two worktrees share entries.

A key has two kinds of input. What the task’s inputs select counts as they say: a file by its whole content and a json input by its fields, whatever the file is. Beside them, qk adds workspace configuration and package-manager state to every task’s key on its own, each by the rule below. Where an input selects a whole file qk would add, the file counts whole, as the input says. A json input on such a file adds its fields beside qk’s copy rather than narrowing it.

Root workspace files (.gitignore, .nxignore, pnpm-workspace.yaml, lockfiles) and the package.json of the task’s project are always included, its scripts and dependency declarations with it. The root tsconfig (tsconfig.base.json, or else tsconfig.json) is included as Nx hashes it: without compilerOptions.paths, so adding or moving a path mapping leaves every key as it was. With selectivelyHashTsConfig: true in the options of nx.json’s tasksRunnerOptions runner (the one --runner names, else the one NX_TASKS_RUNNER names, else default), the key also holds the mappings whose targets lie in the projects whose inputs the task reads: its own, and those its ^ and projects inputs or a continuous task serving it reach. A root tsconfig that cannot be read as JSON with comments is keyed whole. The root package.json is not, as in Nx: what the root importer installs counts for every task through the lockfile, as below, and the rest of it is the root project’s own manifest, so editing a root script misses only the root project’s tasks. Dotenv files are not, as in Nx: they hold per-machine values and credentials, so keying them would keep machines from sharing entries; env inputs key the variables a task declares.

nx.json counts through what it changes rather than by its content, unlike in Nx, which hashes it into every task. Merged with the file it extends and nx.local.json, it reaches a task through the task’s resolved configuration, which the key holds already: targetDefaults through its definition, and namedInputs through what its inputs select, the files, env and runtime inputs, externalDependencies and dependency outputs they name, a dependency’s through ^ inputs included. Beside that, the key holds the merged settings without targetDefaults and namedInputs, and without those that change no task’s work: $schema, extends, defaultBase, affected, parallel, tui, cli other than cli.packageManager, generators, release, sync, s3, nxCloudId, nxCloudAccessToken, nxCloudUrl, nxCloudEncryptionKey, neverConnectToCloud, defaultProject, cacheDirectory, maxCacheSize, qk:storeServer, qk:affectedProfiles, qk:reachability, and the workspace-wide implicitDependencies, which Nx no longer supports. Every other setting, such as plugins, pluginsConfig, workspaceLayout, tasksRunnerOptions, which can set selectivelyHashTsConfig, or one qk does not know, changes every task’s key. Comments and formatting change none, and neither does a named input edited to select the same files. A tool that reads nx.json itself declares {workspaceRoot}/nx.json as an input, which keys the file whole.

A project’s project.json and project.local.json are not added to its tasks’ keys either. An input that selects one, such as {projectRoot}/**/*, which default holds unless the workspace names it otherwise, keys it as the input says, as for any file and as in Nx: whole for a fileset, by its fields for a json input. Where no input selects them, they count through what they change. Their targets reach a task through its definition, and their named inputs through what its inputs select. Tags, the project’s name and implicitDependencies reach a task through which projects its inputs and dependencies select, and sourceRoot through the output paths {project.sourceRoot} names: the key holds each task’s resolved output paths. Beside those, a task’s key holds what is left of its own project’s configuration without name, targets, namedInputs, tags, implicitDependencies, sourceRoot, metadata, projectType, $schema, generators, release and qk:reachability, such as its root, and all of it when a forwarded argument holds a {project.*} token.

To have tasks keyed by their resolved configuration rather than by the files, leave the files out of the inputs that select them, as default does here:

{
  "namedInputs": {
    "default": [
      "{projectRoot}/**/*",
      "!{projectRoot}/project.json",
      "!{projectRoot}/project.local.json"
    ],
    "production": ["default", "!{projectRoot}/**/*.spec.ts"]
  }
}

A comment, or an edit to one target, then misses only the tasks whose keys it changes. The negations hold for every named input built on default, such as production, and for a dependency’s files read through ^production. As in Nx, a negation leaves the file out of all of the task’s filesets, so a target that also names {projectRoot}/project.json does not read it back; one whose inputs do not build on default names it to key it whole.

The package.json of each project the task’s project depends on, directly or not, counts by what it decides for its dependents, such as where their imports of it resolve, even when the task’s inputs leave the file out. Its version, scripts, description, keywords, author, contributors, maintainers, license, homepage, repository, bugs, funding, private and publishConfig are left out, since none of them changes what a dependent reads or runs. A version that moves into or out of the range a dependent declares for the package adds or removes that dependency instead, which changes the dependent’s key through what it depends on. With a pnpm lockfile qk can read, its dependencies, devDependencies, peerDependencies, peerDependenciesMeta, optionalDependencies and dependenciesMeta are left out too: what they install is keyed through the lockfile, as below. A manifest the task’s inputs select, as ^default selects every file of each dependency, counts whole. A dependency releasing a new version within its dependents’ ranges therefore leaves them cached, while a change to its exports reaches them.

A pnpm v9 pnpm-lock.yaml is keyed by what it installs rather than by its content: for the root importer and the importers of the task’s project and its transitive project dependencies, every package installation they reach, by snapshot key (version, peers and patch), integrity and dependencies. The root importer counts for every task because its packages resolve from every package, unless the task names its external dependencies: as in Nx, a task with an externalDependencies input, even an empty one, hashes the packages it names in place of every installed one, so only the root project’s own tasks then count the root importer. Name in it every root-installed tool the task runs; the sandbox’s audit and input analysis report a package a task reads but its key leaves out. A lockfile change therefore invalidates only the tasks whose projects install something that changed, plus every task when the lockfile version, pnpm’s settings or the lock of pnpm itself changes. A lockfile qk cannot read, or one the task’s inputs select, is keyed by its whole content.

With such a lockfile, pnpm-workspace.yaml is keyed without the keys that configure resolution (catalog, catalogs, overrides, patchedDependencies, peerDependencyRules and the like): their effect lands in the lockfile, so a catalog bump invalidates only the tasks whose installs change. A task whose inputs select the file keys it whole, as they say. That includes a root project’s own files and a task that reads the catalog itself, such as a catalog-consistency lint. A task that reads only packages or other keys outside resolution need not select it: the copy qk adds already covers them, while selecting it reruns the task on every catalog change. A task that reads only some resolution keys, such as catalog, can add them with a json input, which reads the file as YAML. Affected selection applies the same rules.

qk show hash <task> prints a task’s key and what it is computed from without running it, and --against compares it with another task’s key or with the task’s key in a recorded run; see Key inspection.

Inputs

Candidate input files are tracked and untracked-but-not-ignored files, minus planned tasks’ declared outputs. Nx keys a tracked declared output like any other file; qk leaves it out, so that a task regenerating a tracked file does not change keys, its own included. Source filesets also select other tasks’ declared outputs after dependencies complete, including newly generated files, while respecting source ignore rules and excluding the consumer’s own outputs. These files are keyed independently of selective dependentTasksOutputFiles inputs. The source list is taken once per run, so a file that a task creates without declaring it as an output is seen from the next run. File contents are re-read whenever their metadata changes; their digests persist between runs in the worktree’s state, keyed by path, size, modification time and, on Unix, change time, inode and mode, so a warm run reads only files whose metadata changed. Without inputs, a task uses default and ^default; default is {projectRoot}/**/* unless a named input overrides it. As in Nx, a project’s filesets select only the files it owns, leaving out those of projects nested in its root.

Every input declaration of Nx 24 is supported, always read as absent:

• filesets, as strings or {"fileset": "…"}, with ! exclusions; a path with no glob pattern names a file or a whole directory, and \ escapes a glob character. With "includeIgnored": true a fileset reads the disk, gitignored files included, and a file it reads is keyed by the content it points to, wherever that is;
• named inputs, also as {"input": "name"};
• ^name and {"input": "name", "dependencies": true} (or "projects": "dependencies"), which as in Nx cover every project the project depends on, directly or not, and ^{projectRoot}/…, ^{workspaceRoot}/… or {"fileset": "…", "dependencies": true}, a fileset of each of them;
• {"input": "name", "projects": […]}, the named input of the projects the project selectors select;
• env, runtime, dependentTasksOutputFiles with transitive, and externalDependencies, which adds every installation of the named packages in the pnpm lockfile, whichever importer installs them;
• {"workingDirectory": "relative" | "absolute"}, the directory qk was run from;
• {"json": "path", "fields": […], "excludeFields": […]}, only the selected dotted fields of a JSON file, or of each file a glob selects. Affected selection counts any change to the file, even when source ignore rules exclude its path.

As in Nx, a named input cannot use dependencies or projects. With a package manager other than pnpm, lockfiles are always keyed by content. Generated artifacts stay out of source filesets. A consumer’s filesets can include outputs from its dependency tasks, including transitive dependencies. Outputs in git-ignored directories are read only by dependentTasksOutputFiles and by filesets with includeIgnored. Outputs from downstream or unrelated tasks do not feed back into its cache key.

runtime commands share one evaluation per run when their effective runtime environments match, including when tasks request them concurrently. That environment includes dotenv, explicit settings and local binary lookup. As in Nx, runtime inputs are evaluated before task metadata such as NX_TASK_TARGET_TARGET is injected. Runner color defaults apply to task execution; an explicitly configured FORCE_COLOR remains visible to runtime commands. dependentTasksOutputFiles keys the selected artifacts directly; qk checks their listings and contents again after execution before saving a result. . and .. segments in paths are resolved within the workspace. A symlinked input is keyed by its target text and the content it resolves to, including the files below a linked directory, even when source ignore rules exclude the target. Nested links are followed; directory cycles and links outside the workspace run uncached. Task-level affected selection follows changes to symlink targets, including deletions, even when the target lies outside the source glob that selected the link.

See Inputs and outputs for glob expansion rules and the affected-selection differences from Nx for bare parenthesized extension groups.

Outputs

Outputs resolve as in Nx 24. {projectRoot} and {projectName} name the project, {options.name} reads the target’s options, with the task’s --name=value arguments applied, and the legacy {project.<path>} reads the project’s configuration, such as {project.sourceRoot}. A placeholder can fall back, left to right, to others and finally to a literal: {options.outputPath ?? options.outDir ?? {projectRoot}/dist}. A value is written as JavaScript writes it, so false is false, 0 is 0 and an array is its items joined with ,; only a missing option, null and "" have no value. Option values have {projectRoot}, {projectName} and a leading {workspaceRoot} resolved first, as Nx does when it builds the project graph. Braces that do not start with a token are glob text, so {projectRoot}/dist/**/*.{js,map} is one output, and so is {env.NAME}: environment variables are no placeholder in outputs.

As in Nx 24, every output starts with a token, after its ! if negated: {projectRoot}/dist or {workspaceRoot}/dist, not dist, which nx repair rewrites. An output whose {options.…} placeholder, or a {projectRoot} or {workspaceRoot} left as text, has no value is left out. {workspaceRoot} may only start an output, and an output from it must name a directory before any pattern: {workspaceRoot}/dist/**, not {workspaceRoot}/**. Building the task graph fails on an output that breaks either rule. An absolute output from an option is read relative to the workspace, and one outside the workspace is an error. ! negates an output: what it matches is neither cached nor removed when an entry is restored. A target without outputs takes options.outputPath, and a build or prepare target dist/<root>, <root>/dist, <root>/build and <root>/public. Those defaults are cached and restored, but because they are guesses they are not declared outputs: they stay source files that other tasks’ filesets can select, and dependentTasksOutputFiles selects none of them.

A negated group such as src/**/!(*.test).ts or dist/!(cache)/** expands as Nx expands it, into a glob with the group widened and one naming the group’s items that excludes, so both tools select the same files, including where Nx’s expansion is surprising: *.!(ts) keeps only files ending in ., as in Nx. Nx applies such exclusions to every pattern of a project at once; qk applies each to the pattern it comes from, which can only add inputs.

A task with any other input declaration, or an output without a fixed directory prefix, runs uncached and reports why.

Saving and restoring results

On a miss, the task runs with stdout and stderr streamed through a pipe while they are recorded, or, when qk’s output is a terminal, through a pseudo-terminal, which records them as one stream that replays on stdout. Runtime inputs always use pipes, so a key does not depend on where qk ran. The entry is saved only when the task succeeds and its inputs are unchanged afterwards. Declared dependency outputs are rechecked too. Cache keys use a new schema so entries saved before these artifact checks are not reused. On a hit, qk removes existing files matching the declared outputs, copies the cached outputs into place and replays the recorded stdout and stderr. As in Nx, outputs a worktree already holds for the key are left as they are: after each restore or save qk records, in the worktree’s state, every output path with its size, times, inode and mode, and a hit that finds exactly those still in place only replays the log, marked [existing outputs match the cache, left as is]. Restores are staged in the worktree’s state. Saving and restoring copy files, which clones them on copy-on-write filesystems such as APFS when the cache and the checkout share a volume. Files are never hardlinked, so editing a restored file never changes the cache. Every file is verified against its content hash; an unreadable or corrupt entry is a miss. A per-key lock makes concurrent runs of the same task, including runs in different worktrees, wait for each other and reuse the result.

Restored files take the time their restore started as their modification time. A task restores only after its dependencies finish, and its sources were written before the run, so restored outputs are newer than the sources they were built from and newer than the outputs of the tasks they depend on. Tools that decide what to rebuild by comparing modification times, such as tsc --build, then find restored outputs up to date. Outputs a worktree already holds are stamped again when a dependency’s outputs changed in the run, so they stay the newer. Symbolic links and directories keep their times.

Entries unused for a week are evicted after each run, as in Nx, and the cache stays under a size limit. See remote caching for size limits and cache directory overrides.