Init & Service Management

Process supervision, dependency resolution, and lifecycle management. The systemd of fishbowl, with npm’s dependency resolution model.

Units

The dependency graph operates on units — named things with a lifecycle and dependencies. Two unit types for v1:

type Unit = ServiceUnit | MountUnit

interface ServiceUnit {
  kind:      'service'
  name:      string                // unique identifier, e.g. 'httpd'
  bin:       string
  argv?:     string[]
  env?:      Record<string, string>

  // Lifecycle
  stop?:     StopAction
  reload?:   ReloadAction
  restart?:  RestartPolicy

  // Dependencies
  requires?: string[]              // must be running — fail if they can't start
  wants?:    string[]              // should be running — don't fail if they can't
  after?:    string[]              // ordering only — start after these
  before?:   string[]             // ordering only — start before these
}

interface MountUnit {
  kind:      'mount'
  name:      string                // derived from path, e.g. 'mount:/var/db'
  path:      string
  server:    Fileserver

  // Dependencies
  requires?: string[]              // e.g. mount:/var/db requires mount:/var
  after?:    string[]
  before?:   string[]
}

Units are identified by namespaced strings: service:httpd, mount:/var/db. Dependencies reference these identifiers. This prevents ambiguity when a service and a mount could share a name.

Decision: two unit kinds for v1, extensible to more. Services and mounts cover the real use cases. The kind discriminator means new unit types (timers, targets, sockets) can be added without changing the graph machinery.

Mount Units

Every mount declared in an Extension becomes a mount unit in the dependency graph. This happens automatically — the developer writes mounts: { '/var/db': sqliteFS() } in their Extension and init creates a mount:/var/db unit.

Mount units have a trivial lifecycle: “start” means mount the fileserver, “stop” means unmount. But their presence in the graph enables two things:

1. Services can depend on mounts. service:httpd requires mount:/var/www — init won’t start httpd until the mount exists.
2. Static analysis. The graph can detect at boot: “service:httpd requires mount:/var/www but no Extension provides that mount.”

Mount-to-mount dependencies are inferred from paths. mount:/var/db implicitly requires mount:/var if both exist. A mount at a child path needs its parent to exist first.

Decision: mount dependencies are inferred from path hierarchy. The developer doesn’t need to declare requires: ['mount:/var'] on mount:/var/db. Init infers it from path nesting. Explicit requires/after can override or extend the inferred dependencies.

Cross-Cutting Mount→Service Dependencies

Mount units are auto-generated from Extension.mounts, so the developer can’t add requires: ['service:dbsyncd'] to a mount directly. When a mount depends on a service (e.g., an S3 mount that needs a credentials service running first), the service declares the reverse relationship:

services: [{
  name: 'creds-agent',
  bin: 'creds-agent',
  before: ['mount:/mnt/s3'],   // "start me before this mount"
}]

The before field on ServiceUnit creates the same graph edge as after on the mount would. This keeps mount declarations simple (just a path and a fileserver) while enabling cross-cutting dependencies through the service that knows about the relationship.

Decision: cross-cutting mount→service deps use before on the service side. The service declares before: ['mount:/path'] rather than extending the mount declaration syntax. This avoids complicating the Extension.mounts shape (which remains Record<string, Fileserver>) and follows the principle that the unit with domain knowledge declares the relationship.

Open Question: Targets

systemd has “targets” — named groups of units that represent a system state (network-ready.target, database.target). A target has no process or mount of its own. It’s a synchronization point: “when all my dependencies are met, I’m reached.”

// Future — not v1
interface TargetUnit {
  kind:      'target'
  name:      string              // e.g. 'target:database-ready'
  requires?: string[]
  wants?:    string[]
}

Use case: package A provides service:postgres and declares target:database-ready with requires: ['service:postgres']. Package B provides service:api with requires: ['target:database-ready']. Later, package A is swapped for package C (service:mysql) which also satisfies target:database-ready. Package B doesn’t change.

Targets decouple consumers from providers. They’re valuable for large compositions but add conceptual weight. Deferring to post-v1 — the dependency graph machinery supports them without changes, only the unit type needs to be added.

Dependency Types

Four dependency types, matching systemd’s model:

Type	Meaning	Failure behavior
requires	Must be running before this unit starts	If dependency can’t start → this unit fails to start
wants	Should be running, but non-critical	If dependency can’t start → log warning, start anyway
after	Ordering only — start after these	No failure coupling — just sequencing
before	Ordering only — start before these	No failure coupling — just sequencing

requires implies after. If service A requires service B, A starts after B. No need to declare both.

wants implies after. Same ordering implication, weaker failure coupling.

before is the reverse of after — a convenience for when a unit wants to declare “I should start before X” rather than X declaring “I start after Y.” Both produce the same graph edge.

Dependency References

Dependencies reference unit identifiers:

services: [{
  name: 'httpd',
  bin: 'httpd',
  requires: ['service:dbsyncd', 'mount:/var/www'],
  wants: ['service:logger'],
  after: ['service:config-loader'],
}]

If the prefix is omitted, service: is assumed. So requires: ['dbsyncd'] means requires: ['service:dbsyncd'].

Decision: default namespace is service:. Services depending on other services is the common case. Requiring mount: or target: prefixes for non-service dependencies is explicit without being noisy for the typical case.

Resolution Algorithm

The same algorithm npm uses for install order: topological sort on a directed acyclic graph.

resolve(units):
  1. Build adjacency graph from all dependency declarations
     - requires, wants → hard edges (dependency + ordering)
     - after, before → soft edges (ordering only)

  2. Detect cycles
     - If cycle found → error at boot with clear message:
       "circular dependency: service:a → service:b → service:a"
     - Cycle detection runs once at boot, not at runtime

  3. Topological sort
     - Produce a start order: leaves first, dependents last
     - Units with no ordering constraints can start in parallel

  4. Validate
     - For each `requires`: does the referenced unit exist?
       If not → error: "service:httpd requires mount:/var/www but no unit provides it"
     - For each `wants`: does the referenced unit exist?
       If not → warning (not an error — wants is best-effort)

Decision: static validation at boot. The full dependency graph is known at boot time (all Extensions have been merged). Validate it once — detect cycles, check that all required units exist, warn about missing wanted units. Fail fast with clear errors rather than discovering problems at runtime when a service tries to start.

Parallel Start

Units with no ordering relationship between them can start concurrently:

Graph:
  mount:/var/db → service:dbsyncd → service:api
  mount:/var/www → service:httpd → service:api

Start groups (concurrent within each group):
  Group 1: mount:/var/db, mount:/var/www        (parallel)
  Group 2: service:dbsyncd, service:httpd        (parallel, after group 1)
  Group 3: service:api                           (after group 2)

Decision: parallel start within topological layers. The topo sort produces layers of independent units. Units in the same layer have no ordering constraints and can start concurrently. This minimizes boot time for complex service graphs without sacrificing correctness.

Service Lifecycle

Start

Starting a service means spawning its bin as a child process of init:

start(service):
  1. Check dependencies: all `requires` units are running?
     - If not → error or wait (depending on whether they're starting)
  2. Spawn bin with argv and env
  3. Service state: starting → running (once bin is executing)
  4. Record pid in service table

A service is “running” once its bin function is executing. There is no readiness protocol in v1 — “process is alive” equals “service is ready.” Health checks and readiness probes are post-v1.

Stop

type StopAction =
  | { signal: Signal }                           // send signal, wait for exit
  | { bin: string, argv?: string[] }             // run a stop command

// Default: { signal: 'SIGTERM' }

stop(service):
  1. Execute stop action:
     - signal: send signal to service pid
     - bin: spawn stop command, wait for completion
  2. Wait for service process to exit (with timeout)
  3. If not exited after grace period (gracePeriod, default 5s) → SIGKILL
     SIGKILL bypasses signal handlers — the kernel forcefully terminates the process.
     Unlike SIGTERM/SIGHUP, SIGKILL is not delivered via proc.on() and cannot be caught.
  4. Service state: running → stopped

Dependents are stopped first. If service:api requires service:httpd, stopping httpd triggers stopping api first. Reverse topological order.

Restart

restart(service):
  1. Stop (as above)
  2. Start (as above)

Restart cascades to all dependents. If httpd restarts and api requires httpd, api is stopped before httpd stops, then started after httpd starts. This is visible in svc restart output:

$ svc restart dbsyncd
stopping service:api (depends on service:httpd, depends on service:dbsyncd)...
stopping service:httpd (depends on service:dbsyncd)...
restarting service:dbsyncd...
starting service:httpd...
starting service:api...
done.

Decision: restart always cascades in v1. Restarting a service that others depend on must cascade — dependents can’t function without their dependency. A --no-cascade flag (stop/start only the named service, leaving dependents to cope) is a possible future addition but not v1. The safe default is to cascade.

Reload

type ReloadAction =
  | { signal: Signal }                           // send signal (convention: SIGHUP)
  | { bin: string, argv?: string[] }             // run a reload command

// Default: { signal: 'SIGHUP' }

Reload tells a service to re-read its configuration without stopping. Not all services support it — if no reload action is declared and the default SIGHUP isn’t handled by the bin, the signal is effectively ignored (or kills the process, triggering restart).

Decision: reload is best-effort, not guaranteed. The service bin must cooperate by handling the signal or implementing a reload command. Init can’t force a process to reload. This matches systemd’s behavior.

Supervision

Init monitors all service processes. When a service exits, init consults its restart policy:

interface RestartPolicy {
  policy:       'always' | 'on-failure' | 'never'
  maxRetries?:  number
  backoff?:     'linear' | 'exponential'
  delay?:       number        // base delay in ms, default: 1000
  gracePeriod?: number        // ms to wait before SIGKILL after stop, default: 5000
  stableAfter?: number        // ms of uptime before restart counter resets, default: 60000
}

// Default: { policy: 'never' }

Policy	Meaning
always	Restart regardless of exit code. For daemons that should never stop. Suspended during shutdown — see below.
on-failure	Restart only on non-zero exit code. Clean exit (code 0) means intentional stop.
never	Don’t restart. Service stays stopped until manually started.

Shutdown Behavior

During shutdown, all restart policies are suspended. Services are stopped in reverse dependency order regardless of their restart policy. A service with restart: { policy: 'always' } is not respawned when init is shutting down. This prevents respawn loops during shutdown — init sets a shuttingDown flag before stopping any services, and the supervision loop checks this flag before consulting restart policies.

Backoff

When a service keeps crashing, init backs off:

linear:       delay, delay*2, delay*3, ...
exponential:  delay, delay*2, delay*4, delay*8, ...

After maxRetries consecutive failures (default: unlimited for always, 5 for on-failure), init gives up and logs an error. The service state becomes failed. It can be manually restarted via svc start.

Decision: restart counter resets after stable running period. If a service runs successfully for stableAfter milliseconds (default: 60000), the retry counter resets to 0. A service that crashes on startup 5 times fails. A service that runs for an hour, crashes, runs for another hour, crashes — that’s healthy operation with occasional restarts, not a failure spiral. Both gracePeriod and stableAfter are configurable per-service on RestartPolicy.

Service Table

Init maintains a service table — the runtime state of all declared services:

interface ServiceEntry {
  unit:       ServiceUnit
  state:      'stopped' | 'starting' | 'running' | 'stopping' | 'failed'
  pid:        Pid | null
  startedAt:  number | null     // ms since epoch
  restarts:   number            // consecutive restart count
  lastExit:   number | null     // last exit code
}

This table is the source of truth for svc status and for init’s supervision loop.

The service table is exposed via ProcFS at /proc/services/:

/proc/services/
  httpd/
    state       → "running"
    pid         → "4"
    uptime      → "120"
    restarts    → "0"
  dbsyncd/
    state       → "running"
    pid         → "3"
    uptime      → "125"
    restarts    → "2"

Decision: service state in ProcFS, not a separate fileserver. ProcFS already exposes kernel state as files. Service state is process-adjacent information — it belongs in /proc/. The svc command reads from here. Other bins can too.

The svc Command

The shell interface for service management:

$ svc status
NAME      STATE     PID   UPTIME   RESTARTS
dbsyncd   running   3     2m 5s    0
httpd     running   4     2m 0s    0
api       running   5     1m 58s   0

$ svc status httpd
Name:       httpd
State:      running
PID:        4
Uptime:     2m 0s
Restarts:   0
Requires:   service:dbsyncd, mount:/var/www
Wanted by:  service:api

$ svc stop httpd
stopping service:api (depends on service:httpd)...
stopping service:httpd...
done.

$ svc start httpd
starting service:httpd...
starting service:api (depends on service:httpd)...
done.

$ svc restart dbsyncd
stopping service:api (depends on service:httpd, depends on service:dbsyncd)...
stopping service:httpd (depends on service:dbsyncd)...
restarting service:dbsyncd...
starting service:httpd...
starting service:api...
done.

$ svc list
NAME      STATE     REQUIRES                        WANTS
dbsyncd   running   mount:/var/db                   -
httpd     running   service:dbsyncd, mount:/var/www  service:logger
api       running   service:httpd                    -

$ svc graph
mount:/var/db → service:dbsyncd → service:httpd → service:api
mount:/var/www ──────────────────┘

svc reads from /proc/services/ for state and communicates with init via signals or a control mechanism for start/stop/restart actions.

Init Communication

svc needs to tell init “start this service” or “stop this service.” Two options:

1. Signal-based. svc writes a command to a control file (/proc/init/ctl) and sends SIGUSR1 to PID 1. Init reads the command and acts.
2. Direct. svc spawns nothing — it’s a builtin or has direct access to init’s service table via a shared reference.

Decision: control file at /proc/init/ctl. Write the command (start httpd, stop httpd), signal init. This is the Plan 9 way — control via file writes, not custom APIs. It also means any process can manage services, not just the svc bin.

Note: Signal type needs SIGUSR1 and SIGUSR2. The control file mechanism requires svc to send SIGUSR1 to PID 1. The current Signal type in types.ts ('SIGTERM' | 'SIGKILL' | 'SIGPIPE' | 'SIGHUP') must be extended with 'SIGUSR1' | 'SIGUSR2' and corresponding entries in SIGNAL_NUMBER. SIGUSR1 is the init notification signal; SIGUSR2 is reserved for future use (e.g., status dump).

Init’s Role

Init is a generated bin. The builder constructs it from the accumulated service and mount declarations across all Extensions. The developer never writes init — it emerges from composition.

Boot Sequence (with services)

boot():
  1. Merge all Extensions → collected units (services + mounts)
  2. Build dependency graph
  3. Static validation:
     - Cycle detection
     - Missing required units
     - Warn on missing wanted units
  4. Generate init bin with embedded unit graph
  5. Spawn init as PID 1

init (PID 1):
  1. Start units in topological order:
     - Mount units: mount fileservers
     - Service units: spawn bins
     - Parallel within each topological layer
  2. All units started → start shell
  3. Supervision loop:
     - Wait for any child to exit
     - If service → consult restart policy, respawn or mark failed
     - If shell → begin shutdown
  4. Shutdown (reverse topological order):
     - Stop services (respecting dependency order)
     - Unmount filesystems
     - Exit with shell's exit code

Boot Sequence (no services)

boot():
  1. Merge all Extensions → no service units
  2. Mount filesystems (no graph needed — just mount order from path hierarchy)
  3. Spawn shell as PID 1
  4. Return UnixInstance

No init overhead. The common case stays simple.

Runtime Service Addition

When pkg install adds a package with services while the system is running:

pkg install @fishnet/http-server:
  1. Apply Extension (mounts, bins, env, files)
  2. Create new unit(s) from service declarations
  3. Validate: check new dependencies against existing graph
  4. Write start command to /proc/init/ctl
  5. Init receives signal, reads command
  6. Init adds unit(s) to graph, starts in dependency order

Decision: init supports dynamic unit addition. The graph is built at boot but can grow at runtime. pkg install can add services without rebooting. pkg remove can remove them. The graph is re-validated on each addition. Cycles introduced by runtime additions are rejected.

Static Analysis

Because the full graph is known at boot (or at pkg install time), init can catch problems early:

Check	When	Severity
Cycle detection	Boot, pkg install	Error — refuse to boot/install
Missing requires unit	Boot, pkg install	Error — refuse to boot/install
Missing wants unit	Boot, pkg install	Warning — proceed anyway
Duplicate unit name	Extension merge	Error — two packages providing service:httpd
Port/resource conflict	Future	Warning — two services binding same resource

# The builder catches this at boot time:
$ # @fishnet/api requires service:cache, but nothing provides it

Error: dependency resolution failed
  service:api requires service:cache
  no unit provides service:cache

Provided units:
  mount:/          (stdSystem)
  mount:/var/db    (@fishnet/sqlite)
  service:dbsyncd  (@fishnet/sqlite)
  service:api      (@fishnet/api)

Missing:
  service:cache    (required by service:api)

Decision: fail fast with actionable errors. The error message lists what’s provided, what’s missing, and which package declared the unmet dependency. The developer can see exactly what Extension they need to add. This is the same experience as a missing npm dependency — clear, fixable, not a runtime surprise.

Summary

Init is the convergence point of three ideas:

1. systemd’s unit model — services and mounts as typed nodes in a dependency graph, with requires/wants/after/before relationships
2. npm’s resolution algorithm — topological sort for start order, cycle detection, static validation of the full graph
3. Plan 9’s control files — manage services by writing to /proc/init/ctl, not through custom APIs

The developer declares services in Extensions. The builder collects them, builds the graph, validates it, and generates init. Init supervises the running system. The svc command provides the shell interface. Everything composes through the same Extension type that drives the rest of the system.