feat: support preferred external ports besides 443 (#3117)

* docs: update preferred external port design in TODO

* docs: add user-controlled public/private and port forward mapping to design

* docs: overhaul interfaces page design with view/manage split and per-address controls

* docs: move address enable/disable to overflow menu, add SSL indicator, defer UI placement decisions

* chore: remove tor from startos core

Tor is being moved from a built-in OS feature to a service. This removes
the Arti-based Tor client, onion address management, hidden service
creation, and all related code from the core backend, frontend, and SDK.

- Delete core/src/net/tor/ module (~2060 lines)
- Remove OnionAddress, TorSecretKey, TorController from all consumers
- Remove HostnameInfo::Onion and HostAddress::Onion variants
- Remove onion CRUD RPC endpoints and tor subcommand
- Remove tor key handling from account and backup/restore
- Remove ~12 tor-related Cargo dependencies (arti-client, torut, etc.)
- Remove tor UI components, API methods, mock data, and routes
- Remove OnionHostname and tor patterns/regexes from SDK
- Add v0_4_0_alpha_20 database migration to strip onion data
- Bump version to 0.4.0-alpha.20

* chore: flatten HostnameInfo from enum to struct

HostnameInfo only had one variant (Ip) after removing Tor. Flatten it
into a plain struct with fields gateway, public, hostname. Remove all
kind === 'ip' type guards and narrowing across SDK, frontend, and
container runtime. Update DB migration to strip the kind field.

* chore: format RPCSpec.md markdown table

* docs: update TODO.md with DerivedAddressInfo design, remove completed tor task

* feat: implement preferred port allocation and per-address enable/disable

- Add AvailablePorts::try_alloc() with SSL tracking (BTreeMap<u16, bool>)
- Add DerivedAddressInfo on BindInfo with private_disabled/public_enabled/possible sets
- Add Bindings wrapper with Map impl for patchdb indexed access
- Flatten HostAddress from single-variant enum to struct
- Replace set-gateway-enabled RPC with set-address-enabled
- Remove hostname_info from Host; computed addresses now in BindInfo.addresses.possible
- Compute possible addresses inline in NetServiceData::update()
- Update DB migration, SDK types, frontend, and container-runtime

* feat: replace InterfaceFilter with ForwardRequirements, add WildcardListener, complete alpha.20 bump

- Replace DynInterfaceFilter with ForwardRequirements for per-IP forward
  precision with source-subnet iptables filtering for private forwards
- Add WildcardListener (binds [::]:port) to replace the per-gateway
  NetworkInterfaceListener/SelfContainedNetworkInterfaceListener/
  UpgradableListener infrastructure
- Update forward-port script with src_subnet and excluded_src env vars
- Remove unused filter types and listener infrastructure from gateway.rs
- Add availablePorts migration (IdPool -> BTreeMap<u16, bool>) to alpha.20
- Complete version bump to 0.4.0-alpha.20 in SDK and web

* outbound gateway support (#3120)

* Multiple (#3111)

* fix alerts i18n, fix status display, better, remove usb media, hide shutdown for install complete

* trigger chnage detection for localize pipe and round out implementing localize pipe for consistency even though not needed

* Fix PackageInfoShort to handle LocaleString on releaseNotes (#3112)

* Fix PackageInfoShort to handle LocaleString on releaseNotes

* fix: filter by target_version in get_matching_models and pass otherVersions from install

* chore: add exver documentation for ai agents

* frontend plus some be types

---------

Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>

* feat: replace SourceFilter with IpNet, add policy routing, remove MASQUERADE

* build ts types and fix i18n

* fix license display in marketplace

* wip refactor

* chore: update ts bindings for preferred port design

* feat: refactor NetService to watch DB and reconcile network state

- NetService sync task now uses PatchDB DbWatch instead of being called
  directly after DB mutations
- Read gateways from DB instead of network interface context when
  updating host addresses
- gateway sync updates all host addresses in the DB
- Add Watch<u64> channel for callers to wait on sync completion
- Fix ts-rs codegen bug with #[ts(skip)] on flattened Plugin field
- Update SDK getServiceInterface.ts for new HostnameInfo shape
- Remove unnecessary HTTPS redirect in static_server.rs
- Fix tunnel/api.rs to filter for WAN IPv4 address

* re-arrange (#3123)

* new service interfacee page

* feat: add mdns hostname metadata variant and fix vhost routing

- Add HostnameMetadata::Mdns variant to distinguish mDNS from private domains
- Mark mDNS addresses as private (public: false) since mDNS is local-only
- Fall back to null SNI entry when hostname not found in vhost mapping
- Simplify public detection in ProxyTarget filter
- Pass hostname to update_addresses for mDNS domain name generation

* looking good

* feat: add port_forwards field to Host for tracking gateway forwarding rules

* update bindings for API types, add ARCHITECTURE (#3124)

* update binding for API types, add ARCHITECTURE

* translations

* fix: add CONNMARK restore-mark to mangle OUTPUT chain

The CONNMARK --restore-mark rule was only in PREROUTING, which handles
forwarded packets. Locally-bound listeners (e.g. vhost) generate replies
through the OUTPUT chain, where the fwmark was never restored. This
caused response packets to route via the default table instead of back
through the originating interface.

* chore: reserialize db on equal version, update bindings and docs

- Run de/ser roundtrip in pre_init even when db version matches, ensuring
  all #[serde(default)] fields are populated before any typed access
- Add patchdb.md documentation for TypedDbWatch patterns
- Update TS bindings for CheckPortParams, CheckPortRes, ifconfigUrl
- Update CLAUDE.md docs with patchdb and component-level references

* fix: include public gateways for IP-based addresses in vhost targets

The server hostname vhost construction only collected private IPs,
always setting public to empty. Public IP addresses (Ipv4/Ipv6 metadata
with public=true) were never added to the vhost target's public gateway
set, causing the vhost filter to reject public traffic for IP-based
addresses.

* fix: add TLS handshake timeout and fix accept loop deadlock

Two issues in TlsListener::poll_accept:

1. No timeout on TLS handshakes: LazyConfigAcceptor waits indefinitely
   for ClientHello. Attackers that complete TCP handshake but never send
   TLS data create zombie futures in `in_progress` that never complete.
   Fix: wrap the entire handshake in tokio::time::timeout(15s).

2. Missing waker on new-connection pending path: when a TCP connection
   is accepted and the TLS handshake is pending, poll_accept returned
   Pending without calling wake_by_ref(). Since the TcpListener returned
   Ready (not Pending), no waker was registered for it. With edge-
   triggered epoll and no other wakeup source, the task sleeps forever
   and remaining connections in the kernel accept queue are never
   drained. Fix: add cx.waker().wake_by_ref() so the task immediately
   re-polls and continues draining the accept queue.

* fix: switch BackgroundJobRunner from Vec to FuturesUnordered

BackgroundJobRunner stored active jobs in a Vec<BoxFuture> and polled
ALL of them on every wakeup — O(n) per poll. Since this runs in the
same tokio::select! as the WebServer accept loop, polling overhead from
active connections directly delayed acceptance of new connections.

FuturesUnordered only polls woken futures — O(woken) instead of O(n).

* chore: update bindings and use typed params for outbound gateway API

* feat: per-service and default outbound gateway routing

Add set-outbound-gateway RPC for packages and set-default-outbound RPC
for the server, with policy routing enforcement via ip rules. Fix
connmark restore to skip packets with existing fwmarks, add bridge
subnet routes to per-interface tables, and fix squashfs path in
update-image-local.sh.

* refactor: manifest wraps PackageMetadata, move dependency_metadata to PackageVersionInfo

Manifest now embeds PackageMetadata via #[serde(flatten)] instead of
duplicating ~14 fields. icon and dependency_metadata moved from
PackageMetadata to PackageVersionInfo since they are registry-enrichment
data loaded from the S9PK archive. merge_with now returns errors on
metadata/icon/dependency_metadata mismatches instead of silently ignoring
them.

* fix: replace .status() with .invoke() for iptables/ip commands

Using .status() leaks stderr directly to system logs, causing noisy
iptables error messages. Switch all networking CLI invocations to use
.invoke() which captures stderr properly. For check-then-act patterns
(iptables -C), use .invoke().await.is_err() instead of
.status().await.map_or(false, |s| s.success()).

* feat: add check-dns gateway endpoint and fix per-interface routing tables

Add a `check-dns` RPC endpoint that verifies whether a gateway's DNS
is properly configured for private domain resolution. Uses a three-tier
check: direct match (DNS == server IP), TXT challenge probe (DNS on
LAN), or failure (DNS off-subnet).

Fix per-interface routing tables to clone all non-default routes from
the main table instead of only the interface's own subnets. This
preserves LAN reachability when the priority-75 catch-all overrides
default routing. Filter out status-only flags (linkdown, dead) that
are invalid for `ip route add`.

* refactor: rename manifest metadata fields and improve error display

Rename wrapperRepo→packageRepo, marketingSite→marketingUrl,
docsUrl→docsUrls (array), remove supportSite. Add display_src/display_dbg
helpers to Error. Fix DepInfo description type to LocaleString. Update
web UI, SDK bindings, tests, and fixtures to match. Clean up cli_attach
error handling and remove dead commented code.

* chore: bump sdk version to 0.4.0-beta.49

* chore: add createTask decoupling TODO

* chore: add TODO to clear service error state on install/update

* round out dns check, dns server check, port forward check, and gateway port forwards

* chore: add TODOs for URL plugins, NAT hairpinning, and start-tunnel OTA updates

* version instead of os query param

* interface row clickable again, bu now with a chevron!

* feat: implement URL plugins with table/row actions and prefill support

- Add URL plugin effects (register, export_url, clear_urls) in core
- Add PluginHostnameInfo, HostnameMetadata::Plugin, and plugin registration types
- Implement plugin URL table in web UI with tableAction button and rowAction overflow menus
- Thread urlPluginMetadata (packageId, hostId, interfaceId, internalPort) as prefill to actions
- Add prefill support to PackageActionData so metadata passes through form dialogs
- Add i18n translations for plugin error messages
- Clean up plugin URLs on package uninstall

* feat: split row_actions into remove_action and overflow_actions for URL plugins

* touch up URL plugins table

* show table even when no addresses

* feat: NAT hairpinning, DNS static servers, clear service error on install

- Add POSTROUTING MASQUERADE rules for container and host hairpin NAT
- Allow bridge subnet containers to reach private forwards via LAN IPs
- Pass bridge_subnet env var from forward.rs to forward-port script
- Use DB-configured static DNS servers in resolver with DB watcher
- Fall back to resolv.conf servers when no static servers configured
- Clear service error state when install/update completes successfully
- Remove completed TODO items

* feat: builder-style InputSpec API, prefill plumbing, and port forward fix

- Add addKey() and add() builder methods to InputSpec with InputSpecTools
- Move OuterType to last generic param on Value, List, and all dynamic methods
- Plumb prefill through getActionInput end-to-end (core → container-runtime → SDK)
- Filter port_forwards to enabled addresses only
- Bump SDK to 0.4.0-beta.50

* fix: propagate host locale into LXC containers and write locale.conf

* chore: remove completed URL plugins TODO

* feat: OTA updates for start-tunnel via apt repository (untested)

- Add apt repo publish script (build/apt/publish-deb.sh) for S3-hosted repo
- Add apt source config and GPG key placeholder (apt/)
- Add tunnel.update.check and tunnel.update.apply RPC endpoints
- Wire up update API in tunnel frontend (api service + mock)
- Uses systemd-run --scope to survive service restart during update

* fix: publish script dpkg-name, s3cfg fallback, and --reinstall for apply

* chore: replace OTA updates TODO with UI TODO for MattDHill

* feat: add getOutboundGateway effect and simplify VersionGraph init/uninit

Add getOutboundGateway effect across core, container-runtime, and SDK
to let services query their effective outbound gateway with callback
support. Remove preInstall/uninstall hooks from VersionGraph as they
are no longer needed.

* frontend start-tunnel updates

* chore: remove completed TODO

* feat: tor hidden service key migration

* chore: migrate from ts-matches to zod across all TypeScript packages

* feat(core): allow setting server hostname

* send prefill for tasks and hide operations to hidden fields

* fix(core): preserve plugin URLs across binding updates

BindInfo::update was replacing addresses with a new DerivedAddressInfo
that cleared the available set, wiping plugin-exported URLs whenever
bind() was called. Also simplify update_addresses plugin preservation
to use retain in place rather than collecting into a separate set.

* minor cleanup from patch-db audit

* clean up prefill flow

* frontend support for setting and changing hostname

* feat(core): refactor hostname to ServerHostnameInfo with name/hostname pair

- Rename Hostname to ServerHostnameInfo, add name + hostname fields
- Add set_hostname_rpc for changing hostname at runtime
- Migrate alpha_20: generate serverInfo.name from hostname, delete ui.name
- Extract gateway.rs helpers to fix rustfmt nesting depth issue
- Add i18n key for hostname validation error
- Update SDK bindings

* add comments to everything potentially consumer facing (#3127)

* add comments to everything potentially consumer facing

* rework smtp

---------

Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>

* implement server name

* setup changes

* clean up copy around addresses table

* feat: add zod-deep-partial, partialValidator on InputSpec, and z.deepPartial re-export

* fix: header color in zoom (#3128)

* fix: merge version ranges when adding existing package signer (#3125)

* fix: merge version ranges when adding existing package signer

   Previously, add_package_signer unconditionally inserted the new
   version range, overwriting any existing authorization for that signer.
   Now it OR-merges the new range with the existing one, so running
   signer add multiple times accumulates permissions rather than
   replacing them.

* add --merge flag to registry package signer add

  Default behavior remains overwrite. When --merge is passed, the new
  version range is OR-merged with the existing one, allowing admins to
  accumulate permissions incrementally.

* add missing attribute to TS type

* make merge optional

* upsert instead of insert

* VersionRange::None on upsert

* fix: header color in zoom

---------

Co-authored-by: Dominion5254 <musashidisciple@proton.me>

* update snake and add about this server to system general

* chore: bump sdk to beta.53, wrap z.deepPartial with passthrough

* reset instead of reset defaults

* action failure show dialog

* chore: bump sdk to beta.54, add device-info RPC, improve SDK abort handling and InputSpec filtering

- Bump SDK version to 0.4.0-beta.54
- Add `server.device-info` RPC endpoint and `s9pk select` CLI command
- Extract `HardwareRequirements::is_compatible()` method, reuse in registry filtering
- Add `AbortedError` class with `muteUnhandled` flag, replace generic abort errors
- Handle unhandled promise rejections in container-runtime with mute support
- Improve `InputSpec.filter()` with `keepByDefault` param and boolean filter values
- Accept readonly tuples in `CommandType` and `splitCommand`
- Remove `sync_host` calls from host API handlers (binding/address changes)
- Filter mDNS hostnames by secure gateway availability
- Derive mDNS enabled state from LAN IPs in web UI
- Add "Open UI" action to address table, disable mDNS toggle
- Hide debug details in service error component
- Update rpc-toolkit docs for no-params handlers

* fix: add --no-nvram to efi grub-install to preserve built-in boot order

* update snake

* diable actions when in error state

* chore: split out nvidia variant

* misc bugfixes

* create manage-release script (untested)

* fix: preserve z namespace types for sdk consumers

* sdk version bump

* new checkPort types

* multiple bugs and better port forward ux

* fix link

* chore: todos and formatting

* fix build

---------

Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>
Co-authored-by: Matt Hill <mattnine@protonmail.com>
Co-authored-by: Alex Inkin <alexander@inkin.ru>
Co-authored-by: Dominion5254 <musashidisciple@proton.me>
This commit is contained in:
Aiden McClelland
2026-03-04 04:37:31 -07:00
committed by GitHub
parent 26a68afdef
commit 3320391fcc
532 changed files with 21594 additions and 20559 deletions

View File

@@ -6,16 +6,15 @@ import {
ActionInfo,
Actions,
} from '../../base/lib/actions/setupActions'
import {
SyncOptions,
ServiceInterfaceId,
PackageId,
ServiceInterfaceType,
Effects,
} from '../../base/lib/types'
import { ServiceInterfaceType, Effects } from '../../base/lib/types'
import * as patterns from '../../base/lib/util/patterns'
import { BackupSync, Backups } from './backup/Backups'
import { smtpInputSpec } from '../../base/lib/actions/input/inputSpecConstants'
import { Backups } from './backup/Backups'
import {
smtpInputSpec,
systemSmtpSpec,
customSmtp,
smtpProviderVariants,
} from '../../base/lib/actions/input/inputSpecConstants'
import { Daemon, Daemons } from './mainFn/Daemons'
import { checkPortListening } from './health/checkFns/checkPortListening'
import { checkWebUrl, runHealthScript } from './health/checkFns'
@@ -25,10 +24,11 @@ import { setupMain } from './mainFn'
import { defaultTrigger } from './trigger/defaultTrigger'
import { changeOnFirstSuccess, cooldownTrigger } from './trigger'
import { setupServiceInterfaces } from '../../base/lib/interfaces/setupInterfaces'
import { setupExportedUrls } from '../../base/lib/interfaces/setupExportedUrls'
import { successFailure } from './trigger/successFailure'
import { MultiHost, Scheme } from '../../base/lib/interfaces/Host'
import { ServiceInterfaceBuilder } from '../../base/lib/interfaces/ServiceInterfaceBuilder'
import { GetSystemSmtp } from './util'
import { GetOutboundGateway, GetSystemSmtp } from './util'
import { nullIfEmpty } from './util'
import { getServiceInterface, getServiceInterfaces } from './util'
import {
@@ -67,7 +67,8 @@ import {
import { getOwnServiceInterfaces } from '../../base/lib/util/getServiceInterfaces'
import { Volumes, createVolumes } from './util/Volume'
export const OSVersion = testTypeVersion('0.4.0-alpha.19')
/** The minimum StartOS version required by this SDK release */
export const OSVersion = testTypeVersion('0.4.0-alpha.20')
// prettier-ignore
type AnyNeverCond<T extends any[], Then, Else> =
@@ -76,17 +77,51 @@ type AnyNeverCond<T extends any[], Then, Else> =
T extends [any, ...infer U] ? AnyNeverCond<U,Then, Else> :
never
/**
* The top-level SDK facade for building StartOS service packages.
*
* Use `StartSdk.of()` to create an uninitialized instance, then call `.withManifest()`
* to bind it to a manifest, and finally `.build()` to obtain the full toolkit of helpers
* for actions, daemons, backups, interfaces, health checks, and more.
*
* @typeParam Manifest - The service manifest type; starts as `never` until `.withManifest()` is called.
*/
export class StartSdk<Manifest extends T.SDKManifest> {
private constructor(readonly manifest: Manifest) {}
/**
* Create an uninitialized StartSdk instance. Call `.withManifest()` next.
* @returns A new StartSdk with no manifest bound.
*/
static of() {
return new StartSdk<never>(null as never)
}
/**
* Bind a manifest to the SDK, producing a typed SDK instance.
* @param manifest - The service manifest definition
* @returns A new StartSdk instance parameterized by the given manifest type
*/
withManifest<Manifest extends T.SDKManifest = never>(manifest: Manifest) {
return new StartSdk<Manifest>(manifest)
}
private ifPluginEnabled<P extends T.PluginId, T>(
plugin: P,
value: T,
): Manifest extends { plugins: P[] } ? T : null {
if (this.manifest.plugins?.includes(plugin)) return value as any
return null as any
}
/**
* Finalize the SDK and return the full set of helpers for building a StartOS service.
*
* This method is only callable after `.withManifest()` has been called (enforced at the type level).
*
* @param isReady - Type-level gate; resolves to `true` only when a manifest is bound.
* @returns An object containing all SDK utilities: actions, daemons, backups, interfaces, health checks, volumes, triggers, and more.
*/
build(isReady: AnyNeverCond<[Manifest], 'Build not ready', true>) {
type NestedEffects = 'subcontainer' | 'store' | 'action'
type NestedEffects = 'subcontainer' | 'store' | 'action' | 'plugin'
type InterfaceEffects =
| 'getServiceInterface'
| 'listServiceInterfaces'
@@ -104,6 +139,7 @@ export class StartSdk<Manifest extends T.SDKManifest> {
type AlreadyExposed =
| 'getSslCertificate'
| 'getSystemSmtp'
| 'getOutboundGateway'
| 'getContainerIp'
| 'getDataVersion'
| 'setDataVersion'
@@ -133,13 +169,19 @@ export class StartSdk<Manifest extends T.SDKManifest> {
}
return {
/** The bound service manifest */
manifest: this.manifest,
/** Volume path helpers derived from the manifest volume definitions */
volumes: createVolumes(this.manifest),
...startSdkEffectWrapper,
/** Persist the current data version to the StartOS effect system */
setDataVersion,
/** Retrieve the current data version from the StartOS effect system */
getDataVersion,
action: {
/** Execute an action by its ID, optionally providing input */
run: actions.runAction,
/** Create a task notification for a specific package's action */
createTask: <T extends ActionInfo<T.ActionId, any>>(
effects: T.Effects,
packageId: T.PackageId,
@@ -154,6 +196,7 @@ export class StartSdk<Manifest extends T.SDKManifest> {
severity,
options: options,
}),
/** Create a task notification for this service's own action (uses manifest.id automatically) */
createOwnTask: <T extends ActionInfo<T.ActionId, any>>(
effects: T.Effects,
action: T,
@@ -167,22 +210,47 @@ export class StartSdk<Manifest extends T.SDKManifest> {
severity,
options: options,
}),
/**
* Clear one or more task notifications by their replay IDs
* @param effects - The effects context
* @param replayIds - One or more replay IDs of the tasks to clear
*/
clearTask: (effects: T.Effects, ...replayIds: string[]) =>
effects.action.clearTasks({ only: replayIds }),
},
/**
* Check whether the specified (or all) dependencies are satisfied.
* @param effects - The effects context
* @param packageIds - Optional subset of dependency IDs to check; defaults to all
* @returns An object describing which dependencies are satisfied and which are not
*/
checkDependencies: checkDependencies as <
DependencyId extends keyof Manifest['dependencies'] &
PackageId = keyof Manifest['dependencies'] & PackageId,
T.PackageId = keyof Manifest['dependencies'] & T.PackageId,
>(
effects: Effects,
packageIds?: DependencyId[],
) => Promise<CheckDependencies<DependencyId>>,
serviceInterface: {
/** Retrieve a single service interface belonging to this package by its ID */
getOwn: getOwnServiceInterface,
/** Retrieve a single service interface from any package */
get: getServiceInterface,
/** Retrieve all service interfaces belonging to this package */
getAllOwn: getOwnServiceInterfaces,
/** Retrieve all service interfaces, optionally filtering by package */
getAll: getServiceInterfaces,
},
/**
* Get the container IP address with reactive subscription support.
*
* Returns an object with multiple read strategies: `const()` for a value
* that retries on change, `once()` for a single read, `watch()` for an async
* generator, `onChange()` for a callback, and `waitFor()` to block until a predicate is met.
*
* @param effects - The effects context
* @param options - Optional filtering options (e.g. `containerId`)
*/
getContainerIp: (
effects: T.Effects,
options: Omit<
@@ -275,9 +343,22 @@ export class StartSdk<Manifest extends T.SDKManifest> {
},
MultiHost: {
/**
* Create a new MultiHost instance for binding ports and exporting interfaces.
* @param effects - The effects context
* @param id - A unique identifier for this multi-host group
*/
of: (effects: Effects, id: string) => new MultiHost({ id, effects }),
},
/**
* Return `null` if the given string is empty, otherwise return the string unchanged.
* Useful for converting empty user input into explicit null values.
*/
nullIfEmpty,
/**
* Indicate that a daemon should use the container image's configured entrypoint.
* @param overrideCmd - Optional command arguments to append after the entrypoint
*/
useEntrypoint: (overrideCmd?: string[]) =>
new T.UseEntrypoint(overrideCmd),
/**
@@ -392,7 +473,12 @@ export class StartSdk<Manifest extends T.SDKManifest> {
run: Run<{}>,
) => Action.withoutInput(id, metadata, run),
},
inputSpecConstants: { smtpInputSpec },
inputSpecConstants: {
smtpInputSpec,
systemSmtpSpec,
customSmtp,
smtpProviderVariants,
},
/**
* @description Use this function to create a service interface.
* @param effects
@@ -440,19 +526,37 @@ export class StartSdk<Manifest extends T.SDKManifest> {
masked: boolean
},
) => new ServiceInterfaceBuilder({ ...options, effects }),
/**
* Get the system SMTP configuration with reactive subscription support.
* @param effects - The effects context
*/
getSystemSmtp: <E extends Effects>(effects: E) =>
new GetSystemSmtp(effects),
/**
* Get the outbound network gateway address with reactive subscription support.
* @param effects - The effects context
*/
getOutboundGateway: <E extends Effects>(effects: E) =>
new GetOutboundGateway(effects),
/**
* Get an SSL certificate for the given hostnames with reactive subscription support.
* @param effects - The effects context
* @param hostnames - The hostnames to obtain a certificate for
* @param algorithm - Optional algorithm preference (e.g. Ed25519)
*/
getSslCertificate: <E extends Effects>(
effects: E,
hostnames: string[],
algorithm?: T.Algorithm,
) => new GetSslCertificate(effects, hostnames, algorithm),
/** Retrieve the manifest of any installed service package by its ID */
getServiceManifest,
healthCheck: {
checkPortListening,
checkWebUrl,
runHealthScript,
},
/** Common utility patterns (e.g. hostname regex, port validators) */
patterns,
/**
* @description Use this function to list every Action offered by the service. Actions will be displayed in the provided order.
@@ -632,21 +736,47 @@ export class StartSdk<Manifest extends T.SDKManifest> {
* ```
*/
setupInterfaces: setupServiceInterfaces,
/**
* Define the main entrypoint for the service. The provided function should
* configure and return a `Daemons` instance describing all long-running processes.
* @param fn - Async function that receives `effects` and returns a `Daemons` instance
*/
setupMain: (
fn: (o: { effects: Effects }) => Promise<Daemons<Manifest, any>>,
) => setupMain<Manifest>(fn),
/** Built-in trigger strategies for controlling health-check polling intervals */
trigger: {
/** Default trigger: polls at a fixed interval */
defaultTrigger,
/** Trigger with a cooldown period between checks */
cooldownTrigger,
/** Switches to a different interval after the first successful check */
changeOnFirstSuccess,
/** Uses different intervals based on success vs failure results */
successFailure,
},
Mounts: {
/**
* Create an empty Mounts builder for declaring volume, asset, dependency, and backup mounts.
* @returns A new Mounts instance with no mounts configured
*/
of: Mounts.of<Manifest>,
},
Backups: {
/**
* Create a Backups configuration that backs up entire volumes by name.
* @param volumeNames - Volume IDs from the manifest to include in backups
*/
ofVolumes: Backups.ofVolumes<Manifest>,
/**
* Create a Backups configuration from explicit sync path pairs.
* @param syncs - Array of `{ dataPath, backupPath }` objects
*/
ofSyncs: Backups.ofSyncs<Manifest>,
/**
* Create a Backups configuration with custom rsync options (e.g. exclude patterns).
* @param options - Partial sync options to override defaults
*/
withOptions: Backups.withOptions<Manifest>,
},
InputSpec: {
@@ -681,11 +811,20 @@ export class StartSdk<Manifest extends T.SDKManifest> {
InputSpec.of<Spec>(spec),
},
Daemon: {
/**
* Create a single Daemon that wraps a long-running process with automatic restart logic.
* Returns a curried function: call with `(effects, subcontainer, exec)`.
*/
get of() {
return Daemon.of<Manifest>()
},
},
Daemons: {
/**
* Create a new Daemons builder for defining the service's daemon topology.
* Chain `.addDaemon()` calls to register each long-running process.
* @param effects - The effects context
*/
of(effects: Effects) {
return Daemons.of<Manifest>({ effects })
},
@@ -737,10 +876,74 @@ export class StartSdk<Manifest extends T.SDKManifest> {
List,
Value,
Variants,
plugin: {
url: this.ifPluginEnabled('url-v0' as const, {
register: (
effects: T.Effects,
options: {
tableAction: ActionInfo<
T.ActionId,
{
urlPluginMetadata: {
packageId: T.PackageId
interfaceId: T.ServiceInterfaceId
hostId: T.HostId
internalPort: number
}
}
>
},
) =>
effects.plugin.url.register({
tableAction: options.tableAction.id,
}),
exportUrl: (
effects: T.Effects,
options: {
hostnameInfo: T.PluginHostnameInfo
removeAction: ActionInfo<
T.ActionId,
{
urlPluginMetadata: T.PluginHostnameInfo & {
interfaceId: T.ServiceInterfaceId
}
}
> | null
overflowActions: ActionInfo<
T.ActionId,
{
urlPluginMetadata: T.PluginHostnameInfo & {
interfaceId: T.ServiceInterfaceId
}
}
>[]
},
) =>
effects.plugin.url.exportUrl({
hostnameInfo: options.hostnameInfo,
removeAction: options.removeAction?.id ?? null,
overflowActions: options.overflowActions.map((a) => a.id),
}),
setupExportedUrls, // similar to setupInterfaces
}),
},
}
}
}
/**
* Run a one-shot command inside a temporary subcontainer.
*
* Creates a subcontainer, executes the command, and destroys the subcontainer when finished.
* Throws an {@link ExitError} if the command exits with a non-zero code or signal.
*
* @param effects - The effects context
* @param image - The container image to use
* @param command - The command to execute (string array or UseEntrypoint)
* @param options - Mount and command options
* @param name - Optional human-readable name for debugging
* @returns The stdout and stderr output of the command
*/
export async function runCommand<Manifest extends T.SDKManifest>(
effects: Effects,
image: { imageId: keyof Manifest['images'] & T.ImageId; sharedRun?: boolean },

View File

@@ -5,10 +5,12 @@ import { Affine, asError } from '../util'
import { ExtendedVersion, VersionRange } from '../../../base/lib'
import { InitKind, InitScript } from '../../../base/lib/inits'
/** Default rsync options used for backup and restore operations */
export const DEFAULT_OPTIONS: T.SyncOptions = {
delete: true,
exclude: [],
}
/** A single source-to-destination sync pair for backup and restore */
export type BackupSync<Volumes extends string> = {
dataPath: `/media/startos/volumes/${Volumes}/${string}`
backupPath: `/media/startos/backup/${string}`
@@ -17,8 +19,18 @@ export type BackupSync<Volumes extends string> = {
restoreOptions?: Partial<T.SyncOptions>
}
/** Effects type narrowed for backup/restore contexts, preventing reuse outside that scope */
export type BackupEffects = T.Effects & Affine<'Backups'>
/**
* Configures backup and restore operations using rsync.
*
* Supports syncing entire volumes or custom path pairs, with optional pre/post hooks
* for both backup and restore phases. Implements {@link InitScript} so it can be used
* as a restore-init step in `setupInit`.
*
* @typeParam M - The service manifest type
*/
export class Backups<M extends T.SDKManifest> implements InitScript {
private constructor(
private options = DEFAULT_OPTIONS,
@@ -31,6 +43,11 @@ export class Backups<M extends T.SDKManifest> implements InitScript {
private postRestore = async (effects: BackupEffects) => {},
) {}
/**
* Create a Backups configuration that backs up entire volumes by name.
* Each volume is synced to a corresponding directory under `/media/startos/backup/volumes/`.
* @param volumeNames - One or more volume IDs from the manifest
*/
static ofVolumes<M extends T.SDKManifest = never>(
...volumeNames: Array<M['volumes'][number]>
): Backups<M> {
@@ -42,18 +59,31 @@ export class Backups<M extends T.SDKManifest> implements InitScript {
)
}
/**
* Create a Backups configuration from explicit source/destination sync pairs.
* @param syncs - Array of `{ dataPath, backupPath }` objects with optional per-sync options
*/
static ofSyncs<M extends T.SDKManifest = never>(
...syncs: BackupSync<M['volumes'][number]>[]
) {
return syncs.reduce((acc, x) => acc.addSync(x), new Backups<M>())
}
/**
* Create an empty Backups configuration with custom default rsync options.
* Chain `.addVolume()` or `.addSync()` to add sync targets.
* @param options - Partial rsync options to override defaults (e.g. `{ exclude: ['cache'] }`)
*/
static withOptions<M extends T.SDKManifest = never>(
options?: Partial<T.SyncOptions>,
) {
return new Backups<M>({ ...DEFAULT_OPTIONS, ...options })
}
/**
* Override the default rsync options for both backup and restore.
* @param options - Partial rsync options to merge with current defaults
*/
setOptions(options?: Partial<T.SyncOptions>) {
this.options = {
...this.options,
@@ -62,6 +92,10 @@ export class Backups<M extends T.SDKManifest> implements InitScript {
return this
}
/**
* Override rsync options used only during backup (not restore).
* @param options - Partial rsync options for the backup phase
*/
setBackupOptions(options?: Partial<T.SyncOptions>) {
this.backupOptions = {
...this.backupOptions,
@@ -70,6 +104,10 @@ export class Backups<M extends T.SDKManifest> implements InitScript {
return this
}
/**
* Override rsync options used only during restore (not backup).
* @param options - Partial rsync options for the restore phase
*/
setRestoreOptions(options?: Partial<T.SyncOptions>) {
this.restoreOptions = {
...this.restoreOptions,
@@ -78,26 +116,47 @@ export class Backups<M extends T.SDKManifest> implements InitScript {
return this
}
/**
* Register a hook to run before backup rsync begins (e.g. dump a database).
* @param fn - Async function receiving backup-scoped effects
*/
setPreBackup(fn: (effects: BackupEffects) => Promise<void>) {
this.preBackup = fn
return this
}
/**
* Register a hook to run after backup rsync completes.
* @param fn - Async function receiving backup-scoped effects
*/
setPostBackup(fn: (effects: BackupEffects) => Promise<void>) {
this.postBackup = fn
return this
}
/**
* Register a hook to run before restore rsync begins.
* @param fn - Async function receiving backup-scoped effects
*/
setPreRestore(fn: (effects: BackupEffects) => Promise<void>) {
this.preRestore = fn
return this
}
/**
* Register a hook to run after restore rsync completes.
* @param fn - Async function receiving backup-scoped effects
*/
setPostRestore(fn: (effects: BackupEffects) => Promise<void>) {
this.postRestore = fn
return this
}
/**
* Add a volume to the backup set by its ID.
* @param volume - The volume ID from the manifest
* @param options - Optional per-volume rsync overrides
*/
addVolume(
volume: M['volumes'][number],
options?: Partial<{
@@ -113,11 +172,19 @@ export class Backups<M extends T.SDKManifest> implements InitScript {
})
}
/**
* Add a custom sync pair to the backup set.
* @param sync - A `{ dataPath, backupPath }` object with optional per-sync rsync options
*/
addSync(sync: BackupSync<M['volumes'][0]>) {
this.backupSet.push(sync)
return this
}
/**
* Execute the backup: runs pre-hook, rsyncs all configured paths, saves the data version, then runs post-hook.
* @param effects - The effects context
*/
async createBackup(effects: T.Effects) {
await this.preBackup(effects as BackupEffects)
for (const item of this.backupSet) {
@@ -149,6 +216,10 @@ export class Backups<M extends T.SDKManifest> implements InitScript {
}
}
/**
* Execute the restore: runs pre-hook, rsyncs all configured paths from backup to data, restores the data version, then runs post-hook.
* @param effects - The effects context
*/
async restoreBackup(effects: T.Effects) {
this.preRestore(effects as BackupEffects)

View File

@@ -3,6 +3,11 @@ import * as T from '../../../base/lib/types'
import { _ } from '../util'
import { InitScript } from '../../../base/lib/inits'
/**
* Parameters for `setupBackups`. Either:
* - An array of volume IDs to back up entirely, or
* - An async factory function that returns a fully configured {@link Backups} instance
*/
export type SetupBackupsParams<M extends T.SDKManifest> =
| M['volumes'][number][]
| ((_: { effects: T.Effects }) => Promise<Backups<M>>)
@@ -12,6 +17,15 @@ type SetupBackupsRes = {
restoreInit: InitScript
}
/**
* Set up backup and restore exports for the service.
*
* Returns `{ createBackup, restoreInit }` which should be exported and wired into
* the service's init and backup entry points.
*
* @param options - Either an array of volume IDs or an async factory returning a Backups instance
* @returns An object with `createBackup` (the backup export) and `restoreInit` (an InitScript for restore)
*/
export function setupBackups<M extends T.SDKManifest>(
options: SetupBackupsParams<M>,
) {

View File

@@ -4,8 +4,8 @@ import { Trigger } from '../trigger'
import { TriggerInput } from '../trigger/TriggerInput'
import { defaultTrigger } from '../trigger/defaultTrigger'
import { once, asError, Drop } from '../util'
import { object, unknown } from 'ts-matches'
/** Parameters for creating a health check */
export type HealthCheckParams = {
id: HealthCheckId
name: string
@@ -14,6 +14,13 @@ export type HealthCheckParams = {
fn(): Promise<HealthCheckResult> | HealthCheckResult
}
/**
* A periodic health check that reports daemon readiness to the StartOS UI.
*
* Polls at an interval controlled by a {@link Trigger}, reporting results as
* "starting" (during the grace period), "success", or "failure". Automatically
* pauses when the daemon is stopped and resumes when restarted.
*/
export class HealthCheck extends Drop {
private started: number | null = null
private setStarted = (started: number | null) => {
@@ -92,13 +99,21 @@ export class HealthCheck extends Drop {
}
})
}
/**
* Create a new HealthCheck instance and begin its polling loop.
* @param effects - The effects context for reporting health status
* @param options - Health check configuration (ID, name, check function, trigger, grace period)
* @returns A new HealthCheck instance
*/
static of(effects: Effects, options: HealthCheckParams): HealthCheck {
return new HealthCheck(effects, options)
}
/** Signal that the daemon is running, enabling health check polling */
start() {
if (this.started) return
this.setStarted(performance.now())
}
/** Signal that the daemon has stopped, pausing health check polling */
stop() {
if (!this.started) return
this.setStarted(null)
@@ -109,7 +124,8 @@ export class HealthCheck extends Drop {
}
function asMessage(e: unknown) {
if (object({ message: unknown }).test(e)) return String(e.message)
if (typeof e === 'object' && e !== null && 'message' in e)
return String((e as any).message)
const value = String(e)
if (value.length == null) return null
return value

View File

@@ -1,3 +1,9 @@
import { T } from '../../../../base/lib'
/**
* The result of a single health check invocation.
*
* Contains a `result` field ("success", "failure", or "starting") and an optional `message`.
* This is the unnamed variant -- the health check name is added by the framework.
*/
export type HealthCheckResult = Omit<T.NamedHealthCheckResult, 'name'>

View File

@@ -3,6 +3,14 @@ export { checkPortListening } from './checkPortListening'
export { HealthCheckResult } from './HealthCheckResult'
export { checkWebUrl } from './checkWebUrl'
/**
* Create a promise that rejects after the specified timeout.
* Useful for racing against long-running health checks.
*
* @param ms - Timeout duration in milliseconds
* @param options.message - Custom error message (defaults to "Timed out")
* @returns A promise that never resolves, only rejects after the timeout
*/
export function timeoutPromise(ms: number, { message = 'Timed out' } = {}) {
return new Promise<never>((resolve, reject) =>
setTimeout(() => reject(new Error(message)), ms),

View File

@@ -7,7 +7,7 @@ import {
ISB,
IST,
types,
matches,
z,
utils,
} from '../../base/lib'
@@ -20,7 +20,7 @@ export {
ISB,
IST,
types,
matches,
z,
utils,
}
export { setupI18n } from './i18n'

View File

@@ -8,6 +8,15 @@ import * as cp from 'child_process'
import * as fs from 'node:fs/promises'
import { DaemonCommandType, ExecCommandOptions, ExecFnOptions } from './Daemons'
/**
* Low-level controller for a single running process inside a subcontainer (or as a JS function).
*
* Manages the child process lifecycle: spawning, waiting, and signal-based termination.
* Used internally by {@link Daemon} to manage individual command executions.
*
* @typeParam Manifest - The service manifest type
* @typeParam C - The subcontainer type, or `null` for JS-only commands
*/
export class CommandController<
Manifest extends T.SDKManifest,
C extends SubContainer<Manifest> | null,
@@ -21,6 +30,13 @@ export class CommandController<
) {
super()
}
/**
* Factory method to create a new CommandController.
*
* Returns a curried async function: `(effects, subcontainer, exec) => CommandController`.
* If the exec spec has an `fn` property, runs the function; otherwise spawns a shell command
* in the subcontainer.
*/
static of<
Manifest extends T.SDKManifest,
C extends SubContainer<Manifest> | null,
@@ -130,6 +146,10 @@ export class CommandController<
}
}
}
/**
* Wait for the command to finish. Optionally terminate after a timeout.
* @param options.timeout - Milliseconds to wait before terminating. Defaults to no timeout.
*/
async wait({ timeout = NO_TIMEOUT } = {}) {
if (timeout > 0)
setTimeout(() => {
@@ -156,6 +176,15 @@ export class CommandController<
await this.subcontainer?.destroy()
}
}
/**
* Terminate the running command by sending a signal.
*
* Sends the specified signal (default: SIGTERM), then escalates to SIGKILL
* after the timeout expires. Destroys the subcontainer after the process exits.
*
* @param options.signal - The signal to send (default: SIGTERM)
* @param options.timeout - Milliseconds before escalating to SIGKILL
*/
async term({ signal = SIGTERM, timeout = this.sigtermTimeout } = {}) {
try {
if (!this.state.exited) {

View File

@@ -13,10 +13,15 @@ import { Oneshot } from './Oneshot'
const TIMEOUT_INCREMENT_MS = 1000
const MAX_TIMEOUT_MS = 30000
/**
* This is a wrapper around CommandController that has a state of off, where the command shouldn't be running
* and the others state of running, where it will keep a living running command
* A managed long-running process wrapper around {@link CommandController}.
*
* When started, the daemon automatically restarts its underlying command on failure
* with exponential backoff (up to 30 seconds). When stopped, the command is terminated
* gracefully. Implements {@link Drop} for automatic cleanup when the context is left.
*
* @typeParam Manifest - The service manifest type
* @typeParam C - The subcontainer type, or `null` for JS-only daemons
*/
export class Daemon<
Manifest extends T.SDKManifest,
C extends SubContainer<Manifest> | null = SubContainer<Manifest> | null,
@@ -33,9 +38,16 @@ export class Daemon<
) {
super()
}
/** Returns true if this daemon is a one-shot process (exits after success) */
isOneshot(): this is Oneshot<Manifest> {
return this.oneshot
}
/**
* Factory method to create a new Daemon.
*
* Returns a curried function: `(effects, subcontainer, exec) => Daemon`.
* The daemon auto-terminates when the effects context is left.
*/
static of<Manifest extends T.SDKManifest>() {
return <C extends SubContainer<Manifest> | null>(
effects: T.Effects,
@@ -57,6 +69,12 @@ export class Daemon<
return res
}
}
/**
* Start the daemon. If it is already running, this is a no-op.
*
* The daemon will automatically restart on failure with increasing backoff
* until {@link term} is called.
*/
async start() {
if (this.commandController) {
return
@@ -105,6 +123,17 @@ export class Daemon<
console.error(asError(err))
})
}
/**
* Terminate the daemon, stopping its underlying command.
*
* Sends the configured signal (default SIGTERM) and waits for the process to exit.
* Optionally destroys the subcontainer after termination.
*
* @param termOptions - Optional termination settings
* @param termOptions.signal - The signal to send (default: SIGTERM)
* @param termOptions.timeout - Milliseconds to wait before SIGKILL
* @param termOptions.destroySubcontainer - Whether to destroy the subcontainer after exit
*/
async term(termOptions?: {
signal?: NodeJS.Signals | undefined
timeout?: number | undefined
@@ -125,14 +154,20 @@ export class Daemon<
this.exiting = null
}
}
/** Get a reference-counted handle to the daemon's subcontainer, or null if there is none */
subcontainerRc(): SubContainerRc<Manifest> | null {
return this.subcontainer?.rc() ?? null
}
/** Check whether this daemon shares the same subcontainer as another daemon */
sharesSubcontainerWith(
other: Daemon<Manifest, SubContainer<Manifest> | null>,
): boolean {
return this.subcontainer?.guid === other.subcontainer?.guid
}
/**
* Register a callback to be invoked each time the daemon's process exits.
* @param fn - Callback receiving `true` on clean exit, `false` on error
*/
onExit(fn: (success: boolean) => void) {
this.onExitFns.push(fn)
}

View File

@@ -16,8 +16,15 @@ import { Daemon } from './Daemon'
import { CommandController } from './CommandController'
import { Oneshot } from './Oneshot'
/** Promisified version of `child_process.exec` */
export const cpExec = promisify(CP.exec)
/** Promisified version of `child_process.execFile` */
export const cpExecFile = promisify(CP.execFile)
/**
* Configuration for a daemon's health-check readiness probe.
*
* Determines how the system knows when a daemon is healthy and ready to serve.
*/
export type Ready = {
/** A human-readable display name for the health check. If null, the health check itself will be from the UI */
display: string | null
@@ -45,6 +52,10 @@ export type Ready = {
trigger?: Trigger
}
/**
* Options for running a daemon as a shell command inside a subcontainer.
* Includes the command to run, optional signal/timeout, environment, user, and stdio callbacks.
*/
export type ExecCommandOptions = {
command: T.CommandType
// Defaults to the DEFAULT_SIGTERM_TIMEOUT = 30_000ms
@@ -61,6 +72,11 @@ export type ExecCommandOptions = {
onStderr?: (chunk: Buffer | string | any) => void
}
/**
* Options for running a daemon via an async function that may optionally return
* a command to execute in the subcontainer. The function receives an `AbortSignal`
* for cooperative cancellation.
*/
export type ExecFnOptions<
Manifest extends T.SDKManifest,
C extends SubContainer<Manifest> | null,
@@ -73,6 +89,10 @@ export type ExecFnOptions<
sigtermTimeout?: number
}
/**
* The execution specification for a daemon: either an {@link ExecFnOptions} (async function)
* or an {@link ExecCommandOptions} (shell command, only valid when a subcontainer is provided).
*/
export type DaemonCommandType<
Manifest extends T.SDKManifest,
C extends SubContainer<Manifest> | null,
@@ -385,6 +405,13 @@ export class Daemons<Manifest extends T.SDKManifest, Ids extends string>
return null
}
/**
* Gracefully terminate all daemons in reverse dependency order.
*
* Daemons with no remaining dependents are shut down first, proceeding
* until all daemons have been terminated. Falls back to a bulk shutdown
* if a dependency cycle is detected.
*/
async term() {
const remaining = new Set(this.healthDaemons)
@@ -427,6 +454,10 @@ export class Daemons<Manifest extends T.SDKManifest, Ids extends string>
}
}
/**
* Start all registered daemons and their health checks.
* @returns This `Daemons` instance, now running
*/
async build() {
for (const daemon of this.healthDaemons) {
await daemon.updateStatus()

View File

@@ -49,6 +49,15 @@ type DependencyOpts<Manifest extends T.SDKManifest> = {
readonly: boolean
} & SharedOptions
/**
* Immutable builder for declaring filesystem mounts into a subcontainer.
*
* Supports mounting volumes, static assets, dependency volumes, and backup directories.
* Each `mount*` method returns a new `Mounts` instance (immutable builder pattern).
*
* @typeParam Manifest - The service manifest type
* @typeParam Backups - Tracks whether backup mounts have been added (type-level flag)
*/
export class Mounts<
Manifest extends T.SDKManifest,
Backups extends SharedOptions = never,
@@ -60,10 +69,19 @@ export class Mounts<
readonly backups: Backups[],
) {}
/**
* Create an empty Mounts builder with no mounts configured.
* @returns A new Mounts instance ready for chaining mount declarations
*/
static of<Manifest extends T.SDKManifest>() {
return new Mounts<Manifest>([], [], [], [])
}
/**
* Add a volume mount from the service's own volumes.
* @param options - Volume ID, mountpoint, readonly flag, and optional subpath
* @returns A new Mounts instance with this volume added
*/
mountVolume(options: VolumeOpts<Manifest>) {
return new Mounts<Manifest, Backups>(
[...this.volumes, options],
@@ -73,6 +91,11 @@ export class Mounts<
)
}
/**
* Add a read-only mount of the service's packaged static assets.
* @param options - Mountpoint and optional subpath within the assets directory
* @returns A new Mounts instance with this asset mount added
*/
mountAssets(options: SharedOptions) {
return new Mounts<Manifest, Backups>(
[...this.volumes],
@@ -82,6 +105,11 @@ export class Mounts<
)
}
/**
* Add a mount from a dependency package's volume.
* @param options - Dependency ID, volume ID, mountpoint, readonly flag, and optional subpath
* @returns A new Mounts instance with this dependency mount added
*/
mountDependency<DependencyManifest extends T.SDKManifest>(
options: DependencyOpts<DependencyManifest>,
) {
@@ -93,6 +121,11 @@ export class Mounts<
)
}
/**
* Add a mount of the backup directory. Only valid during backup/restore operations.
* @param options - Mountpoint and optional subpath within the backup directory
* @returns A new Mounts instance with this backup mount added
*/
mountBackups(options: SharedOptions) {
return new Mounts<
Manifest,
@@ -108,6 +141,11 @@ export class Mounts<
)
}
/**
* Compile all declared mounts into the low-level mount array consumed by the subcontainer runtime.
* @throws If any two mounts share the same mountpoint
* @returns An array of `{ mountpoint, options }` objects
*/
build(): MountArray {
const mountpoints = new Set()
for (let mountpoint of this.volumes

View File

@@ -3,6 +3,7 @@ import { Daemons } from './Daemons'
import '../../../base/lib/interfaces/ServiceInterfaceBuilder'
import '../../../base/lib/interfaces/Origin'
/** Default time in milliseconds to wait for a process to exit after SIGTERM before escalating to SIGKILL */
export const DEFAULT_SIGTERM_TIMEOUT = 60_000
/**
* Used to ensure that the main function is running with the valid proofs.

View File

@@ -24,6 +24,15 @@ export function setupManifest<
return manifest
}
/**
* Build the final publishable manifest by combining the SDK manifest definition
* with version graph metadata, OS version, SDK version, and computed fields
* (migration ranges, hardware requirements, alerts, etc.).
*
* @param versions - The service's VersionGraph, used to extract the current version, release notes, and migration ranges
* @param manifest - The SDK manifest definition (from `setupManifest`)
* @returns A fully resolved Manifest ready for packaging
*/
export function buildManifest<
Id extends string,
Version extends string,
@@ -89,5 +98,6 @@ export function buildManifest<
),
},
hardwareAcceleration: manifest.hardwareAcceleration ?? false,
plugins: manifest.plugins ?? [],
}
}

View File

@@ -1,4 +1,3 @@
import { testOutput } from './output.test'
import { InputSpec } from '../../../base/lib/actions/input/builder/inputSpec'
import { List } from '../../../base/lib/actions/input/builder/list'
import { Value } from '../../../base/lib/actions/input/builder/value'
@@ -7,6 +6,12 @@ import { ValueSpec } from '../../../base/lib/actions/input/inputSpecTypes'
import { setupManifest } from '../manifest/setupManifest'
import { StartSdk } from '../StartSdk'
export type IfEquals<T, U, Y = unknown, N = never> =
(<G>() => G extends T ? 1 : 2) extends <G>() => G extends U ? 1 : 2 ? Y : N
export function testOutput<A, B>(): (c: IfEquals<A, B>) => null {
return () => null
}
describe('builder tests', () => {
test('text', async () => {
const bitcoinPropertiesBuilt: {
@@ -50,8 +55,8 @@ describe('values', () => {
default: false,
}).build({} as any)
const validator = value.validator
validator.unsafeCast(false)
testOutput<typeof validator._TYPE, boolean>()(null)
validator.parse(false)
testOutput<typeof validator._output, boolean>()(null)
})
test('text', async () => {
const value = await Value.text({
@@ -61,9 +66,9 @@ describe('values', () => {
}).build({} as any)
const validator = value.validator
const rawIs = value.spec
validator.unsafeCast('test text')
expect(() => validator.unsafeCast(null)).toThrowError()
testOutput<typeof validator._TYPE, string>()(null)
validator.parse('test text')
expect(() => validator.parse(null)).toThrowError()
testOutput<typeof validator._output, string>()(null)
})
test('text with default', async () => {
const value = await Value.text({
@@ -73,9 +78,9 @@ describe('values', () => {
}).build({} as any)
const validator = value.validator
const rawIs = value.spec
validator.unsafeCast('test text')
expect(() => validator.unsafeCast(null)).toThrowError()
testOutput<typeof validator._TYPE, string>()(null)
validator.parse('test text')
expect(() => validator.parse(null)).toThrowError()
testOutput<typeof validator._output, string>()(null)
})
test('optional text', async () => {
const value = await Value.text({
@@ -85,9 +90,9 @@ describe('values', () => {
}).build({} as any)
const validator = value.validator
const rawIs = value.spec
validator.unsafeCast('test text')
validator.unsafeCast(null)
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('test text')
validator.parse(null)
testOutput<typeof validator._output, string | null>()(null)
})
test('color', async () => {
const value = await Value.color({
@@ -98,8 +103,8 @@ describe('values', () => {
warning: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast('#000000')
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('#000000')
testOutput<typeof validator._output, string | null>()(null)
})
test('datetime', async () => {
const value = await Value.datetime({
@@ -113,8 +118,8 @@ describe('values', () => {
max: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast('2021-01-01')
testOutput<typeof validator._TYPE, string>()(null)
validator.parse('2021-01-01')
testOutput<typeof validator._output, string>()(null)
})
test('optional datetime', async () => {
const value = await Value.datetime({
@@ -128,8 +133,8 @@ describe('values', () => {
max: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast('2021-01-01')
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('2021-01-01')
testOutput<typeof validator._output, string | null>()(null)
})
test('textarea', async () => {
const value = await Value.textarea({
@@ -145,8 +150,8 @@ describe('values', () => {
placeholder: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast('test text')
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('test text')
testOutput<typeof validator._output, string | null>()(null)
})
test('number', async () => {
const value = await Value.number({
@@ -163,8 +168,8 @@ describe('values', () => {
placeholder: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast(2)
testOutput<typeof validator._TYPE, number>()(null)
validator.parse(2)
testOutput<typeof validator._output, number>()(null)
})
test('optional number', async () => {
const value = await Value.number({
@@ -181,8 +186,8 @@ describe('values', () => {
placeholder: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast(2)
testOutput<typeof validator._TYPE, number | null>()(null)
validator.parse(2)
testOutput<typeof validator._output, number | null>()(null)
})
test('select', async () => {
const value = await Value.select({
@@ -196,10 +201,10 @@ describe('values', () => {
warning: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast('a')
validator.unsafeCast('b')
expect(() => validator.unsafeCast('c')).toThrowError()
testOutput<typeof validator._TYPE, 'a' | 'b'>()(null)
validator.parse('a')
validator.parse('b')
expect(() => validator.parse('c')).toThrowError()
testOutput<typeof validator._output, 'a' | 'b'>()(null)
})
test('nullable select', async () => {
const value = await Value.select({
@@ -213,9 +218,9 @@ describe('values', () => {
warning: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast('a')
validator.unsafeCast('b')
testOutput<typeof validator._TYPE, 'a' | 'b'>()(null)
validator.parse('a')
validator.parse('b')
testOutput<typeof validator._output, 'a' | 'b'>()(null)
})
test('multiselect', async () => {
const value = await Value.multiselect({
@@ -231,12 +236,12 @@ describe('values', () => {
maxLength: null,
}).build({} as any)
const validator = value.validator
validator.unsafeCast([])
validator.unsafeCast(['a', 'b'])
validator.parse([])
validator.parse(['a', 'b'])
expect(() => validator.unsafeCast(['e'])).toThrowError()
expect(() => validator.unsafeCast([4])).toThrowError()
testOutput<typeof validator._TYPE, Array<'a' | 'b'>>()(null)
expect(() => validator.parse(['e'])).toThrowError()
expect(() => validator.parse([4])).toThrowError()
testOutput<typeof validator._output, Array<'a' | 'b'>>()(null)
})
test('object', async () => {
const value = await Value.object(
@@ -254,8 +259,8 @@ describe('values', () => {
}),
).build({} as any)
const validator = value.validator
validator.unsafeCast({ a: true })
testOutput<typeof validator._TYPE, { a: boolean }>()(null)
validator.parse({ a: true })
testOutput<typeof validator._output, { a: boolean }>()(null)
})
test('union', async () => {
const value = await Value.union({
@@ -278,8 +283,8 @@ describe('values', () => {
}),
}).build({} as any)
const validator = value.validator
validator.unsafeCast({ selection: 'a', value: { b: false } })
type Test = typeof validator._TYPE
validator.parse({ selection: 'a', value: { b: false } })
type Test = typeof validator._output
testOutput<
Test,
{
@@ -306,9 +311,9 @@ describe('values', () => {
default: false,
})).build({} as any)
const validator = value.validator
validator.unsafeCast(false)
expect(() => validator.unsafeCast(null)).toThrowError()
testOutput<typeof validator._TYPE, boolean>()(null)
validator.parse(false)
expect(() => validator.parse(null)).toThrowError()
testOutput<typeof validator._output, boolean>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
description: null,
@@ -324,9 +329,9 @@ describe('values', () => {
})).build({} as any)
const validator = value.validator
const rawIs = value.spec
validator.unsafeCast('test text')
validator.unsafeCast(null)
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('test text')
validator.parse(null)
testOutput<typeof validator._output, string | null>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
required: false,
@@ -340,9 +345,9 @@ describe('values', () => {
default: 'this is a default value',
})).build({} as any)
const validator = value.validator
validator.unsafeCast('test text')
validator.unsafeCast(null)
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('test text')
validator.parse(null)
testOutput<typeof validator._output, string | null>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
required: false,
@@ -357,9 +362,9 @@ describe('values', () => {
})).build({} as any)
const validator = value.validator
const rawIs = value.spec
validator.unsafeCast('test text')
validator.unsafeCast(null)
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('test text')
validator.parse(null)
testOutput<typeof validator._output, string | null>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
required: false,
@@ -375,9 +380,9 @@ describe('values', () => {
warning: null,
})).build({} as any)
const validator = value.validator
validator.unsafeCast('#000000')
validator.unsafeCast(null)
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('#000000')
validator.parse(null)
testOutput<typeof validator._output, string | null>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
required: false,
@@ -393,12 +398,11 @@ describe('values', () => {
id: 'testOutput',
title: '',
license: '',
wrapperRepo: '',
packageRepo: '',
upstreamRepo: '',
supportSite: '',
marketingSite: '',
marketingUrl: '',
donationUrl: null,
docsUrl: '',
docsUrls: [],
description: {
short: '',
long: '',
@@ -433,9 +437,9 @@ describe('values', () => {
}
}).build({} as any)
const validator = value.validator
validator.unsafeCast('2021-01-01')
validator.unsafeCast(null)
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('2021-01-01')
validator.parse(null)
testOutput<typeof validator._output, string | null>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
required: false,
@@ -459,8 +463,8 @@ describe('values', () => {
placeholder: null,
})).build({} as any)
const validator = value.validator
validator.unsafeCast('test text')
testOutput<typeof validator._TYPE, string | null>()(null)
validator.parse('test text')
testOutput<typeof validator._output, string | null>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
required: false,
@@ -481,10 +485,10 @@ describe('values', () => {
placeholder: null,
})).build({} as any)
const validator = value.validator
validator.unsafeCast(2)
validator.unsafeCast(null)
expect(() => validator.unsafeCast('null')).toThrowError()
testOutput<typeof validator._TYPE, number | null>()(null)
validator.parse(2)
validator.parse(null)
expect(() => validator.parse('null')).toThrowError()
testOutput<typeof validator._output, number | null>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
required: false,
@@ -502,9 +506,9 @@ describe('values', () => {
warning: null,
})).build({} as any)
const validator = value.validator
validator.unsafeCast('a')
validator.unsafeCast('b')
testOutput<typeof validator._TYPE, 'a' | 'b'>()(null)
validator.parse('a')
validator.parse('b')
testOutput<typeof validator._output, 'a' | 'b'>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
})
@@ -523,12 +527,12 @@ describe('values', () => {
maxLength: null,
})).build({} as any)
const validator = value.validator
validator.unsafeCast([])
validator.unsafeCast(['a', 'b'])
validator.parse([])
validator.parse(['a', 'b'])
expect(() => validator.unsafeCast([4])).toThrowError()
expect(() => validator.unsafeCast(null)).toThrowError()
testOutput<typeof validator._TYPE, Array<'a' | 'b'>>()(null)
expect(() => validator.parse([4])).toThrowError()
expect(() => validator.parse(null)).toThrowError()
testOutput<typeof validator._output, Array<'a' | 'b'>>()(null)
expect(value.spec).toMatchObject({
name: 'Testing',
default: [],
@@ -569,8 +573,8 @@ describe('values', () => {
}),
})).build({} as any)
const validator = value.validator
validator.unsafeCast({ selection: 'a', value: { b: false } })
type Test = typeof validator._TYPE
validator.parse({ selection: 'a', value: { b: false } })
type Test = typeof validator._output
testOutput<
Test,
| {
@@ -654,8 +658,8 @@ describe('values', () => {
}),
})).build({} as any)
const validator = value.validator
validator.unsafeCast({ selection: 'a', value: { b: false } })
type Test = typeof validator._TYPE
validator.parse({ selection: 'a', value: { b: false } })
type Test = typeof validator._output
testOutput<
Test,
| {
@@ -727,8 +731,8 @@ describe('Builder List', () => {
),
).build({} as any)
const validator = value.validator
validator.unsafeCast([{ test: true }])
testOutput<typeof validator._TYPE, { test: boolean }[]>()(null)
validator.parse([{ test: true }])
testOutput<typeof validator._output, { test: boolean }[]>()(null)
})
test('text', async () => {
const value = await Value.list(
@@ -742,8 +746,8 @@ describe('Builder List', () => {
),
).build({} as any)
const validator = value.validator
validator.unsafeCast(['test', 'text'])
testOutput<typeof validator._TYPE, string[]>()(null)
validator.parse(['test', 'text'])
testOutput<typeof validator._output, string[]>()(null)
})
describe('dynamic', () => {
test('text', async () => {
@@ -754,10 +758,10 @@ describe('Builder List', () => {
})),
).build({} as any)
const validator = value.validator
validator.unsafeCast(['test', 'text'])
expect(() => validator.unsafeCast([3, 4])).toThrowError()
expect(() => validator.unsafeCast(null)).toThrowError()
testOutput<typeof validator._TYPE, string[]>()(null)
validator.parse(['test', 'text'])
expect(() => validator.parse([3, 4])).toThrowError()
expect(() => validator.parse(null)).toThrowError()
testOutput<typeof validator._output, string[]>()(null)
expect(value.spec).toMatchObject({
name: 'test',
spec: { patterns: [] },
@@ -778,10 +782,10 @@ describe('Nested nullable values', () => {
}),
}).build({} as any)
const validator = value.validator
validator.unsafeCast({ a: null })
validator.unsafeCast({ a: 'test' })
expect(() => validator.unsafeCast({ a: 4 })).toThrowError()
testOutput<typeof validator._TYPE, { a: string | null }>()(null)
validator.parse({ a: null })
validator.parse({ a: 'test' })
expect(() => validator.parse({ a: 4 })).toThrowError()
testOutput<typeof validator._output, { a: string | null }>()(null)
})
test('Testing number', async () => {
const value = await InputSpec.of({
@@ -801,10 +805,10 @@ describe('Nested nullable values', () => {
}),
}).build({} as any)
const validator = value.validator
validator.unsafeCast({ a: null })
validator.unsafeCast({ a: 5 })
expect(() => validator.unsafeCast({ a: '4' })).toThrowError()
testOutput<typeof validator._TYPE, { a: number | null }>()(null)
validator.parse({ a: null })
validator.parse({ a: 5 })
expect(() => validator.parse({ a: '4' })).toThrowError()
testOutput<typeof validator._output, { a: number | null }>()(null)
})
test('Testing color', async () => {
const value = await InputSpec.of({
@@ -818,10 +822,10 @@ describe('Nested nullable values', () => {
}),
}).build({} as any)
const validator = value.validator
validator.unsafeCast({ a: null })
validator.unsafeCast({ a: '5' })
expect(() => validator.unsafeCast({ a: 4 })).toThrowError()
testOutput<typeof validator._TYPE, { a: string | null }>()(null)
validator.parse({ a: null })
validator.parse({ a: '5' })
expect(() => validator.parse({ a: 4 })).toThrowError()
testOutput<typeof validator._output, { a: string | null }>()(null)
})
test('Testing select', async () => {
const value = await InputSpec.of({
@@ -848,9 +852,9 @@ describe('Nested nullable values', () => {
}).build({} as any)
const validator = value.validator
validator.unsafeCast({ a: 'a' })
expect(() => validator.unsafeCast({ a: '4' })).toThrowError()
testOutput<typeof validator._TYPE, { a: 'a' }>()(null)
validator.parse({ a: 'a' })
expect(() => validator.parse({ a: '4' })).toThrowError()
testOutput<typeof validator._output, { a: 'a' }>()(null)
})
test('Testing multiselect', async () => {
const value = await InputSpec.of({
@@ -869,10 +873,10 @@ describe('Nested nullable values', () => {
}),
}).build({} as any)
const validator = value.validator
validator.unsafeCast({ a: [] })
validator.unsafeCast({ a: ['a'] })
expect(() => validator.unsafeCast({ a: ['4'] })).toThrowError()
expect(() => validator.unsafeCast({ a: '4' })).toThrowError()
testOutput<typeof validator._TYPE, { a: 'a'[] }>()(null)
validator.parse({ a: [] })
validator.parse({ a: ['a'] })
expect(() => validator.parse({ a: ['4'] })).toThrowError()
expect(() => validator.parse({ a: '4' })).toThrowError()
testOutput<typeof validator._output, { a: 'a'[] }>()(null)
})
})

View File

@@ -1,428 +0,0 @@
import { oldSpecToBuilder } from '../../scripts/oldSpecToBuilder'
oldSpecToBuilder(
// Make the location
'./lib/test/output.ts',
// Put the inputSpec here
{
mediasources: {
type: 'list',
subtype: 'enum',
name: 'Media Sources',
description: 'List of Media Sources to use with Jellyfin',
range: '[1,*)',
default: ['nextcloud'],
spec: {
values: ['nextcloud', 'filebrowser'],
'value-names': {
nextcloud: 'NextCloud',
filebrowser: 'File Browser',
},
},
},
testListUnion: {
type: 'list',
subtype: 'union',
name: 'Lightning Nodes',
description: 'List of Lightning Network node instances to manage',
range: '[1,*)',
default: ['lnd'],
spec: {
type: 'string',
'display-as': '{{name}}',
'unique-by': 'name',
name: 'Node Implementation',
tag: {
id: 'type',
name: 'Type',
description:
'- LND: Lightning Network Daemon from Lightning Labs\n- CLN: Core Lightning from Blockstream\n',
'variant-names': {
lnd: 'Lightning Network Daemon (LND)',
'c-lightning': 'Core Lightning (CLN)',
},
},
default: 'lnd',
variants: {
lnd: {
name: {
type: 'string',
name: 'Node Name',
description: 'Name of this node in the list',
default: 'LND Wrapper',
nullable: false,
},
},
},
},
},
rpc: {
type: 'object',
name: 'RPC Settings',
description: 'RPC configuration options.',
spec: {
enable: {
type: 'boolean',
name: 'Enable',
description: 'Allow remote RPC requests.',
default: true,
},
username: {
type: 'string',
nullable: false,
name: 'Username',
description: 'The username for connecting to Bitcoin over RPC.',
default: 'bitcoin',
masked: true,
pattern: '^[a-zA-Z0-9_]+$',
'pattern-description':
'Must be alphanumeric (can contain underscore).',
},
password: {
type: 'string',
nullable: false,
name: 'RPC Password',
description: 'The password for connecting to Bitcoin over RPC.',
default: {
charset: 'a-z,2-7',
len: 20,
},
pattern: '^[^\\n"]*$',
'pattern-description':
'Must not contain newline or quote characters.',
copyable: true,
masked: true,
},
bio: {
type: 'string',
nullable: false,
name: 'Username',
description: 'The username for connecting to Bitcoin over RPC.',
default: 'bitcoin',
masked: true,
pattern: '^[a-zA-Z0-9_]+$',
'pattern-description':
'Must be alphanumeric (can contain underscore).',
textarea: true,
},
advanced: {
type: 'object',
name: 'Advanced',
description: 'Advanced RPC Settings',
spec: {
auth: {
name: 'Authorization',
description:
'Username and hashed password for JSON-RPC connections. RPC clients connect using the usual http basic authentication.',
type: 'list',
subtype: 'string',
default: [],
spec: {
pattern:
'^[a-zA-Z0-9_-]+:([0-9a-fA-F]{2})+\\$([0-9a-fA-F]{2})+$',
'pattern-description':
'Each item must be of the form "<USERNAME>:<SALT>$<HASH>".',
masked: false,
},
range: '[0,*)',
},
serialversion: {
name: 'Serialization Version',
description:
'Return raw transaction or block hex with Segwit or non-SegWit serialization.',
type: 'enum',
values: ['non-segwit', 'segwit'],
'value-names': {},
default: 'segwit',
},
servertimeout: {
name: 'Rpc Server Timeout',
description:
'Number of seconds after which an uncompleted RPC call will time out.',
type: 'number',
nullable: false,
range: '[5,300]',
integral: true,
units: 'seconds',
default: 30,
},
threads: {
name: 'Threads',
description:
'Set the number of threads for handling RPC calls. You may wish to increase this if you are making lots of calls via an integration.',
type: 'number',
nullable: false,
default: 16,
range: '[1,64]',
integral: true,
},
workqueue: {
name: 'Work Queue',
description:
'Set the depth of the work queue to service RPC calls. Determines how long the backlog of RPC requests can get before it just rejects new ones.',
type: 'number',
nullable: false,
default: 128,
range: '[8,256]',
integral: true,
units: 'requests',
},
},
},
},
},
'zmq-enabled': {
type: 'boolean',
name: 'ZeroMQ Enabled',
description: 'Enable the ZeroMQ interface',
default: true,
},
txindex: {
type: 'boolean',
name: 'Transaction Index',
description: 'Enable the Transaction Index (txindex)',
default: true,
},
wallet: {
type: 'object',
name: 'Wallet',
description: 'Wallet Settings',
spec: {
enable: {
name: 'Enable Wallet',
description: 'Load the wallet and enable wallet RPC calls.',
type: 'boolean',
default: true,
},
avoidpartialspends: {
name: 'Avoid Partial Spends',
description:
'Group outputs by address, selecting all or none, instead of selecting on a per-output basis. This improves privacy at the expense of higher transaction fees.',
type: 'boolean',
default: true,
},
discardfee: {
name: 'Discard Change Tolerance',
description:
'The fee rate (in BTC/kB) that indicates your tolerance for discarding change by adding it to the fee.',
type: 'number',
nullable: false,
default: 0.0001,
range: '[0,.01]',
integral: false,
units: 'BTC/kB',
},
},
},
advanced: {
type: 'object',
name: 'Advanced',
description: 'Advanced Settings',
spec: {
mempool: {
type: 'object',
name: 'Mempool',
description: 'Mempool Settings',
spec: {
mempoolfullrbf: {
name: 'Enable Full RBF',
description:
'Policy for your node to use for relaying and mining unconfirmed transactions. For details, see https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-24.0.md#notice-of-new-option-for-transaction-replacement-policies',
type: 'boolean',
default: false,
},
persistmempool: {
type: 'boolean',
name: 'Persist Mempool',
description: 'Save the mempool on shutdown and load on restart.',
default: true,
},
maxmempool: {
type: 'number',
nullable: false,
name: 'Max Mempool Size',
description:
'Keep the transaction memory pool below <n> megabytes.',
range: '[1,*)',
integral: true,
units: 'MiB',
default: 300,
},
mempoolexpiry: {
type: 'number',
nullable: false,
name: 'Mempool Expiration',
description:
'Do not keep transactions in the mempool longer than <n> hours.',
range: '[1,*)',
integral: true,
units: 'Hr',
default: 336,
},
},
},
peers: {
type: 'object',
name: 'Peers',
description: 'Peer Connection Settings',
spec: {
listen: {
type: 'boolean',
name: 'Make Public',
description:
'Allow other nodes to find your server on the network.',
default: true,
},
onlyconnect: {
type: 'boolean',
name: 'Disable Peer Discovery',
description: 'Only connect to specified peers.',
default: false,
},
onlyonion: {
type: 'boolean',
name: 'Disable Clearnet',
description: 'Only connect to peers over Tor.',
default: false,
},
addnode: {
name: 'Add Nodes',
description: 'Add addresses of nodes to connect to.',
type: 'list',
subtype: 'object',
range: '[0,*)',
default: [],
spec: {
'unique-by': null,
spec: {
hostname: {
type: 'string',
nullable: true,
name: 'Hostname',
description: 'Domain or IP address of bitcoin peer',
pattern:
'(^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$)|((^(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))$)|(^[a-z2-7]{16}\\.onion$)|(^([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\\.)+[a-z0-9][a-z0-9-]{0,61}[a-z0-9]$))',
'pattern-description':
"Must be either a domain name, or an IPv4 or IPv6 address. Do not include protocol scheme (eg 'http://') or port.",
masked: false,
},
port: {
type: 'number',
nullable: true,
name: 'Port',
description:
'Port that peer is listening on for inbound p2p connections',
range: '[0,65535]',
integral: true,
},
},
},
},
},
},
dbcache: {
type: 'number',
nullable: true,
name: 'Database Cache',
description:
"How much RAM to allocate for caching the TXO set. Higher values improve syncing performance, but increase your chance of using up all your system's memory or corrupting your database in the event of an ungraceful shutdown. Set this high but comfortably below your system's total RAM during IBD, then turn down to 450 (or leave blank) once the sync completes.",
warning:
'WARNING: Increasing this value results in a higher chance of ungraceful shutdowns, which can leave your node unusable if it happens during the initial block download. Use this setting with caution. Be sure to set this back to the default (450 or leave blank) once your node is synced. DO NOT press the STOP button if your dbcache is large. Instead, set this number back to the default, hit save, and wait for bitcoind to restart on its own.',
range: '(0,*)',
integral: true,
units: 'MiB',
},
pruning: {
type: 'union',
name: 'Pruning Settings',
description:
'Blockchain Pruning Options\nReduce the blockchain size on disk\n',
warning:
'If you set pruning to Manual and your disk is smaller than the total size of the blockchain, you MUST have something running that prunes these blocks or you may overfill your disk!\nDisabling pruning will convert your node into a full archival node. This requires a resync of the entire blockchain, a process that may take several days. Make sure you have enough free disk space or you may fill up your disk.\n',
tag: {
id: 'mode',
name: 'Pruning Mode',
description:
'- Disabled: Disable pruning\n- Automatic: Limit blockchain size on disk to a certain number of megabytes\n- Manual: Prune blockchain with the "pruneblockchain" RPC\n',
'variant-names': {
disabled: 'Disabled',
automatic: 'Automatic',
manual: 'Manual',
},
},
variants: {
disabled: {},
automatic: {
size: {
type: 'number',
nullable: false,
name: 'Max Chain Size',
description: 'Limit of blockchain size on disk.',
warning:
'Increasing this value will require re-syncing your node.',
default: 550,
range: '[550,1000000)',
integral: true,
units: 'MiB',
},
},
manual: {
size: {
type: 'number',
nullable: false,
name: 'Failsafe Chain Size',
description: 'Prune blockchain if size expands beyond this.',
default: 65536,
range: '[550,1000000)',
integral: true,
units: 'MiB',
},
},
},
default: 'disabled',
},
blockfilters: {
type: 'object',
name: 'Block Filters',
description: 'Settings for storing and serving compact block filters',
spec: {
blockfilterindex: {
type: 'boolean',
name: 'Compute Compact Block Filters (BIP158)',
description:
"Generate Compact Block Filters during initial sync (IBD) to enable 'getblockfilter' RPC. This is useful if dependent services need block filters to efficiently scan for addresses/transactions etc.",
default: true,
},
peerblockfilters: {
type: 'boolean',
name: 'Serve Compact Block Filters to Peers (BIP157)',
description:
"Serve Compact Block Filters as a peer service to other nodes on the network. This is useful if you wish to connect an SPV client to your node to make it efficient to scan transactions without having to download all block data. 'Compute Compact Block Filters (BIP158)' is required.",
default: false,
},
},
},
bloomfilters: {
type: 'object',
name: 'Bloom Filters (BIP37)',
description: 'Setting for serving Bloom Filters',
spec: {
peerbloomfilters: {
type: 'boolean',
name: 'Serve Bloom Filters to Peers',
description:
'Peers have the option of setting filters on each connection they make after the version handshake has completed. Bloom filters are for clients implementing SPV (Simplified Payment Verification) that want to check that block headers connect together correctly, without needing to verify the full blockchain. The client must trust that the transactions in the chain are in fact valid. It is highly recommended AGAINST using for anything except Bisq integration.',
warning:
'This is ONLY for use with Bisq integration, please use Block Filters for all other applications.',
default: false,
},
},
},
},
},
},
{
// convert this to `start-sdk/lib` for conversions
StartSdk: './output.sdk',
},
)

View File

@@ -9,12 +9,11 @@ export const sdk = StartSdk.of()
id: 'testOutput',
title: '',
license: '',
wrapperRepo: '',
packageRepo: '',
upstreamRepo: '',
supportSite: '',
marketingSite: '',
marketingUrl: '',
donationUrl: null,
docsUrl: '',
docsUrls: [],
description: {
short: '',
long: '',

View File

@@ -1,148 +0,0 @@
import { inputSpecSpec, InputSpecSpec } from './output'
import * as _I from '../index'
import { camelCase } from '../../scripts/oldSpecToBuilder'
import { deepMerge } from '../../../base/lib/util'
export type IfEquals<T, U, Y = unknown, N = never> =
(<G>() => G extends T ? 1 : 2) extends <G>() => G extends U ? 1 : 2 ? Y : N
export function testOutput<A, B>(): (c: IfEquals<A, B>) => null {
return () => null
}
/// Testing the types of the input spec
testOutput<InputSpecSpec['rpc']['enable'], boolean>()(null)
testOutput<InputSpecSpec['rpc']['username'], string>()(null)
testOutput<InputSpecSpec['rpc']['username'], string>()(null)
testOutput<InputSpecSpec['rpc']['advanced']['auth'], string[]>()(null)
testOutput<
InputSpecSpec['rpc']['advanced']['serialversion'],
'segwit' | 'non-segwit'
>()(null)
testOutput<InputSpecSpec['rpc']['advanced']['servertimeout'], number>()(null)
testOutput<
InputSpecSpec['advanced']['peers']['addnode'][0]['hostname'],
string | null
>()(null)
testOutput<
InputSpecSpec['testListUnion'][0]['union']['value']['name'],
string
>()(null)
testOutput<InputSpecSpec['testListUnion'][0]['union']['selection'], 'lnd'>()(
null,
)
testOutput<InputSpecSpec['mediasources'], Array<'filebrowser' | 'nextcloud'>>()(
null,
)
// @ts-expect-error Because enable should be a boolean
testOutput<InputSpecSpec['rpc']['enable'], string>()(null)
// prettier-ignore
// @ts-expect-error Expect that the string is the one above
testOutput<InputSpecSpec["testListUnion"][0]['selection']['selection'], "selection">()(null);
/// Here we test the output of the matchInputSpecSpec function
describe('Inputs', () => {
const validInput: InputSpecSpec = {
mediasources: ['filebrowser'],
testListUnion: [
{
union: { selection: 'lnd', value: { name: 'string' } },
},
],
rpc: {
enable: true,
bio: 'This is a bio',
username: 'test',
password: 'test',
advanced: {
auth: ['test'],
serialversion: 'segwit',
servertimeout: 6,
threads: 3,
workqueue: 9,
},
},
'zmq-enabled': false,
txindex: false,
wallet: { enable: false, avoidpartialspends: false, discardfee: 0.0001 },
advanced: {
mempool: {
maxmempool: 1,
persistmempool: true,
mempoolexpiry: 23,
mempoolfullrbf: true,
},
peers: {
listen: true,
onlyconnect: true,
onlyonion: true,
addnode: [
{
hostname: 'test',
port: 1,
},
],
},
dbcache: 5,
pruning: {
selection: 'disabled',
value: { disabled: {} },
},
blockfilters: {
blockfilterindex: false,
peerblockfilters: false,
},
bloomfilters: { peerbloomfilters: false },
},
}
test('test valid input', async () => {
const { validator } = await inputSpecSpec.build({} as any)
const output = validator.unsafeCast(validInput)
expect(output).toEqual(validInput)
})
test('test no longer care about the conversion of min/max and validating', async () => {
const { validator } = await inputSpecSpec.build({} as any)
validator.unsafeCast(
deepMerge({}, validInput, { rpc: { advanced: { threads: 0 } } }),
)
})
test('test errors should throw for number in string', async () => {
const { validator } = await inputSpecSpec.build({} as any)
expect(() =>
validator.unsafeCast(deepMerge({}, validInput, { rpc: { enable: 2 } })),
).toThrowError()
})
test('Test that we set serialversion to something not segwit or non-segwit', async () => {
const { validator } = await inputSpecSpec.build({} as any)
expect(() =>
validator.unsafeCast(
deepMerge({}, validInput, {
rpc: { advanced: { serialversion: 'testing' } },
}),
),
).toThrowError()
})
})
describe('camelCase', () => {
test("'EquipmentClass name'", () => {
expect(camelCase('EquipmentClass name')).toEqual('equipmentClassName')
})
test("'Equipment className'", () => {
expect(camelCase('Equipment className')).toEqual('equipmentClassName')
})
test("'equipment class name'", () => {
expect(camelCase('equipment class name')).toEqual('equipmentClassName')
})
test("'Equipment Class Name'", () => {
expect(camelCase('Equipment Class Name')).toEqual('equipmentClassName')
})
test("'hyphen-name-format'", () => {
expect(camelCase('hyphen-name-format')).toEqual('hyphenNameFormat')
})
test("'underscore_name_format'", () => {
expect(camelCase('underscore_name_format')).toEqual('underscoreNameFormat')
})
})

View File

@@ -1,5 +1,6 @@
import { Effects } from '../../../base/lib/Effects'
import { Manifest, PackageId } from '../../../base/lib/osBindings'
import { AbortedError } from '../../../base/lib/util/AbortedError'
import { DropGenerator, DropPromise } from '../../../base/lib/util/Drop'
import { deepEqual } from '../../../base/lib/util/deepEqual'
@@ -64,7 +65,7 @@ export class GetServiceManifest<Mapped = Manifest> {
}
await waitForNext
}
return new Promise<never>((_, rej) => rej(new Error('aborted')))
return new Promise<never>((_, rej) => rej(new AbortedError()))
}
/**

View File

@@ -1,5 +1,6 @@
import { T } from '..'
import { Effects } from '../../../base/lib/Effects'
import { AbortedError } from '../../../base/lib/util/AbortedError'
import { DropGenerator, DropPromise } from '../../../base/lib/util/Drop'
export class GetSslCertificate {
@@ -50,7 +51,7 @@ export class GetSslCertificate {
})
await waitForNext
}
return new Promise<never>((_, rej) => rej(new Error('aborted')))
return new Promise<never>((_, rej) => rej(new AbortedError()))
}
/**

View File

@@ -69,6 +69,14 @@ async function bind(
await execFile('mount', [...args, from, to])
}
/**
* Interface representing an isolated container environment for running service processes.
*
* Provides methods for executing commands, spawning processes, mounting filesystems,
* and writing files within the container's rootfs. Comes in two flavors:
* {@link SubContainerOwned} (owns the underlying filesystem) and
* {@link SubContainerRc} (reference-counted handle to a shared container).
*/
export interface SubContainer<
Manifest extends T.SDKManifest,
Effects extends T.Effects = T.Effects,
@@ -84,6 +92,11 @@ export interface SubContainer<
*/
subpath(path: string): string
/**
* Apply filesystem mounts (volumes, assets, dependencies, backups) to this subcontainer.
* @param mounts - The Mounts configuration to apply
* @returns This subcontainer instance for chaining
*/
mount(
mounts: Effects extends BackupEffects
? Mounts<
@@ -96,6 +109,7 @@ export interface SubContainer<
: Mounts<Manifest, never>,
): Promise<this>
/** Destroy this subcontainer and clean up its filesystem */
destroy: () => Promise<null>
/**
@@ -136,11 +150,22 @@ export interface SubContainer<
stderr: string | Buffer
}>
/**
* Launch a command as the init (PID 1) process of the subcontainer.
* Replaces the current leader process.
* @param command - The command and arguments to execute
* @param options - Optional environment, working directory, and user overrides
*/
launch(
command: string[],
options?: CommandOptions,
): Promise<cp.ChildProcessWithoutNullStreams>
/**
* Spawn a command inside the subcontainer as a non-init process.
* @param command - The command and arguments to execute
* @param options - Optional environment, working directory, user, and stdio overrides
*/
spawn(
command: string[],
options?: CommandOptions & StdioOptions,
@@ -162,8 +187,13 @@ export interface SubContainer<
options?: Parameters<typeof fs.writeFile>[2],
): Promise<void>
/**
* Create a reference-counted handle to this subcontainer.
* The underlying container is only destroyed when all handles are released.
*/
rc(): SubContainerRc<Manifest, Effects>
/** Returns true if this is an owned subcontainer (not a reference-counted handle) */
isOwned(): this is SubContainerOwned<Manifest, Effects>
}
@@ -679,6 +709,12 @@ export class SubContainerOwned<
}
}
/**
* A reference-counted handle to a {@link SubContainerOwned}.
*
* Multiple `SubContainerRc` instances can share one underlying subcontainer.
* The subcontainer is destroyed only when the last reference is released via `destroy()`.
*/
export class SubContainerRc<
Manifest extends T.SDKManifest,
Effects extends T.Effects = T.Effects,
@@ -901,14 +937,17 @@ export type StdioOptions = {
stdio?: cp.IOType
}
/** UID/GID mapping for mount id-remapping (see kernel idmappings docs) */
export type IdMap = { fromId: number; toId: number; range: number }
/** Union of all mount option types supported by the subcontainer runtime */
export type MountOptions =
| MountOptionsVolume
| MountOptionsAssets
| MountOptionsPointer
| MountOptionsBackup
/** Mount options for binding a service volume into a subcontainer */
export type MountOptionsVolume = {
type: 'volume'
volumeId: string
@@ -918,6 +957,7 @@ export type MountOptionsVolume = {
idmap: IdMap[]
}
/** Mount options for binding packaged static assets into a subcontainer */
export type MountOptionsAssets = {
type: 'assets'
subpath: string | null
@@ -925,6 +965,7 @@ export type MountOptionsAssets = {
idmap: { fromId: number; toId: number; range: number }[]
}
/** Mount options for binding a dependency package's volume into a subcontainer */
export type MountOptionsPointer = {
type: 'pointer'
packageId: string
@@ -934,6 +975,7 @@ export type MountOptionsPointer = {
idmap: { fromId: number; toId: number; range: number }[]
}
/** Mount options for binding the backup directory into a subcontainer */
export type MountOptionsBackup = {
type: 'backup'
subpath: string | null
@@ -944,6 +986,10 @@ function wait(time: number) {
return new Promise((resolve) => setTimeout(resolve, time))
}
/**
* Error thrown when a subcontainer command exits with a non-zero code or signal.
* Contains the full result including stdout, stderr, exit code, and exit signal.
*/
export class ExitError extends Error {
constructor(
readonly command: string,

View File

@@ -1,10 +1,10 @@
import * as matches from 'ts-matches'
import { z } from 'zod'
import * as YAML from 'yaml'
import * as TOML from '@iarna/toml'
import * as INI from 'ini'
import * as T from '../../../base/lib/types'
import * as fs from 'node:fs/promises'
import { asError, deepEqual } from '../../../base/lib/util'
import { AbortedError, asError, deepEqual } from '../../../base/lib/util'
import { DropGenerator, DropPromise } from '../../../base/lib/util/Drop'
import { PathBase } from './Volume'
@@ -84,9 +84,22 @@ function filterUndefined<A>(a: A): A {
return a
}
export type Transformers<Raw = unknown, Transformed = unknown> = {
/**
* Bidirectional transformers for converting between the raw file format and
* the application-level data type. Used with FileHelper factory methods.
*
* @typeParam Raw - The native type the file format parses to (e.g. `Record<string, unknown>` for JSON)
* @typeParam Transformed - The application-level type after transformation
*/
export type Transformers<
Raw = unknown,
Transformed = unknown,
Validated extends Transformed = Transformed,
> = {
/** Transform raw parsed data into the application type */
onRead: (value: Raw) => Transformed
onWrite: (value: Transformed) => Raw
/** Transform application data back into the raw format for writing */
onWrite: (value: Validated) => Raw
}
type ToPath = string | { base: PathBase; subpath: string }
@@ -97,7 +110,7 @@ function toPath(path: ToPath): string {
return path.base.subpath(path.subpath)
}
type Validator<T, U> = matches.Validator<T, U> | matches.Validator<unknown, U>
type Validator<_T, U> = z.ZodType<U>
type ReadType<A> = {
once: () => Promise<A | null>
@@ -276,7 +289,7 @@ export class FileHelper<A> {
await onCreated(this.path).catch((e) => console.error(asError(e)))
}
}
return new Promise<never>((_, rej) => rej(new Error('aborted')))
return new Promise<never>((_, rej) => rej(new AbortedError()))
}
private readOnChange<B>(
@@ -343,6 +356,19 @@ export class FileHelper<A> {
)
}
/**
* Create a reactive reader for this file.
*
* Returns an object with multiple read strategies:
* - `once()` - Read the file once and return the parsed value
* - `const(effects)` - Read once but re-read when the file changes (for use with constRetry)
* - `watch(effects)` - Async generator yielding new values on each file change
* - `onChange(effects, callback)` - Fire a callback on each file change
* - `waitFor(effects, predicate)` - Block until the file value satisfies a predicate
*
* @param map - Optional transform function applied after validation
* @param eq - Optional equality function to deduplicate watch emissions
*/
read(): ReadType<A>
read<B>(
map: (value: A) => B,
@@ -461,7 +487,7 @@ export class FileHelper<A> {
toFile: (dataIn: Raw) => string,
fromFile: (rawData: string) => Raw,
validate: (data: Transformed) => A,
transformers: Transformers<Raw, Transformed> | undefined,
transformers: Transformers<Raw, Transformed, A> | undefined,
) {
return FileHelper.raw<A>(
path,
@@ -471,7 +497,12 @@ export class FileHelper<A> {
}
return toFile(inData as any as Raw)
},
fromFile,
(fileData) => {
if (transformers) {
return transformers.onRead(fromFile(fileData))
}
return fromFile(fileData)
},
validate as (a: unknown) => A,
)
}
@@ -487,19 +518,19 @@ export class FileHelper<A> {
static string<A extends Transformed, Transformed = string>(
path: ToPath,
shape: Validator<Transformed, A>,
transformers: Transformers<string, Transformed>,
transformers: Transformers<string, Transformed, A>,
): FileHelper<A>
static string<A extends Transformed, Transformed = string>(
path: ToPath,
shape?: Validator<Transformed, A>,
transformers?: Transformers<string, Transformed>,
transformers?: Transformers<string, Transformed, A>,
) {
return FileHelper.rawTransformed<A, string, Transformed>(
path,
(inData) => inData,
(inString) => inString,
(data) =>
(shape || (matches.string as Validator<Transformed, A>)).unsafeCast(
(shape || (z.string() as unknown as Validator<Transformed, A>)).parse(
data,
),
transformers,
@@ -509,16 +540,22 @@ export class FileHelper<A> {
/**
* Create a File Helper for a .json file.
*/
static json<A>(
static json<A>(path: ToPath, shape: Validator<unknown, A>): FileHelper<A>
static json<A extends Transformed, Transformed = unknown>(
path: ToPath,
shape: Validator<unknown, A>,
transformers?: Transformers,
transformers: Transformers<unknown, Transformed, A>,
): FileHelper<A>
static json<A extends Transformed, Transformed = unknown>(
path: ToPath,
shape: Validator<unknown, A>,
transformers?: Transformers<unknown, Transformed, A>,
) {
return FileHelper.rawTransformed(
path,
(inData) => JSON.stringify(inData, null, 2),
(inString) => JSON.parse(inString),
(data) => shape.unsafeCast(data),
(data) => shape.parse(data),
transformers,
)
}
@@ -533,18 +570,18 @@ export class FileHelper<A> {
static yaml<A extends Transformed, Transformed = Record<string, unknown>>(
path: ToPath,
shape: Validator<Transformed, A>,
transformers: Transformers<Record<string, unknown>, Transformed>,
transformers: Transformers<Record<string, unknown>, Transformed, A>,
): FileHelper<A>
static yaml<A extends Transformed, Transformed = Record<string, unknown>>(
path: ToPath,
shape: Validator<Transformed, A>,
transformers?: Transformers<Record<string, unknown>, Transformed>,
transformers?: Transformers<Record<string, unknown>, Transformed, A>,
) {
return FileHelper.rawTransformed<A, Record<string, unknown>, Transformed>(
path,
(inData) => YAML.stringify(inData, null, 2),
(inString) => YAML.parse(inString),
(data) => shape.unsafeCast(data),
(data) => shape.parse(data),
transformers,
)
}
@@ -559,22 +596,27 @@ export class FileHelper<A> {
static toml<A extends Transformed, Transformed = Record<string, unknown>>(
path: ToPath,
shape: Validator<Transformed, A>,
transformers: Transformers<Record<string, unknown>, Transformed>,
transformers: Transformers<Record<string, unknown>, Transformed, A>,
): FileHelper<A>
static toml<A extends Transformed, Transformed = Record<string, unknown>>(
path: ToPath,
shape: Validator<Transformed, A>,
transformers?: Transformers<Record<string, unknown>, Transformed>,
transformers?: Transformers<Record<string, unknown>, Transformed, A>,
) {
return FileHelper.rawTransformed<A, Record<string, unknown>, Transformed>(
path,
(inData) => TOML.stringify(inData as TOML.JsonMap),
(inString) => TOML.parse(inString),
(data) => shape.unsafeCast(data),
(data) => shape.parse(data),
transformers,
)
}
/**
* Create a File Helper for a .ini file.
*
* Supports optional encode/decode options and custom transformers.
*/
static ini<A extends Record<string, unknown>>(
path: ToPath,
shape: Validator<Record<string, unknown>, A>,
@@ -584,23 +626,28 @@ export class FileHelper<A> {
path: ToPath,
shape: Validator<Transformed, A>,
options: INI.EncodeOptions & INI.DecodeOptions,
transformers: Transformers<Record<string, unknown>, Transformed>,
transformers: Transformers<Record<string, unknown>, Transformed, A>,
): FileHelper<A>
static ini<A extends Transformed, Transformed = Record<string, unknown>>(
path: ToPath,
shape: Validator<Transformed, A>,
options?: INI.EncodeOptions & INI.DecodeOptions,
transformers?: Transformers<Record<string, unknown>, Transformed>,
transformers?: Transformers<Record<string, unknown>, Transformed, A>,
): FileHelper<A> {
return FileHelper.rawTransformed<A, Record<string, unknown>, Transformed>(
path,
(inData) => INI.stringify(filterUndefined(inData), options),
(inString) => INI.parse(inString, options),
(data) => shape.unsafeCast(data),
(data) => shape.parse(data),
transformers,
)
}
/**
* Create a File Helper for a .env file (KEY=VALUE format, one per line).
*
* Lines starting with `#` are treated as comments and ignored on read.
*/
static env<A extends Record<string, string>>(
path: ToPath,
shape: Validator<Record<string, string>, A>,
@@ -608,12 +655,12 @@ export class FileHelper<A> {
static env<A extends Transformed, Transformed = Record<string, string>>(
path: ToPath,
shape: Validator<Transformed, A>,
transformers: Transformers<Record<string, string>, Transformed>,
transformers: Transformers<Record<string, string>, Transformed, A>,
): FileHelper<A>
static env<A extends Transformed, Transformed = Record<string, string>>(
path: ToPath,
shape: Validator<Transformed, A>,
transformers?: Transformers<Record<string, string>, Transformed>,
transformers?: Transformers<Record<string, string>, Transformed, A>,
) {
return FileHelper.rawTransformed<A, Record<string, string>, Transformed>(
path,
@@ -632,7 +679,7 @@ export class FileHelper<A> {
return [line.slice(0, pos), line.slice(pos + 1)]
}),
),
(data) => shape.unsafeCast(data),
(data) => shape.parse(data),
transformers,
)
}

View File

@@ -12,6 +12,11 @@ import {
import { Graph, Vertex, once } from '../util'
import { IMPOSSIBLE, VersionInfo } from './VersionInfo'
/**
* Read the current data version from the effects system.
* @param effects - The effects context
* @returns The parsed ExtendedVersion or VersionRange, or null if no version is set
*/
export async function getDataVersion(effects: T.Effects) {
const versionStr = await effects.getDataVersion()
if (!versionStr) return null
@@ -22,6 +27,11 @@ export async function getDataVersion(effects: T.Effects) {
}
}
/**
* Persist a data version to the effects system.
* @param effects - The effects context
* @param version - The version to set, or null to clear it
*/
export async function setDataVersion(
effects: T.Effects,
version: ExtendedVersion | VersionRange | null,
@@ -37,6 +47,14 @@ function isRange(v: ExtendedVersion | VersionRange): v is VersionRange {
return 'satisfiedBy' in v
}
/**
* Check whether two version specifiers overlap (i.e. share at least one common version).
* Works with any combination of ExtendedVersion and VersionRange.
*
* @param a - First version or range
* @param b - Second version or range
* @returns True if the two specifiers overlap
*/
export function overlaps(
a: ExtendedVersion | VersionRange,
b: ExtendedVersion | VersionRange,
@@ -49,6 +67,16 @@ export function overlaps(
)
}
/**
* A directed graph of service versions and their migration paths.
*
* Builds a graph from {@link VersionInfo} definitions, then uses shortest-path
* search to find and execute migration sequences between any two versions.
* Implements both {@link InitScript} (for install/update migrations) and
* {@link UninitScript} (for uninstall/downgrade migrations).
*
* @typeParam CurrentVersion - The string literal type of the current service version
*/
export class VersionGraph<CurrentVersion extends string>
implements InitScript, UninitScript
{
@@ -58,14 +86,13 @@ export class VersionGraph<CurrentVersion extends string>
ExtendedVersion | VersionRange,
((opts: { effects: T.Effects }) => Promise<void>) | undefined
>
/** Dump the version graph as a human-readable string for debugging */
dump(): string {
return this.graph().dump((metadata) => metadata?.toString())
}
private constructor(
readonly current: VersionInfo<CurrentVersion>,
versions: Array<VersionInfo<any>>,
private readonly preInstall?: InitScriptOrFn<'install'>,
private readonly uninstall?: UninitScript | UninitFn,
) {
this.graph = once(() => {
const graph = new Graph<
@@ -167,25 +194,21 @@ export class VersionGraph<CurrentVersion extends string>
static of<
CurrentVersion extends string,
OtherVersions extends Array<VersionInfo<any>>,
>(options: {
current: VersionInfo<CurrentVersion>
other: OtherVersions
/**
* A script to run only on fresh install
*/
preInstall?: InitScriptOrFn<'install'>
/**
* A script to run only on uninstall
*/
uninstall?: UninitScriptOrFn
}) {
return new VersionGraph(
options.current,
options.other,
options.preInstall,
options.uninstall,
)
>(options: { current: VersionInfo<CurrentVersion>; other: OtherVersions }) {
return new VersionGraph(options.current, options.other)
}
/**
* Execute the shortest migration path between two versions.
*
* Finds the shortest path in the version graph from `from` to `to`,
* executes each migration step in order, and updates the data version after each step.
*
* @param options.effects - The effects context
* @param options.from - The source version or range
* @param options.to - The target version or range
* @returns The final data version after migration
* @throws If no migration path exists between the two versions
*/
async migrate({
effects,
from,
@@ -235,6 +258,10 @@ export class VersionGraph<CurrentVersion extends string>
`cannot migrate from ${from.toString()} to ${to.toString()}`,
)
}
/**
* Compute the version range from which the current version can be reached via migration.
* Uses reverse breadth-first search from the current version vertex.
*/
canMigrateFrom = once(() =>
Array.from(
this.graph().reverseBreadthFirstSearch((v) =>
@@ -252,6 +279,10 @@ export class VersionGraph<CurrentVersion extends string>
)
.normalize(),
)
/**
* Compute the version range that the current version can migrate to.
* Uses forward breadth-first search from the current version vertex.
*/
canMigrateTo = once(() =>
Array.from(
this.graph().breadthFirstSearch((v) =>
@@ -270,7 +301,12 @@ export class VersionGraph<CurrentVersion extends string>
.normalize(),
)
async init(effects: T.Effects, kind: InitKind): Promise<void> {
/**
* InitScript implementation: migrate from the stored data version to the current version.
* If no data version exists (fresh install), sets it to the current version.
* @param effects - The effects context
*/
async init(effects: T.Effects): Promise<void> {
const from = await getDataVersion(effects)
if (from) {
await this.migrate({
@@ -279,14 +315,17 @@ export class VersionGraph<CurrentVersion extends string>
to: this.currentVersion(),
})
} else {
kind = 'install' // implied by !dataVersion
if (this.preInstall)
if ('init' in this.preInstall) await this.preInstall.init(effects, kind)
else await this.preInstall(effects, kind)
await effects.setDataVersion({ version: this.current.options.version })
}
}
/**
* UninitScript implementation: migrate from the current data version to the target version.
* Used during uninstall or downgrade to prepare data for the target version.
*
* @param effects - The effects context
* @param target - The target version to migrate to, or null to clear the data version
*/
async uninit(
effects: T.Effects,
target: VersionRange | ExtendedVersion | null,
@@ -300,11 +339,6 @@ export class VersionGraph<CurrentVersion extends string>
to: target,
})
}
} else {
if (this.uninstall)
if ('uninit' in this.uninstall)
await this.uninstall.uninit(effects, target)
else await this.uninstall(effects, target)
}
await setDataVersion(effects, target)
}

View File

@@ -1,8 +1,17 @@
import { ValidateExVer } from '../../../base/lib/exver'
import * as T from '../../../base/lib/types'
/**
* Sentinel value indicating that a migration in a given direction is not possible.
* Use this for `migrations.up` or `migrations.down` to prevent migration.
*/
export const IMPOSSIBLE: unique symbol = Symbol('IMPOSSIBLE')
/**
* Configuration options for a single service version definition.
*
* @typeParam Version - The string literal exver version number
*/
export type VersionOptions<Version extends string> = {
/** The exver-compliant version number */
version: Version & ValidateExVer<Version>
@@ -33,6 +42,14 @@ export type VersionOptions<Version extends string> = {
}
}
/**
* Represents a single version of the service, including its release notes,
* migration scripts, and backwards-compatibility declarations.
*
* By convention, each version gets its own file (e.g. `versions/v1_0_0.ts`).
*
* @typeParam Version - The string literal exver version number
*/
export class VersionInfo<Version extends string> {
private _version: null | Version = null
private constructor(

View File

@@ -1,12 +1,12 @@
{
"name": "@start9labs/start-sdk",
"version": "0.4.0-beta.48",
"version": "0.4.0-beta.55",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@start9labs/start-sdk",
"version": "0.4.0-beta.48",
"version": "0.4.0-beta.55",
"license": "MIT",
"dependencies": {
"@iarna/toml": "^3.0.0",
@@ -17,8 +17,9 @@
"ini": "^5.0.0",
"isomorphic-fetch": "^3.0.0",
"mime": "^4.0.7",
"ts-matches": "^6.3.2",
"yaml": "^2.7.1"
"yaml": "^2.7.1",
"zod": "^4.3.6",
"zod-deep-partial": "^1.2.0"
},
"devDependencies": {
"@types/jest": "^29.4.0",
@@ -4815,12 +4816,6 @@
"node": ">=10"
}
},
"node_modules/ts-matches": {
"version": "6.3.2",
"resolved": "https://registry.npmjs.org/ts-matches/-/ts-matches-6.3.2.tgz",
"integrity": "sha512-UhSgJymF8cLd4y0vV29qlKVCkQpUtekAaujXbQVc729FezS8HwqzepqvtjzQ3HboatIqN/Idor85O2RMwT7lIQ==",
"license": "MIT"
},
"node_modules/ts-morph": {
"version": "18.0.0",
"resolved": "https://registry.npmjs.org/ts-morph/-/ts-morph-18.0.0.tgz",
@@ -5232,6 +5227,25 @@
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/zod": {
"version": "4.3.6",
"resolved": "https://registry.npmjs.org/zod/-/zod-4.3.6.tgz",
"integrity": "sha512-rftlrkhHZOcjDwkGlnUtZZkvaPHCsDATp4pGpuOOMDaTdDDXF91wuVDJoWoPsKX/3YPQ5fHuF3STjcYyKr+Qhg==",
"license": "MIT",
"peer": true,
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
},
"node_modules/zod-deep-partial": {
"version": "1.4.4",
"resolved": "https://registry.npmjs.org/zod-deep-partial/-/zod-deep-partial-1.4.4.tgz",
"integrity": "sha512-aWkPl7hVStgE01WzbbSxCgX4O+sSpgt8JOjvFUtMTF75VgL6MhWQbiZi+AWGN85SfSTtI9gsOtL1vInoqfDVaA==",
"license": "MIT",
"peerDependencies": {
"zod": "^4.1.13"
}
}
}
}

View File

@@ -1,6 +1,6 @@
{
"name": "@start9labs/start-sdk",
"version": "0.4.0-beta.48",
"version": "0.4.0-beta.55",
"description": "Software development kit to facilitate packaging services for StartOS",
"main": "./package/lib/index.js",
"types": "./package/lib/index.d.ts",
@@ -31,16 +31,17 @@
},
"homepage": "https://github.com/Start9Labs/start-os#readme",
"dependencies": {
"isomorphic-fetch": "^3.0.0",
"mime": "^4.0.7",
"ts-matches": "^6.3.2",
"yaml": "^2.7.1",
"deep-equality-data-structures": "^2.0.0",
"ini": "^5.0.0",
"@types/ini": "^4.1.1",
"@iarna/toml": "^3.0.0",
"@noble/curves": "^1.8.2",
"@noble/hashes": "^1.7.2"
"@noble/hashes": "^1.7.2",
"@types/ini": "^4.1.1",
"deep-equality-data-structures": "^2.0.0",
"ini": "^5.0.0",
"isomorphic-fetch": "^3.0.0",
"mime": "^4.0.7",
"yaml": "^2.7.1",
"zod": "^4.3.6",
"zod-deep-partial": "^1.2.0"
},
"prettier": {
"trailingComma": "all",

View File

@@ -1,384 +0,0 @@
import * as fs from "fs"
// https://stackoverflow.com/questions/2970525/converting-any-string-into-camel-case
export function camelCase(value: string) {
return value
.replace(/([\(\)\[\]])/g, "")
.replace(/^([A-Z])|[\s-_](\w)/g, function (match, p1, p2, offset) {
if (p2) return p2.toUpperCase()
return p1.toLowerCase()
})
}
export async function oldSpecToBuilder(
file: string,
inputData: Promise<any> | any,
options?: Parameters<typeof makeFileContentFromOld>[1],
) {
await fs.writeFile(
file,
await makeFileContentFromOld(inputData, options),
(err) => console.error(err),
)
}
function isString(x: unknown): x is string {
return typeof x === "string"
}
export default async function makeFileContentFromOld(
inputData: Promise<any> | any,
{ StartSdk = "start-sdk", nested = true } = {},
) {
const outputLines: string[] = []
outputLines.push(`
import { sdk } from "${StartSdk}"
const {InputSpec, List, Value, Variants} = sdk
`)
const data = await inputData
const namedConsts = new Set(["InputSpec", "Value", "List"])
const inputSpecName = newConst("inputSpecSpec", convertInputSpec(data))
outputLines.push(`export type InputSpecSpec = typeof ${inputSpecName}._TYPE;`)
return outputLines.join("\n")
function newConst(key: string, data: string, type?: string) {
const variableName = getNextConstName(camelCase(key))
outputLines.push(
`export const ${variableName}${!type ? "" : `: ${type}`} = ${data};`,
)
return variableName
}
function maybeNewConst(key: string, data: string) {
if (nested) return data
return newConst(key, data)
}
function convertInputSpecInner(data: any) {
let answer = "{"
for (const [key, value] of Object.entries(data)) {
const variableName = maybeNewConst(key, convertValueSpec(value))
answer += `${JSON.stringify(key)}: ${variableName},`
}
return `${answer}}`
}
function convertInputSpec(data: any) {
return `InputSpec.of(${convertInputSpecInner(data)})`
}
function convertValueSpec(value: any): string {
switch (value.type) {
case "string": {
if (value.textarea) {
return `${rangeToTodoComment(
value?.range,
)}Value.textarea(${JSON.stringify(
{
name: value.name || null,
description: value.description || null,
warning: value.warning || null,
required: !(value.nullable || false),
default: value.default,
placeholder: value.placeholder || null,
minLength: null,
maxLength: null,
minRows: 3,
maxRows: 6,
},
null,
2,
)})`
}
return `${rangeToTodoComment(value?.range)}Value.text(${JSON.stringify(
{
name: value.name || null,
default: value.default || null,
required: !value.nullable,
description: value.description || null,
warning: value.warning || null,
masked: value.masked || false,
placeholder: value.placeholder || null,
inputmode: "text",
patterns: value.pattern
? [
{
regex: value.pattern,
description: value["pattern-description"],
},
]
: [],
minLength: null,
maxLength: null,
},
null,
2,
)})`
}
case "number": {
return `${rangeToTodoComment(
value?.range,
)}Value.number(${JSON.stringify(
{
name: value.name || null,
description: value.description || null,
warning: value.warning || null,
default: value.default || null,
required: !value.nullable,
min: null,
max: null,
step: null,
integer: value.integral || false,
units: value.units || null,
placeholder: value.placeholder || null,
},
null,
2,
)})`
}
case "boolean": {
return `Value.toggle(${JSON.stringify(
{
name: value.name || null,
default: value.default || false,
description: value.description || null,
warning: value.warning || null,
},
null,
2,
)})`
}
case "enum": {
const allValueNames = new Set([
...(value?.["values"] || []),
...Object.keys(value?.["value-names"] || {}),
])
const values = Object.fromEntries(
Array.from(allValueNames)
.filter(isString)
.map((key) => [key, value?.spec?.["value-names"]?.[key] || key]),
)
return `Value.select(${JSON.stringify(
{
name: value.name || null,
description: value.description || null,
warning: value.warning || null,
default: value.default,
values,
},
null,
2,
)} as const)`
}
case "object": {
const specName = maybeNewConst(
value.name + "_spec",
convertInputSpec(value.spec),
)
return `Value.object({
name: ${JSON.stringify(value.name || null)},
description: ${JSON.stringify(value.description || null)},
}, ${specName})`
}
case "union": {
const variants = maybeNewConst(
value.name + "_variants",
convertVariants(value.variants, value.tag["variant-names"] || {}),
)
return `Value.union({
name: ${JSON.stringify(value.name || null)},
description: ${JSON.stringify(value.tag.description || null)},
warning: ${JSON.stringify(value.tag.warning || null)},
default: ${JSON.stringify(value.default)},
variants: ${variants},
})`
}
case "list": {
if (value.subtype === "enum") {
const allValueNames = new Set([
...(value?.spec?.["values"] || []),
...Object.keys(value?.spec?.["value-names"] || {}),
])
const values = Object.fromEntries(
Array.from(allValueNames)
.filter(isString)
.map((key: string) => [
key,
value?.spec?.["value-names"]?.[key] ?? key,
]),
)
return `Value.multiselect(${JSON.stringify(
{
name: value.name || null,
minLength: null,
maxLength: null,
default: value.default ?? null,
description: value.description || null,
warning: value.warning || null,
values,
},
null,
2,
)})`
}
const list = maybeNewConst(value.name + "_list", convertList(value))
return `Value.list(${list})`
}
case "pointer": {
return `/* TODO deal with point removed point "${value.name}" */null as any`
}
}
throw Error(`Unknown type "${value.type}"`)
}
function convertList(value: any) {
switch (value.subtype) {
case "string": {
return `${rangeToTodoComment(value?.range)}List.text(${JSON.stringify(
{
name: value.name || null,
minLength: null,
maxLength: null,
default: value.default || null,
description: value.description || null,
warning: value.warning || null,
},
null,
2,
)}, ${JSON.stringify({
masked: value?.spec?.masked || false,
placeholder: value?.spec?.placeholder || null,
patterns: value?.spec?.pattern
? [
{
regex: value.spec.pattern,
description: value?.spec?.["pattern-description"],
},
]
: [],
minLength: null,
maxLength: null,
})})`
}
// case "number": {
// return `${rangeToTodoComment(value?.range)}List.number(${JSON.stringify(
// {
// name: value.name || null,
// minLength: null,
// maxLength: null,
// default: value.default || null,
// description: value.description || null,
// warning: value.warning || null,
// },
// null,
// 2,
// )}, ${JSON.stringify({
// integer: value?.spec?.integral || false,
// min: null,
// max: null,
// units: value?.spec?.units || null,
// placeholder: value?.spec?.placeholder || null,
// })})`
// }
case "enum": {
return "/* error!! list.enum */"
}
case "object": {
const specName = maybeNewConst(
value.name + "_spec",
convertInputSpec(value.spec.spec),
)
return `${rangeToTodoComment(value?.range)}List.obj({
name: ${JSON.stringify(value.name || null)},
minLength: ${JSON.stringify(null)},
maxLength: ${JSON.stringify(null)},
default: ${JSON.stringify(value.default || null)},
description: ${JSON.stringify(value.description || null)},
}, {
spec: ${specName},
displayAs: ${JSON.stringify(value?.spec?.["display-as"] || null)},
uniqueBy: ${JSON.stringify(value?.spec?.["unique-by"] || null)},
})`
}
case "union": {
const variants = maybeNewConst(
value.name + "_variants",
convertVariants(
value.spec.variants,
value.spec["variant-names"] || {},
),
)
const unionValueName = maybeNewConst(
value.name + "_union",
`${rangeToTodoComment(value?.range)}
Value.union({
name: ${JSON.stringify(value?.spec?.tag?.name || null)},
description: ${JSON.stringify(
value?.spec?.tag?.description || null,
)},
warning: ${JSON.stringify(value?.spec?.tag?.warning || null)},
default: ${JSON.stringify(value?.spec?.default || null)},
variants: ${variants},
})
`,
)
const listInputSpec = maybeNewConst(
value.name + "_list_inputSpec",
`
InputSpec.of({
"union": ${unionValueName}
})
`,
)
return `${rangeToTodoComment(value?.range)}List.obj({
name:${JSON.stringify(value.name || null)},
minLength:${JSON.stringify(null)},
maxLength:${JSON.stringify(null)},
default: [],
description: ${JSON.stringify(value.description || null)},
warning: ${JSON.stringify(value.warning || null)},
}, {
spec: ${listInputSpec},
displayAs: ${JSON.stringify(value?.spec?.["display-as"] || null)},
uniqueBy: ${JSON.stringify(value?.spec?.["unique-by"] || null)},
})`
}
}
throw new Error(`Unknown subtype "${value.subtype}"`)
}
function convertVariants(
variants: Record<string, unknown>,
variantNames: Record<string, string>,
): string {
let answer = "Variants.of({"
for (const [key, value] of Object.entries(variants)) {
const variantSpec = maybeNewConst(key, convertInputSpec(value))
answer += `"${key}": {name: "${
variantNames[key] || key
}", spec: ${variantSpec}},`
}
return `${answer}})`
}
function getNextConstName(name: string, i = 0): string {
const newName = !i ? name : name + i
if (namedConsts.has(newName)) {
return getNextConstName(name, i + 1)
}
namedConsts.add(newName)
return newName
}
}
function rangeToTodoComment(range: string | undefined) {
if (!range) return ""
return `/* TODO: Convert range for this value (${range})*/`
}
// oldSpecToBuilder(
// "./inputSpec.ts",
// // Put inputSpec here
// {},
// )

View File

@@ -16,5 +16,5 @@
"resolveJsonModule": true
},
"include": ["lib/**/*"],
"exclude": ["lib/**/*.spec.ts", "lib/**/*.gen.ts", "list", "node_modules"]
"exclude": ["lib/**/*.spec.ts", "lib/**/*.test.ts", "lib/**/*.gen.ts", "list", "node_modules"]
}