mirror of
https://github.com/Start9Labs/start-os.git
synced 2026-03-26 02:11:53 +00:00
add documentation for ai agents (#3115)
* add documentation for ai agents * docs: consolidate CLAUDE.md and CONTRIBUTING.md, add style guidelines - Refactor CLAUDE.md to reference CONTRIBUTING.md for build/test/format info - Expand CONTRIBUTING.md with comprehensive build targets, env vars, and testing - Add code style guidelines section with conventional commits - Standardize SDK prettier config to use single quotes (matching web) - Add project-level Claude Code settings to disable co-author attribution * style(sdk): apply prettier with single quotes Run prettier across sdk/base and sdk/package to apply the standardized quote style (single quotes matching web). * docs: add USER.md for per-developer TODO filtering - Add agents/USER.md to .gitignore (contains user identifier) - Document session startup flow in CLAUDE.md: - Create USER.md if missing, prompting for identifier - Filter TODOs by @username tags - Offer relevant TODOs on session start * docs: add i18n documentation task to agent TODOs * docs: document i18n ID patterns in core/ Add agents/i18n-patterns.md covering rust-i18n setup, translation file format, t!() macro usage, key naming conventions, and locale selection. Remove completed TODO item and add reference in CLAUDE.md. * chore: clarify that all builds work on any OS with Docker
This commit is contained in:
9
agents/TODO.md
Normal file
9
agents/TODO.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# AI Agent TODOs
|
||||
|
||||
Pending tasks for AI agents. Remove items when completed.
|
||||
|
||||
## Unreviewed CLAUDE.md Sections
|
||||
|
||||
- [ ] Architecture - Web (`/web`) - @MattDHill
|
||||
|
||||
|
||||
249
agents/core-rust-patterns.md
Normal file
249
agents/core-rust-patterns.md
Normal file
@@ -0,0 +1,249 @@
|
||||
# Utilities & Patterns
|
||||
|
||||
This document covers common utilities and patterns used throughout the StartOS codebase.
|
||||
|
||||
## Util Module (`core/src/util/`)
|
||||
|
||||
The `util` module contains reusable utilities. Key submodules:
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `actor/` | Actor pattern implementation for concurrent state management |
|
||||
| `collections/` | Custom collection types |
|
||||
| `crypto.rs` | Cryptographic utilities (encryption, hashing) |
|
||||
| `future.rs` | Future/async utilities |
|
||||
| `io.rs` | File I/O helpers (create_file, canonicalize, etc.) |
|
||||
| `iter.rs` | Iterator extensions |
|
||||
| `net.rs` | Network utilities |
|
||||
| `rpc.rs` | RPC helpers |
|
||||
| `rpc_client.rs` | RPC client utilities |
|
||||
| `serde.rs` | Serialization helpers (Base64, display/fromstr, etc.) |
|
||||
| `sync.rs` | Synchronization primitives (SyncMutex, etc.) |
|
||||
|
||||
## Command Invocation (`Invoke` trait)
|
||||
|
||||
The `Invoke` trait provides a clean way to run external commands with error handling:
|
||||
|
||||
```rust
|
||||
use crate::util::Invoke;
|
||||
|
||||
// Simple invocation
|
||||
tokio::process::Command::new("ls")
|
||||
.arg("-la")
|
||||
.invoke(ErrorKind::Filesystem)
|
||||
.await?;
|
||||
|
||||
// With timeout
|
||||
tokio::process::Command::new("slow-command")
|
||||
.timeout(Some(Duration::from_secs(30)))
|
||||
.invoke(ErrorKind::Timeout)
|
||||
.await?;
|
||||
|
||||
// With input
|
||||
let mut input = Cursor::new(b"input data");
|
||||
tokio::process::Command::new("cat")
|
||||
.input(Some(&mut input))
|
||||
.invoke(ErrorKind::Filesystem)
|
||||
.await?;
|
||||
|
||||
// Piped commands
|
||||
tokio::process::Command::new("cat")
|
||||
.arg("file.txt")
|
||||
.pipe(&mut tokio::process::Command::new("grep").arg("pattern"))
|
||||
.invoke(ErrorKind::Filesystem)
|
||||
.await?;
|
||||
```
|
||||
|
||||
## Guard Pattern
|
||||
|
||||
Guards ensure cleanup happens when they go out of scope.
|
||||
|
||||
### `GeneralGuard` / `GeneralBoxedGuard`
|
||||
|
||||
For arbitrary cleanup actions:
|
||||
|
||||
```rust
|
||||
use crate::util::GeneralGuard;
|
||||
|
||||
let guard = GeneralGuard::new(|| {
|
||||
println!("Cleanup runs on drop");
|
||||
});
|
||||
|
||||
// Do work...
|
||||
|
||||
// Explicit drop with action
|
||||
guard.drop();
|
||||
|
||||
// Or skip the action
|
||||
// guard.drop_without_action();
|
||||
```
|
||||
|
||||
### `FileLock`
|
||||
|
||||
File-based locking with automatic unlock:
|
||||
|
||||
```rust
|
||||
use crate::util::FileLock;
|
||||
|
||||
let lock = FileLock::new("/path/to/lockfile", true).await?; // blocking=true
|
||||
// Lock held until dropped or explicitly unlocked
|
||||
lock.unlock().await?;
|
||||
```
|
||||
|
||||
## Mount Guard Pattern (`core/src/disk/mount/guard.rs`)
|
||||
|
||||
RAII guards for filesystem mounts. Ensures filesystems are unmounted when guards are dropped.
|
||||
|
||||
### `MountGuard`
|
||||
|
||||
Basic mount guard:
|
||||
|
||||
```rust
|
||||
use crate::disk::mount::guard::MountGuard;
|
||||
use crate::disk::mount::filesystem::{MountType, ReadOnly};
|
||||
|
||||
let guard = MountGuard::mount(&filesystem, "/mnt/target", ReadOnly).await?;
|
||||
|
||||
// Use the mounted filesystem at guard.path()
|
||||
do_something(guard.path()).await?;
|
||||
|
||||
// Explicit unmount (or auto-unmounts on drop)
|
||||
guard.unmount(false).await?; // false = don't delete mountpoint
|
||||
```
|
||||
|
||||
### `TmpMountGuard`
|
||||
|
||||
Reference-counted temporary mount (mounts to `/media/startos/tmp/`):
|
||||
|
||||
```rust
|
||||
use crate::disk::mount::guard::TmpMountGuard;
|
||||
use crate::disk::mount::filesystem::ReadOnly;
|
||||
|
||||
// Multiple clones share the same mount
|
||||
let guard1 = TmpMountGuard::mount(&filesystem, ReadOnly).await?;
|
||||
let guard2 = guard1.clone();
|
||||
|
||||
// Mount stays alive while any guard exists
|
||||
// Auto-unmounts when last guard is dropped
|
||||
```
|
||||
|
||||
### `GenericMountGuard` trait
|
||||
|
||||
All mount guards implement this trait:
|
||||
|
||||
```rust
|
||||
pub trait GenericMountGuard: std::fmt::Debug + Send + Sync + 'static {
|
||||
fn path(&self) -> &Path;
|
||||
fn unmount(self) -> impl Future<Output = Result<(), Error>> + Send;
|
||||
}
|
||||
```
|
||||
|
||||
### `SubPath`
|
||||
|
||||
Wraps a mount guard to point to a subdirectory:
|
||||
|
||||
```rust
|
||||
use crate::disk::mount::guard::SubPath;
|
||||
|
||||
let mount = TmpMountGuard::mount(&filesystem, ReadOnly).await?;
|
||||
let subdir = SubPath::new(mount, "data/subdir");
|
||||
|
||||
// subdir.path() returns the full path including subdirectory
|
||||
```
|
||||
|
||||
## FileSystem Implementations (`core/src/disk/mount/filesystem/`)
|
||||
|
||||
Various filesystem types that can be mounted:
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `bind.rs` | Bind mounts |
|
||||
| `block_dev.rs` | Block device mounts |
|
||||
| `cifs.rs` | CIFS/SMB network shares |
|
||||
| `ecryptfs.rs` | Encrypted filesystem |
|
||||
| `efivarfs.rs` | EFI variables |
|
||||
| `httpdirfs.rs` | HTTP directory as filesystem |
|
||||
| `idmapped.rs` | ID-mapped mounts |
|
||||
| `label.rs` | Mount by label |
|
||||
| `loop_dev.rs` | Loop device mounts |
|
||||
| `overlayfs.rs` | Overlay filesystem |
|
||||
|
||||
## Other Useful Utilities
|
||||
|
||||
### `Apply` / `ApplyRef` traits
|
||||
|
||||
Fluent method chaining:
|
||||
|
||||
```rust
|
||||
use crate::util::Apply;
|
||||
|
||||
let result = some_value
|
||||
.apply(|v| transform(v))
|
||||
.apply(|v| another_transform(v));
|
||||
```
|
||||
|
||||
### `Container<T>`
|
||||
|
||||
Async-safe optional container:
|
||||
|
||||
```rust
|
||||
use crate::util::Container;
|
||||
|
||||
let container = Container::new(None);
|
||||
container.set(value).await;
|
||||
let taken = container.take().await;
|
||||
```
|
||||
|
||||
### `HashWriter<H, W>`
|
||||
|
||||
Write data while computing hash:
|
||||
|
||||
```rust
|
||||
use crate::util::HashWriter;
|
||||
use sha2::Sha256;
|
||||
|
||||
let writer = HashWriter::new(Sha256::new(), file);
|
||||
// Write data...
|
||||
let (hasher, file) = writer.finish();
|
||||
let hash = hasher.finalize();
|
||||
```
|
||||
|
||||
### `Never` type
|
||||
|
||||
Uninhabited type for impossible cases:
|
||||
|
||||
```rust
|
||||
use crate::util::Never;
|
||||
|
||||
fn impossible() -> Never {
|
||||
// This function can never return
|
||||
}
|
||||
|
||||
let never: Never = impossible();
|
||||
never.absurd::<String>() // Can convert to any type
|
||||
```
|
||||
|
||||
### `MaybeOwned<'a, T>`
|
||||
|
||||
Either borrowed or owned data:
|
||||
|
||||
```rust
|
||||
use crate::util::MaybeOwned;
|
||||
|
||||
fn accept_either(data: MaybeOwned<'_, String>) {
|
||||
// Use &*data to access the value
|
||||
}
|
||||
|
||||
accept_either(MaybeOwned::from(&existing_string));
|
||||
accept_either(MaybeOwned::from(owned_string));
|
||||
```
|
||||
|
||||
### `new_guid()`
|
||||
|
||||
Generate a random GUID:
|
||||
|
||||
```rust
|
||||
use crate::util::new_guid;
|
||||
|
||||
let guid = new_guid(); // Returns InternedString
|
||||
```
|
||||
100
agents/i18n-patterns.md
Normal file
100
agents/i18n-patterns.md
Normal file
@@ -0,0 +1,100 @@
|
||||
# i18n Patterns in `core/`
|
||||
|
||||
## Library & Setup
|
||||
|
||||
**Crate:** [`rust-i18n`](https://crates.io/crates/rust-i18n) v3.1.5 (`core/Cargo.toml`)
|
||||
|
||||
**Initialization** (`core/src/lib.rs:3`):
|
||||
```rust
|
||||
rust_i18n::i18n!("locales", fallback = ["en_US"]);
|
||||
```
|
||||
This macro scans `core/locales/` at compile time and embeds all translations as constants.
|
||||
|
||||
**Prelude re-export** (`core/src/prelude.rs:4`):
|
||||
```rust
|
||||
pub use rust_i18n::t;
|
||||
```
|
||||
Most modules import `t!` via the prelude.
|
||||
|
||||
## Translation File
|
||||
|
||||
**Location:** `core/locales/i18n.yaml`
|
||||
**Format:** YAML v2 (~755 keys)
|
||||
|
||||
**Supported languages:** `en_US`, `de_DE`, `es_ES`, `fr_FR`, `pl_PL`
|
||||
|
||||
**Entry structure:**
|
||||
```yaml
|
||||
namespace.sub.key-name:
|
||||
en_US: "English text with %{param}"
|
||||
de_DE: "German text with %{param}"
|
||||
# ...
|
||||
```
|
||||
|
||||
## Using `t!()`
|
||||
|
||||
```rust
|
||||
// Simple key
|
||||
t!("error.unknown")
|
||||
|
||||
// With parameter interpolation (%{name} in YAML)
|
||||
t!("bins.deprecated.renamed", old = old_name, new = new_name)
|
||||
```
|
||||
|
||||
## Key Naming Conventions
|
||||
|
||||
Keys use **dot-separated hierarchical namespaces** with **kebab-case** for multi-word segments:
|
||||
|
||||
```
|
||||
<module>.<submodule>.<descriptive-name>
|
||||
```
|
||||
|
||||
Examples:
|
||||
- `error.incorrect-password` — error kind label
|
||||
- `bins.start-init.updating-firmware` — startup phase message
|
||||
- `backup.bulk.complete-title` — backup notification title
|
||||
- `help.arg.acme-contact` — CLI help text for an argument
|
||||
- `context.diagnostic.starting-diagnostic-ui` — diagnostic context status
|
||||
|
||||
### Top-Level Namespaces
|
||||
|
||||
| Namespace | Purpose |
|
||||
|-----------|---------|
|
||||
| `error.*` | `ErrorKind` display strings (see `src/error.rs`) |
|
||||
| `bins.*` | CLI binary messages (deprecated, start-init, startd, etc.) |
|
||||
| `init.*` | Initialization phase labels |
|
||||
| `setup.*` | First-run setup messages |
|
||||
| `context.*` | Context startup messages (diagnostic, setup, CLI) |
|
||||
| `service.*` | Service lifecycle messages |
|
||||
| `backup.*` | Backup/restore operation messages |
|
||||
| `registry.*` | Package registry messages |
|
||||
| `net.*` | Network-related messages |
|
||||
| `middleware.*` | Request middleware messages (auth, etc.) |
|
||||
| `disk.*` | Disk operation messages |
|
||||
| `lxc.*` | Container management messages |
|
||||
| `system.*` | System monitoring/metrics messages |
|
||||
| `notifications.*` | User-facing notification messages |
|
||||
| `update.*` | OS update messages |
|
||||
| `util.*` | Utility messages (TUI, RPC) |
|
||||
| `ssh.*` | SSH operation messages |
|
||||
| `shutdown.*` | Shutdown-related messages |
|
||||
| `logs.*` | Log-related messages |
|
||||
| `auth.*` | Authentication messages |
|
||||
| `help.*` | CLI help text (`help.arg.<arg-name>`) |
|
||||
| `about.*` | CLI command descriptions |
|
||||
|
||||
## Locale Selection
|
||||
|
||||
`core/src/bins/mod.rs:15-36` — `set_locale_from_env()`:
|
||||
|
||||
1. Reads `LANG` environment variable
|
||||
2. Strips `.UTF-8` suffix
|
||||
3. Exact-matches against available locales, falls back to language-prefix match (e.g. `en_GB` matches `en_US`)
|
||||
|
||||
## Adding New Keys
|
||||
|
||||
1. Add the key to `core/locales/i18n.yaml` with all 5 language translations
|
||||
2. Use the `t!("your.key.name")` macro in Rust code
|
||||
3. Follow existing namespace conventions — match the module path where the key is used
|
||||
4. Use kebab-case for multi-word segments
|
||||
5. Translations are validated at compile time
|
||||
226
agents/rpc-toolkit.md
Normal file
226
agents/rpc-toolkit.md
Normal file
@@ -0,0 +1,226 @@
|
||||
# rpc-toolkit
|
||||
|
||||
StartOS uses [rpc-toolkit](https://github.com/Start9Labs/rpc-toolkit) for its JSON-RPC API. This document covers the patterns used in this codebase.
|
||||
|
||||
## Overview
|
||||
|
||||
The API is JSON-RPC (not REST). All endpoints are RPC methods organized in a hierarchical command structure.
|
||||
|
||||
## Handler Functions
|
||||
|
||||
There are four types of handler functions, chosen based on the function's characteristics:
|
||||
|
||||
### `from_fn_async` - Async handlers
|
||||
For standard async functions. Most handlers use this.
|
||||
|
||||
```rust
|
||||
pub async fn my_handler(ctx: RpcContext, params: MyParams) -> Result<MyResponse, Error> {
|
||||
// Can use .await
|
||||
}
|
||||
|
||||
from_fn_async(my_handler)
|
||||
```
|
||||
|
||||
### `from_fn_async_local` - Non-thread-safe async handlers
|
||||
For async functions that are not `Send` (cannot be safely moved between threads). Use when working with non-thread-safe types.
|
||||
|
||||
```rust
|
||||
pub async fn cli_download(ctx: CliContext, params: Params) -> Result<(), Error> {
|
||||
// Non-Send async operations
|
||||
}
|
||||
|
||||
from_fn_async_local(cli_download)
|
||||
```
|
||||
|
||||
### `from_fn_blocking` - Sync blocking handlers
|
||||
For synchronous functions that perform blocking I/O or long computations.
|
||||
|
||||
```rust
|
||||
pub fn query_dns(ctx: RpcContext, params: DnsParams) -> Result<DnsResponse, Error> {
|
||||
// Blocking operations (file I/O, DNS lookup, etc.)
|
||||
}
|
||||
|
||||
from_fn_blocking(query_dns)
|
||||
```
|
||||
|
||||
### `from_fn` - Sync non-blocking handlers
|
||||
For pure functions or quick synchronous operations with no I/O.
|
||||
|
||||
```rust
|
||||
pub fn echo(ctx: RpcContext, params: EchoParams) -> Result<String, Error> {
|
||||
Ok(params.message)
|
||||
}
|
||||
|
||||
from_fn(echo)
|
||||
```
|
||||
|
||||
## ParentHandler
|
||||
|
||||
Groups related RPC methods into a hierarchy:
|
||||
|
||||
```rust
|
||||
use rpc_toolkit::{Context, HandlerExt, ParentHandler, from_fn_async};
|
||||
|
||||
pub fn my_api<C: Context>() -> ParentHandler<C> {
|
||||
ParentHandler::new()
|
||||
.subcommand("list", from_fn_async(list_handler).with_call_remote::<CliContext>())
|
||||
.subcommand("create", from_fn_async(create_handler).with_call_remote::<CliContext>())
|
||||
}
|
||||
```
|
||||
|
||||
## Handler Extensions
|
||||
|
||||
Chain methods to configure handler behavior.
|
||||
|
||||
**Ordering rules:**
|
||||
1. `with_about()` must come AFTER other CLI modifiers (`no_display()`, `with_custom_display_fn()`, etc.)
|
||||
2. `with_call_remote()` must be the LAST adapter in the chain
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `.with_metadata("key", Value)` | Attach metadata for middleware |
|
||||
| `.no_cli()` | RPC-only, not available via CLI |
|
||||
| `.no_display()` | No CLI output |
|
||||
| `.with_display_serializable()` | Default JSON/YAML output for CLI |
|
||||
| `.with_custom_display_fn(\|_, res\| ...)` | Custom CLI output formatting |
|
||||
| `.with_about("about.description")` | Add help text (i18n key) - **after CLI modifiers** |
|
||||
| `.with_call_remote::<CliContext>()` | Enable CLI to call remotely - **must be last** |
|
||||
|
||||
### Correct ordering example:
|
||||
```rust
|
||||
from_fn_async(my_handler)
|
||||
.with_metadata("sync_db", Value::Bool(true)) // metadata early
|
||||
.no_display() // CLI modifier
|
||||
.with_about("about.my-handler") // after CLI modifiers
|
||||
.with_call_remote::<CliContext>() // always last
|
||||
```
|
||||
|
||||
## Metadata by Middleware
|
||||
|
||||
Metadata tags are processed by different middleware. Group them logically:
|
||||
|
||||
### Auth Middleware (`middleware/auth/mod.rs`)
|
||||
|
||||
| Metadata | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `authenticated` | `true` | Whether endpoint requires authentication. Set to `false` for public endpoints. |
|
||||
|
||||
### Session Auth Middleware (`middleware/auth/session.rs`)
|
||||
|
||||
| Metadata | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `login` | `false` | Special handling for login endpoints (rate limiting, cookie setting) |
|
||||
| `get_session` | `false` | Inject session ID into params as `__Auth_session` |
|
||||
|
||||
### Signature Auth Middleware (`middleware/auth/signature.rs`)
|
||||
|
||||
| Metadata | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `get_signer` | `false` | Inject signer public key into params as `__Auth_signer` |
|
||||
|
||||
### Registry Auth (extends Signature Auth)
|
||||
|
||||
| Metadata | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `admin` | `false` | Require admin privileges (signer must be in admin list) |
|
||||
| `get_device_info` | `false` | Inject device info header for hardware filtering |
|
||||
|
||||
### Database Middleware (`middleware/db.rs`)
|
||||
|
||||
| Metadata | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `sync_db` | `false` | Sync database after mutation, add `X-Patch-Sequence` header |
|
||||
|
||||
## Context Types
|
||||
|
||||
Different contexts for different execution environments:
|
||||
|
||||
- `RpcContext` - Web/RPC requests with full service access
|
||||
- `CliContext` - CLI operations, calls remote RPC
|
||||
- `InitContext` - During system initialization
|
||||
- `DiagnosticContext` - Diagnostic/recovery mode
|
||||
- `RegistryContext` - Registry daemon context
|
||||
- `EffectContext` - Service effects context (container-to-host calls)
|
||||
|
||||
## Parameter Structs
|
||||
|
||||
Parameters use derive macros for JSON-RPC, CLI parsing, and TypeScript generation:
|
||||
|
||||
```rust
|
||||
#[derive(Deserialize, Serialize, Parser, TS)]
|
||||
#[serde(rename_all = "camelCase")] // JSON-RPC uses camelCase
|
||||
#[command(rename_all = "kebab-case")] // CLI uses kebab-case
|
||||
#[ts(export)] // Generate TypeScript types
|
||||
pub struct MyParams {
|
||||
pub package_id: PackageId,
|
||||
}
|
||||
```
|
||||
|
||||
### Middleware Injection
|
||||
|
||||
Auth middleware can inject values into params using special field names:
|
||||
|
||||
```rust
|
||||
#[derive(Deserialize, Serialize, Parser, TS)]
|
||||
pub struct MyParams {
|
||||
#[ts(skip)]
|
||||
#[serde(rename = "__Auth_session")] // Injected by session auth
|
||||
session: InternedString,
|
||||
|
||||
#[ts(skip)]
|
||||
#[serde(rename = "__Auth_signer")] // Injected by signature auth
|
||||
signer: AnyVerifyingKey,
|
||||
|
||||
#[ts(skip)]
|
||||
#[serde(rename = "__Auth_userAgent")] // Injected during login
|
||||
user_agent: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Adding a New RPC Endpoint
|
||||
|
||||
1. Define params struct with `Deserialize, Serialize, Parser, TS`
|
||||
2. Choose handler type based on sync/async and thread-safety
|
||||
3. Write handler function taking `(Context, Params) -> Result<Response, Error>`
|
||||
4. Add to parent handler with appropriate extensions (display modifiers before `with_about`)
|
||||
5. TypeScript types auto-generated via `make ts-bindings`
|
||||
|
||||
### Public (Unauthenticated) Endpoint
|
||||
|
||||
```rust
|
||||
from_fn_async(get_info)
|
||||
.with_metadata("authenticated", Value::Bool(false))
|
||||
.with_display_serializable()
|
||||
.with_about("about.get-info")
|
||||
.with_call_remote::<CliContext>() // last
|
||||
```
|
||||
|
||||
### Mutating Endpoint with DB Sync
|
||||
|
||||
```rust
|
||||
from_fn_async(update_config)
|
||||
.with_metadata("sync_db", Value::Bool(true))
|
||||
.no_display()
|
||||
.with_about("about.update-config")
|
||||
.with_call_remote::<CliContext>() // last
|
||||
```
|
||||
|
||||
### Session-Aware Endpoint
|
||||
|
||||
```rust
|
||||
from_fn_async(logout)
|
||||
.with_metadata("get_session", Value::Bool(true))
|
||||
.no_display()
|
||||
.with_about("about.logout")
|
||||
.with_call_remote::<CliContext>() // last
|
||||
```
|
||||
|
||||
## File Locations
|
||||
|
||||
- Handler definitions: Throughout `core/src/` modules
|
||||
- Main API tree: `core/src/lib.rs` (`main_api()`, `server()`, `package()`)
|
||||
- Auth middleware: `core/src/middleware/auth/`
|
||||
- DB middleware: `core/src/middleware/db.rs`
|
||||
- Context types: `core/src/context/`
|
||||
122
agents/s9pk-structure.md
Normal file
122
agents/s9pk-structure.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# S9PK Package Format
|
||||
|
||||
S9PK is the package format for StartOS services. Version 2 uses a merkle archive structure for efficient downloading and cryptographic verification.
|
||||
|
||||
## File Format
|
||||
|
||||
S9PK files begin with a 3-byte header: `0x3b 0x3b 0x02` (magic bytes + version 2).
|
||||
|
||||
The archive is cryptographically signed using Ed25519 with prehashed content (SHA-512 over blake3 merkle root hash).
|
||||
|
||||
## Archive Structure
|
||||
|
||||
```
|
||||
/
|
||||
├── manifest.json # Package metadata (required)
|
||||
├── icon.<ext> # Package icon - any image/* format (required)
|
||||
├── LICENSE.md # License text (required)
|
||||
├── dependencies/ # Dependency metadata (optional)
|
||||
│ └── <package-id>/
|
||||
│ ├── metadata.json # DependencyMetadata
|
||||
│ └── icon.<ext> # Dependency icon
|
||||
├── javascript.squashfs # Package JavaScript code (required)
|
||||
├── assets.squashfs # Static assets (optional, legacy: assets/ directory)
|
||||
└── images/ # Container images by architecture
|
||||
└── <arch>/ # e.g., x86_64, aarch64, riscv64
|
||||
├── <image-id>.squashfs # Container filesystem
|
||||
├── <image-id>.json # Image metadata
|
||||
└── <image-id>.env # Environment variables
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
### manifest.json
|
||||
|
||||
The package manifest contains all metadata:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | string | Package identifier (e.g., `bitcoind`) |
|
||||
| `title` | string | Display name |
|
||||
| `version` | string | Extended version string |
|
||||
| `satisfies` | string[] | Version ranges this version satisfies |
|
||||
| `releaseNotes` | string/object | Release notes (localized) |
|
||||
| `canMigrateTo` | string | Version range for forward migration |
|
||||
| `canMigrateFrom` | string | Version range for backward migration |
|
||||
| `license` | string | License type |
|
||||
| `wrapperRepo` | string | StartOS wrapper repository URL |
|
||||
| `upstreamRepo` | string | Upstream project URL |
|
||||
| `supportSite` | string | Support site URL |
|
||||
| `marketingSite` | string | Marketing site URL |
|
||||
| `donationUrl` | string? | Optional donation URL |
|
||||
| `docsUrl` | string? | Optional documentation URL |
|
||||
| `description` | object | Short and long descriptions (localized) |
|
||||
| `images` | object | Image configurations by image ID |
|
||||
| `volumes` | string[] | Volume IDs for persistent data |
|
||||
| `alerts` | object | User alerts for lifecycle events |
|
||||
| `dependencies` | object | Package dependencies |
|
||||
| `hardwareRequirements` | object | Hardware requirements (arch, RAM, devices) |
|
||||
| `hardwareAcceleration` | boolean | Whether package uses hardware acceleration |
|
||||
| `gitHash` | string? | Git commit hash |
|
||||
| `osVersion` | string | Minimum StartOS version |
|
||||
| `sdkVersion` | string? | SDK version used to build |
|
||||
|
||||
### javascript.squashfs
|
||||
|
||||
Contains the package JavaScript that implements the `ABI` interface from `@start9labs/start-sdk-base`. This code runs in the container runtime and manages the package lifecycle.
|
||||
|
||||
The squashfs is mounted at `/usr/lib/startos/package/` and the runtime loads `index.js`.
|
||||
|
||||
### images/
|
||||
|
||||
Container images organized by architecture:
|
||||
|
||||
- **`<image-id>.squashfs`** - Container root filesystem
|
||||
- **`<image-id>.json`** - Image metadata (entrypoint, user, workdir, etc.)
|
||||
- **`<image-id>.env`** - Environment variables for the container
|
||||
|
||||
Images are built from Docker/Podman and converted to squashfs. The `ImageConfig` in manifest specifies:
|
||||
- `arch` - Supported architectures
|
||||
- `emulateMissingAs` - Fallback architecture for emulation
|
||||
- `nvidiaContainer` - Whether to enable NVIDIA container support
|
||||
|
||||
### assets.squashfs
|
||||
|
||||
Static assets accessible to the package, mounted read-only at `/media/startos/assets/` in the container.
|
||||
|
||||
### dependencies/
|
||||
|
||||
Metadata for dependencies displayed in the UI:
|
||||
- `metadata.json` - Just title for now
|
||||
- `icon.<ext>` - Icon for the dependency
|
||||
|
||||
## Merkle Archive
|
||||
|
||||
The S9PK uses a merkle tree structure where each file and directory has a blake3 hash. This enables:
|
||||
|
||||
1. **Partial downloads** - Download and verify individual files
|
||||
2. **Integrity verification** - Verify any subset of the archive
|
||||
3. **Efficient updates** - Only download changed portions
|
||||
4. **DOS protection** - Size limits enforced before downloading content
|
||||
|
||||
Files are sorted by priority for streaming (manifest first, then icon, license, dependencies, javascript, assets, images).
|
||||
|
||||
## Building S9PK
|
||||
|
||||
Use `start-cli s9pk pack` to build packages:
|
||||
|
||||
```bash
|
||||
start-cli s9pk pack <manifest-path> -o <output.s9pk>
|
||||
```
|
||||
|
||||
Images can be sourced from:
|
||||
- Docker/Podman build (`--docker-build`)
|
||||
- Existing Docker tag (`--docker-tag`)
|
||||
- Pre-built squashfs files
|
||||
|
||||
## Related Code
|
||||
|
||||
- `core/src/s9pk/v2/mod.rs` - S9pk struct and serialization
|
||||
- `core/src/s9pk/v2/manifest.rs` - Manifest types
|
||||
- `core/src/s9pk/v2/pack.rs` - Packing logic
|
||||
- `core/src/s9pk/merkle_archive/` - Merkle archive implementation
|
||||
Reference in New Issue
Block a user