Feature/lxc container runtime (#2514)

* wip: static-server errors * wip: fix wifi * wip: Fix the service_effects * wip: Fix cors in the middleware * wip(chore): Auth clean up the lint. * wip(fix): Vhost * wip: continue manager refactor Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: service manager refactor * wip: Some fixes * wip(fix): Fix the lib.rs * wip * wip(fix): Logs * wip: bins * wip(innspect): Add in the inspect * wip: config * wip(fix): Diagnostic * wip(fix): Dependencies * wip: context * wip(fix) Sorta auth * wip: warnings * wip(fix): registry/admin * wip(fix) marketplace * wip(fix) Some more converted and fixed with the linter and config * wip: Working on the static server * wip(fix)static server * wip: Remove some asynnc * wip: Something about the request and regular rpc * wip: gut install Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: Convert the static server into the new system * wip delete file * test * wip(fix) vhost does not need the with safe defaults * wip: Adding in the wifi * wip: Fix the developer and the verify * wip: new install flow Co-authored-by: J H <Blu-J@users.noreply.github.com> * fix middleware * wip * wip: Fix the auth * wip * continue service refactor * feature: Service get_config * feat: Action * wip: Fighting the great fight against the borrow checker * wip: Remove an error in a file that I just need to deel with later * chore: Add in some more lifetime stuff to the services * wip: Install fix on lifetime * cleanup * wip: Deal with the borrow later * more cleanup * resolve borrowchecker errors * wip(feat): add in the handler for the socket, for now * wip(feat): Update the service_effect_handler::action * chore: Add in the changes to make sure the from_service goes to context * chore: Change the * refactor service map * fix references to service map * fill out restore * wip: Before I work on the store stuff * fix backup module * handle some warnings * feat: add in the ui components on the rust side * feature: Update the procedures * chore: Update the js side of the main and a few of the others * chore: Update the rpc listener to match the persistant container * wip: Working on updating some things to have a better name * wip(feat): Try and get the rpc to return the correct shape? * lxc wip * wip(feat): Try and get the rpc to return the correct shape? * build for container runtime wip * remove container-init * fix build * fix error * chore: Update to work I suppose * lxc wip * remove docker module and feature * download alpine squashfs automatically * overlays effect Co-authored-by: Jade <Blu-J@users.noreply.github.com> * chore: Add the overlay effect * feat: Add the mounter in the main * chore: Convert to use the mounts, still need to work with the sandbox * install fixes * fix ssl * fixes from testing * implement tmpfile for upload * wip * misc fixes * cleanup * cleanup * better progress reporting * progress for sideload * return real guid * add devmode script * fix lxc rootfs path * fix percentage bar * fix progress bar styling * fix build for unstable * tweaks * label progress * tweaks * update progress more often * make symlink in rpc_client * make socket dir * fix parent path * add start-cli to container * add echo and gitInfo commands * wip: Add the init + errors * chore: Add in the exit effect for the system * chore: Change the type to null for failure to parse * move sigterm timeout to stopping status * update order * chore: Update the return type * remove dbg * change the map error * chore: Update the thing to capture id * chore add some life changes * chore: Update the loging * chore: Update the package to run module * us From for RpcError * chore: Update to use import instead * chore: update * chore: Use require for the backup * fix a default * update the type that is wrong * chore: Update the type of the manifest * chore: Update to make null * only symlink if not exists * get rid of double result * better debug info for ErrorCollection * chore: Update effects * chore: fix * mount assets and volumes * add exec instead of spawn * fix mounting in image * fix overlay mounts Co-authored-by: Jade <Blu-J@users.noreply.github.com> * misc fixes * feat: Fix two * fix: systemForEmbassy main * chore: Fix small part of main loop * chore: Modify the bundle * merge * fixMain loop" * move tsc to makefile * chore: Update the return types of the health check * fix client * chore: Convert the todo to use tsmatches * add in the fixes for the seen and create the hack to allow demo * chore: Update to include the systemForStartOs * chore UPdate to the latest types from the expected outout * fixes * fix typo * Don't emit if failure on tsc * wip Co-authored-by: Jade <Blu-J@users.noreply.github.com> * add s9pk api * add inspection * add inspect manifest * newline after display serializable * fix squashfs in image name * edit manifest Co-authored-by: Jade <Blu-J@users.noreply.github.com> * wait for response on repl * ignore sig for now * ignore sig for now * re-enable sig verification * fix * wip * env and chroot * add profiling logs * set uid & gid in squashfs to 100000 * set uid of sqfs to 100000 * fix mksquashfs args * add env to compat * fix * re-add docker feature flag * fix docker output format being stupid * here be dragons * chore: Add in the cross compiling for something * fix npm link * extract logs from container on exit * chore: Update for testing * add log capture to drop trait * chore: add in the modifications that I make * chore: Update small things for no updates * chore: Update the types of something * chore: Make main not complain * idmapped mounts * idmapped volumes * re-enable kiosk * chore: Add in some logging for the new system * bring in start-sdk * remove avahi * chore: Update the deps * switch to musl * chore: Update the version of prettier * chore: Organize' * chore: Update some of the headers back to the standard of fetch * fix musl build * fix idmapped mounts * fix cross build * use cross compiler for correct arch * feat: Add in the faked ssl stuff for the effects * @dr_bonez Did a solution here * chore: Something that DrBonez * chore: up * wip: We have a working server!!! * wip * uninstall * wip * tes --------- Co-authored-by: J H <dragondef@gmail.com> Co-authored-by: J H <Blu-J@users.noreply.github.com> Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com>
2026-03-30 20:14:49 +00:00 · 2024-02-17 11:14:14 -07:00
parent 65009e2f69
commit fab13db4b4
326 changed files with 31708 additions and 13987 deletions
--- a/core/startos/src/system.rs
+++ b/core/startos/src/system.rs
@@ -2,11 +2,11 @@ use std::collections::BTreeSet;
 use std::fmt;

 use chrono::Utc;
-use clap::ArgMatches;
+use clap::Parser;
 use color_eyre::eyre::eyre;
 use futures::FutureExt;
-use rpc_toolkit::command;
 use rpc_toolkit::yajrc::RpcError;
+use rpc_toolkit::{command, from_fn_async, AnyContext, Empty, HandlerExt, ParentHandler};
 use serde::{Deserialize, Deserializer, Serialize, Serializer};
 use tokio::process::Command;
 use tokio::sync::broadcast::Receiver;
@@ -22,13 +22,27 @@ use crate::logs::{
 use crate::prelude::*;
 use crate::shutdown::Shutdown;
 use crate::util::cpupower::{get_available_governors, set_governor, Governor};
-use crate::util::serde::{display_serializable, IoFormat};
-use crate::util::{display_none, Invoke};
+use crate::util::serde::{display_serializable, HandlerExtSerde, WithIoFormat};
+use crate::util::Invoke;
 use crate::{Error, ErrorKind, ResultExt};

-#[command(subcommands(zram, governor))]
-pub async fn experimental() -> Result<(), Error> {
-    Ok(())
+pub fn experimental() -> ParentHandler {
+    ParentHandler::new()
+        .subcommand(
+            "zram",
+            from_fn_async(zram)
+                .no_display()
+                .with_remote_cli::<CliContext>(),
+        )
+        .subcommand(
+            "governor",
+            from_fn_async(governor)
+                .with_display_serializable()
+                .with_custom_display_fn::<AnyContext, _>(|handle, result| {
+                    Ok(display_governor_info(handle.params, result))
+                })
+                .with_remote_cli::<CliContext>(),
+        )
 }

 pub async fn enable_zram() -> Result<(), Error> {
@@ -59,8 +73,14 @@ pub async fn enable_zram() -> Result<(), Error> {
    Ok(())
 }

-#[command(display(display_none))]
-pub async fn zram(#[context] ctx: RpcContext, #[arg] enable: bool) -> Result<(), Error> {
+#[derive(Deserialize, Serialize, Parser)]
+#[serde(rename_all = "kebab-case")]
+#[command(rename_all = "kebab-case")]
+pub struct ZramParams {
+    enable: bool,
+}
+
+pub async fn zram(ctx: RpcContext, ZramParams { enable }: ZramParams) -> Result<(), Error> {
    let db = ctx.db.peek().await;

    let zram = db.as_server_info().as_zram().de()?;
@@ -93,17 +113,17 @@ pub struct GovernorInfo {
    available: BTreeSet<Governor>,
 }

-fn display_governor_info(arg: GovernorInfo, matches: &ArgMatches) {
+fn display_governor_info(params: WithIoFormat<GovernorParams>, result: GovernorInfo) {
    use prettytable::*;

-    if matches.is_present("format") {
-        return display_serializable(arg, matches);
+    if let Some(format) = params.format {
+        return display_serializable(format, params);
    }

    let mut table = Table::new();
    table.add_row(row![bc -> "GOVERNORS"]);
-    for entry in arg.available {
-        if Some(&entry) == arg.current.as_ref() {
+    for entry in result.available {
+        if Some(&entry) == result.current.as_ref() {
            table.add_row(row![g -> format!("* {entry} (current)")]);
        } else {
            table.add_row(row![entry]);
@@ -112,13 +132,16 @@ fn display_governor_info(arg: GovernorInfo, matches: &ArgMatches) {
    table.print_tty(false).unwrap();
 }

-#[command(display(display_governor_info))]
+#[derive(Deserialize, Serialize, Parser)]
+#[serde(rename_all = "kebab-case")]
+#[command(rename_all = "kebab-case")]
+pub struct GovernorParams {
+    set: Option<Governor>,
+}
+
 pub async fn governor(
-    #[context] ctx: RpcContext,
-    #[allow(unused_variables)]
-    #[arg(long = "format")]
-    format: Option<IoFormat>,
-    #[arg] set: Option<Governor>,
+    ctx: RpcContext,
+    GovernorParams { set, .. }: GovernorParams,
 ) -> Result<GovernorInfo, Error> {
    let available = get_available_governors().await?;
    if let Some(set) = set {
@@ -143,13 +166,13 @@ pub struct TimeInfo {
    uptime: u64,
 }

-fn display_time(arg: TimeInfo, matches: &ArgMatches) {
+pub fn display_time(params: WithIoFormat<Empty>, arg: TimeInfo) {
    use std::fmt::Write;

    use prettytable::*;

-    if matches.is_present("format") {
-        return display_serializable(arg, matches);
+    if let Some(format) = params.format {
+        return display_serializable(format, arg);
    }

    let days = arg.uptime / (24 * 60 * 60);
@@ -185,35 +208,57 @@ fn display_time(arg: TimeInfo, matches: &ArgMatches) {
    table.print_tty(false).unwrap();
 }

-#[command(display(display_time))]
-pub async fn time(
-    #[context] ctx: RpcContext,
-    #[allow(unused_variables)]
-    #[arg(long = "format")]
-    format: Option<IoFormat>,
-) -> Result<TimeInfo, Error> {
+pub async fn time(ctx: RpcContext, _: Empty) -> Result<TimeInfo, Error> {
    Ok(TimeInfo {
        now: Utc::now().to_rfc3339(),
        uptime: ctx.start_time.elapsed().as_secs(),
    })
 }
-
-#[command(
-    custom_cli(cli_logs(async, context(CliContext))),
-    subcommands(self(logs_nofollow(async)), logs_follow),
-    display(display_none)
-)]
-pub async fn logs(
-    #[arg(short = 'l', long = "limit")] limit: Option<usize>,
-    #[arg(short = 'c', long = "cursor")] cursor: Option<String>,
-    #[arg(short = 'B', long = "before", default)] before: bool,
-    #[arg(short = 'f', long = "follow", default)] follow: bool,
-) -> Result<(Option<usize>, Option<String>, bool, bool), Error> {
-    Ok((limit, cursor, before, follow))
+#[derive(Deserialize, Serialize, Parser)]
+#[serde(rename_all = "kebab-case")]
+#[command(rename_all = "kebab-case")]
+pub struct LogsParams {
+    #[arg(short = 'l', long = "limit")]
+    limit: Option<usize>,
+    #[arg(short = 'c', long = "cursor")]
+    cursor: Option<String>,
+    #[arg(short = 'B', long = "before")]
+    #[serde(default)]
+    before: bool,
+    #[arg(short = 'f', long = "follow")]
+    #[serde(default)]
+    follow: bool,
 }
+
+pub fn logs() -> ParentHandler<LogsParams> {
+    ParentHandler::new()
+        .root_handler(
+            from_fn_async(cli_logs)
+                .no_display()
+                .with_inherited(|params, _| params),
+        )
+        .root_handler(
+            from_fn_async(logs_nofollow)
+                .with_inherited(|params, _| params)
+                .no_cli(),
+        )
+        .subcommand(
+            "follow",
+            from_fn_async(logs_follow)
+                .with_inherited(|params, _| params)
+                .no_cli(),
+        )
+}
+
 pub async fn cli_logs(
    ctx: CliContext,
-    (limit, cursor, before, follow): (Option<usize>, Option<String>, bool, bool),
+    _: Empty,
+    LogsParams {
+        limit,
+        cursor,
+        before,
+        follow,
+    }: LogsParams,
 ) -> Result<(), RpcError> {
    if follow {
        if cursor.is_some() {
@@ -234,37 +279,68 @@ pub async fn cli_logs(
    }
 }
 pub async fn logs_nofollow(
-    _ctx: (),
-    (limit, cursor, before, _): (Option<usize>, Option<String>, bool, bool),
+    _ctx: AnyContext,
+    _: Empty,
+    LogsParams {
+        limit,
+        cursor,
+        before,
+        ..
+    }: LogsParams,
 ) -> Result<LogResponse, Error> {
    fetch_logs(LogSource::System, limit, cursor, before).await
 }

-#[command(rpc_only, rename = "follow", display(display_none))]
 pub async fn logs_follow(
-    #[context] ctx: RpcContext,
-    #[parent_data] (limit, _, _, _): (Option<usize>, Option<String>, bool, bool),
+    ctx: RpcContext,
+    _: Empty,
+    LogsParams { limit, .. }: LogsParams,
 ) -> Result<LogFollowResponse, Error> {
    follow_logs(ctx, LogSource::System, limit).await
 }
-
-#[command(
-    rename = "kernel-logs",
-    custom_cli(cli_kernel_logs(async, context(CliContext))),
-    subcommands(self(kernel_logs_nofollow(async)), kernel_logs_follow),
-    display(display_none)
-)]
-pub async fn kernel_logs(
-    #[arg(short = 'l', long = "limit")] limit: Option<usize>,
-    #[arg(short = 'c', long = "cursor")] cursor: Option<String>,
-    #[arg(short = 'B', long = "before", default)] before: bool,
-    #[arg(short = 'f', long = "follow", default)] follow: bool,
-) -> Result<(Option<usize>, Option<String>, bool, bool), Error> {
-    Ok((limit, cursor, before, follow))
+#[derive(Deserialize, Serialize, Parser)]
+#[serde(rename_all = "kebab-case")]
+#[command(rename_all = "kebab-case")]
+pub struct KernelLogsParams {
+    #[arg(short = 'l', long = "limit")]
+    limit: Option<usize>,
+    #[arg(short = 'c', long = "cursor")]
+    cursor: Option<String>,
+    #[arg(short = 'B', long = "before")]
+    #[serde(default)]
+    before: bool,
+    #[arg(short = 'f', long = "follow")]
+    #[serde(default)]
+    follow: bool,
+}
+pub fn kernel_logs() -> ParentHandler<KernelLogsParams> {
+    ParentHandler::new()
+        .root_handler(
+            from_fn_async(cli_kernel_logs)
+                .no_display()
+                .with_inherited(|params, _| params),
+        )
+        .root_handler(
+            from_fn_async(kernel_logs_nofollow)
+                .with_inherited(|params, _| params)
+                .no_cli(),
+        )
+        .subcommand(
+            "follow",
+            from_fn_async(kernel_logs_follow)
+                .with_inherited(|params, _| params)
+                .no_cli(),
+        )
 }
 pub async fn cli_kernel_logs(
    ctx: CliContext,
-    (limit, cursor, before, follow): (Option<usize>, Option<String>, bool, bool),
+    _: Empty,
+    KernelLogsParams {
+        limit,
+        cursor,
+        before,
+        follow,
+    }: KernelLogsParams,
 ) -> Result<(), RpcError> {
    if follow {
        if cursor.is_some() {
@@ -285,16 +361,22 @@ pub async fn cli_kernel_logs(
    }
 }
 pub async fn kernel_logs_nofollow(
-    _ctx: (),
-    (limit, cursor, before, _): (Option<usize>, Option<String>, bool, bool),
+    _ctx: AnyContext,
+    _: Empty,
+    KernelLogsParams {
+        limit,
+        cursor,
+        before,
+        ..
+    }: KernelLogsParams,
 ) -> Result<LogResponse, Error> {
    fetch_logs(LogSource::Kernel, limit, cursor, before).await
 }

-#[command(rpc_only, rename = "follow", display(display_none))]
 pub async fn kernel_logs_follow(
-    #[context] ctx: RpcContext,
-    #[parent_data] (limit, _, _, _): (Option<usize>, Option<String>, bool, bool),
+    ctx: RpcContext,
+    _: Empty,
+    KernelLogsParams { limit, .. }: KernelLogsParams,
 ) -> Result<LogFollowResponse, Error> {
    follow_logs(ctx, LogSource::Kernel, limit).await
 }
@@ -453,13 +535,8 @@ pub struct Metrics {
    disk: MetricsDisk,
 }

-#[command(display(display_serializable))]
-pub async fn metrics(
-    #[context] ctx: RpcContext,
-    #[allow(unused_variables)]
-    #[arg(long = "format")]
-    format: Option<IoFormat>,
-) -> Result<Metrics, Error> {
+// #[command(display(display_serializable))]
+pub async fn metrics(ctx: RpcContext, _: Empty) -> Result<Metrics, Error> {
    match ctx.metrics_cache.read().await.clone() {
        None => Err(Error {
            source: color_eyre::eyre::eyre!("No Metrics Found"),