Feature/lxc container runtime (#2514)

* wip: static-server errors * wip: fix wifi * wip: Fix the service_effects * wip: Fix cors in the middleware * wip(chore): Auth clean up the lint. * wip(fix): Vhost * wip: continue manager refactor Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: service manager refactor * wip: Some fixes * wip(fix): Fix the lib.rs * wip * wip(fix): Logs * wip: bins * wip(innspect): Add in the inspect * wip: config * wip(fix): Diagnostic * wip(fix): Dependencies * wip: context * wip(fix) Sorta auth * wip: warnings * wip(fix): registry/admin * wip(fix) marketplace * wip(fix) Some more converted and fixed with the linter and config * wip: Working on the static server * wip(fix)static server * wip: Remove some asynnc * wip: Something about the request and regular rpc * wip: gut install Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: Convert the static server into the new system * wip delete file * test * wip(fix) vhost does not need the with safe defaults * wip: Adding in the wifi * wip: Fix the developer and the verify * wip: new install flow Co-authored-by: J H <Blu-J@users.noreply.github.com> * fix middleware * wip * wip: Fix the auth * wip * continue service refactor * feature: Service get_config * feat: Action * wip: Fighting the great fight against the borrow checker * wip: Remove an error in a file that I just need to deel with later * chore: Add in some more lifetime stuff to the services * wip: Install fix on lifetime * cleanup * wip: Deal with the borrow later * more cleanup * resolve borrowchecker errors * wip(feat): add in the handler for the socket, for now * wip(feat): Update the service_effect_handler::action * chore: Add in the changes to make sure the from_service goes to context * chore: Change the * refactor service map * fix references to service map * fill out restore * wip: Before I work on the store stuff * fix backup module * handle some warnings * feat: add in the ui components on the rust side * feature: Update the procedures * chore: Update the js side of the main and a few of the others * chore: Update the rpc listener to match the persistant container * wip: Working on updating some things to have a better name * wip(feat): Try and get the rpc to return the correct shape? * lxc wip * wip(feat): Try and get the rpc to return the correct shape? * build for container runtime wip * remove container-init * fix build * fix error * chore: Update to work I suppose * lxc wip * remove docker module and feature * download alpine squashfs automatically * overlays effect Co-authored-by: Jade <Blu-J@users.noreply.github.com> * chore: Add the overlay effect * feat: Add the mounter in the main * chore: Convert to use the mounts, still need to work with the sandbox * install fixes * fix ssl * fixes from testing * implement tmpfile for upload * wip * misc fixes * cleanup * cleanup * better progress reporting * progress for sideload * return real guid * add devmode script * fix lxc rootfs path * fix percentage bar * fix progress bar styling * fix build for unstable * tweaks * label progress * tweaks * update progress more often * make symlink in rpc_client * make socket dir * fix parent path * add start-cli to container * add echo and gitInfo commands * wip: Add the init + errors * chore: Add in the exit effect for the system * chore: Change the type to null for failure to parse * move sigterm timeout to stopping status * update order * chore: Update the return type * remove dbg * change the map error * chore: Update the thing to capture id * chore add some life changes * chore: Update the loging * chore: Update the package to run module * us From for RpcError * chore: Update to use import instead * chore: update * chore: Use require for the backup * fix a default * update the type that is wrong * chore: Update the type of the manifest * chore: Update to make null * only symlink if not exists * get rid of double result * better debug info for ErrorCollection * chore: Update effects * chore: fix * mount assets and volumes * add exec instead of spawn * fix mounting in image * fix overlay mounts Co-authored-by: Jade <Blu-J@users.noreply.github.com> * misc fixes * feat: Fix two * fix: systemForEmbassy main * chore: Fix small part of main loop * chore: Modify the bundle * merge * fixMain loop" * move tsc to makefile * chore: Update the return types of the health check * fix client * chore: Convert the todo to use tsmatches * add in the fixes for the seen and create the hack to allow demo * chore: Update to include the systemForStartOs * chore UPdate to the latest types from the expected outout * fixes * fix typo * Don't emit if failure on tsc * wip Co-authored-by: Jade <Blu-J@users.noreply.github.com> * add s9pk api * add inspection * add inspect manifest * newline after display serializable * fix squashfs in image name * edit manifest Co-authored-by: Jade <Blu-J@users.noreply.github.com> * wait for response on repl * ignore sig for now * ignore sig for now * re-enable sig verification * fix * wip * env and chroot * add profiling logs * set uid & gid in squashfs to 100000 * set uid of sqfs to 100000 * fix mksquashfs args * add env to compat * fix * re-add docker feature flag * fix docker output format being stupid * here be dragons * chore: Add in the cross compiling for something * fix npm link * extract logs from container on exit * chore: Update for testing * add log capture to drop trait * chore: add in the modifications that I make * chore: Update small things for no updates * chore: Update the types of something * chore: Make main not complain * idmapped mounts * idmapped volumes * re-enable kiosk * chore: Add in some logging for the new system * bring in start-sdk * remove avahi * chore: Update the deps * switch to musl * chore: Update the version of prettier * chore: Organize' * chore: Update some of the headers back to the standard of fetch * fix musl build * fix idmapped mounts * fix cross build * use cross compiler for correct arch * feat: Add in the faked ssl stuff for the effects * @dr_bonez Did a solution here * chore: Something that DrBonez * chore: up * wip: We have a working server!!! * wip * uninstall * wip * tes --------- Co-authored-by: J H <dragondef@gmail.com> Co-authored-by: J H <Blu-J@users.noreply.github.com> Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com>
2026-03-30 04:01:58 +00:00 · 2024-02-17 11:14:14 -07:00
parent 65009e2f69
commit fab13db4b4
326 changed files with 31708 additions and 13987 deletions
--- a/core/models/src/errors.rs
+++ b/core/models/src/errors.rs
@@ -1,14 +1,19 @@
-use std::fmt::Display;
+use std::fmt::{Debug, Display};

 use color_eyre::eyre::eyre;
+use num_enum::TryFromPrimitive;
 use patch_db::Revision;
 use rpc_toolkit::hyper::http::uri::InvalidUri;
 use rpc_toolkit::reqwest;
-use rpc_toolkit::yajrc::RpcError;
+use rpc_toolkit::yajrc::{
+    RpcError, INVALID_PARAMS_ERROR, INVALID_REQUEST_ERROR, METHOD_NOT_FOUND_ERROR, PARSE_ERROR,
+};
+use serde::{Deserialize, Serialize};

 use crate::InvalidId;

-#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, TryFromPrimitive)]
+#[repr(i32)]
 pub enum ErrorKind {
    Unknown = 1,
    Filesystem = 2,
@@ -81,6 +86,8 @@ pub enum ErrorKind {
    CpuSettings = 69,
    Firmware = 70,
    Timeout = 71,
+    Lxc = 72,
+    Cancelled = 73,
 }
 impl ErrorKind {
    pub fn as_str(&self) -> &'static str {
@@ -157,6 +164,8 @@ impl ErrorKind {
            CpuSettings => "CPU Settings Error",
            Firmware => "Firmware Error",
            Timeout => "Timeout Error",
+            Lxc => "LXC Error",
+            Cancelled => "Cancelled",
        }
    }
 }
@@ -186,6 +195,17 @@ impl Error {
            revision: None,
        }
    }
+    pub fn clone_output(&self) -> Self {
+        Error {
+            source: ErrorData {
+                details: format!("{}", self.source),
+                debug: format!("{:?}", self.source),
+            }
+            .into(),
+            kind: self.kind,
+            revision: self.revision.clone(),
+        }
+    }
 }
 impl From<InvalidId> for Error {
    fn from(err: InvalidId) -> Self {
@@ -300,6 +320,53 @@ impl From<patch_db::value::Error> for Error {
    }
 }

+#[derive(Clone, Deserialize, Serialize)]
+pub struct ErrorData {
+    pub details: String,
+    pub debug: String,
+}
+impl Display for ErrorData {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        Display::fmt(&self.details, f)
+    }
+}
+impl Debug for ErrorData {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        Display::fmt(&self.debug, f)
+    }
+}
+impl std::error::Error for ErrorData {}
+impl From<&RpcError> for ErrorData {
+    fn from(value: &RpcError) -> Self {
+        Self {
+            details: value
+                .data
+                .as_ref()
+                .and_then(|d| {
+                    d.as_object()
+                        .and_then(|d| {
+                            d.get("details")
+                                .and_then(|d| d.as_str().map(|s| s.to_owned()))
+                        })
+                        .or_else(|| d.as_str().map(|s| s.to_owned()))
+                })
+                .unwrap_or_else(|| value.message.clone().into_owned()),
+            debug: value
+                .data
+                .as_ref()
+                .and_then(|d| {
+                    d.as_object()
+                        .and_then(|d| {
+                            d.get("debug")
+                                .and_then(|d| d.as_str().map(|s| s.to_owned()))
+                        })
+                        .or_else(|| d.as_str().map(|s| s.to_owned()))
+                })
+                .unwrap_or_else(|| value.message.clone().into_owned()),
+        }
+    }
+}
+
 impl From<Error> for RpcError {
    fn from(e: Error) -> Self {
        let mut data_object = serde_json::Map::with_capacity(3);
@@ -318,10 +385,40 @@ impl From<Error> for RpcError {
        RpcError {
            code: e.kind as i32,
            message: e.kind.as_str().into(),
-            data: Some(data_object.into()),
+            data: Some(
+                match serde_json::to_value(&ErrorData {
+                    details: format!("{}", e.source),
+                    debug: format!("{:?}", e.source),
+                }) {
+                    Ok(a) => a,
+                    Err(e) => {
+                        tracing::warn!("Error serializing revision for Error object: {}", e);
+                        serde_json::Value::Null
+                    }
+                },
+            ),
        }
    }
 }
+impl From<RpcError> for Error {
+    fn from(e: RpcError) -> Self {
+        Error::new(
+            ErrorData::from(&e),
+            if let Ok(kind) = e.code.try_into() {
+                kind
+            } else if e.code == METHOD_NOT_FOUND_ERROR.code {
+                ErrorKind::NotFound
+            } else if e.code == PARSE_ERROR.code
+                || e.code == INVALID_PARAMS_ERROR.code
+                || e.code == INVALID_REQUEST_ERROR.code
+            {
+                ErrorKind::Deserialization
+            } else {
+                ErrorKind::Unknown
+            },
+        )
+    }
+}

 #[derive(Debug, Default)]
 pub struct ErrorCollection(Vec<Error>);
@@ -377,10 +474,7 @@ where
    Self: Sized,
 {
    fn with_kind(self, kind: ErrorKind) -> Result<T, Error>;
-    fn with_ctx<F: FnOnce(&E) -> (ErrorKind, D), D: Display + Send + Sync + 'static>(
-        self,
-        f: F,
-    ) -> Result<T, Error>;
+    fn with_ctx<F: FnOnce(&E) -> (ErrorKind, D), D: Display>(self, f: F) -> Result<T, Error>;
 }
 impl<T, E> ResultExt<T, E> for Result<T, E>
 where
@@ -394,10 +488,7 @@ where
        })
    }

-    fn with_ctx<F: FnOnce(&E) -> (ErrorKind, D), D: Display + Send + Sync + 'static>(
-        self,
-        f: F,
-    ) -> Result<T, Error> {
+    fn with_ctx<F: FnOnce(&E) -> (ErrorKind, D), D: Display>(self, f: F) -> Result<T, Error> {
        self.map_err(|e| {
            let (kind, ctx) = f(&e);
            let source = color_eyre::eyre::Error::from(e);
@@ -411,6 +502,29 @@ where
        })
    }
 }
+impl<T> ResultExt<T, Error> for Result<T, Error> {
+    fn with_kind(self, kind: ErrorKind) -> Result<T, Error> {
+        self.map_err(|e| Error {
+            source: e.source,
+            kind,
+            revision: e.revision,
+        })
+    }
+
+    fn with_ctx<F: FnOnce(&Error) -> (ErrorKind, D), D: Display>(self, f: F) -> Result<T, Error> {
+        self.map_err(|e| {
+            let (kind, ctx) = f(&e);
+            let source = e.source;
+            let ctx = format!("{}: {}", ctx, source);
+            let source = source.wrap_err(ctx);
+            Error {
+                kind,
+                source,
+                revision: e.revision,
+            }
+        })
+    }
+}

 pub trait OptionExt<T>
 where
--- a/core/models/src/id/image.rs
+++ b/core/models/src/id/image.rs
@@ -1,4 +1,5 @@
 use std::fmt::Debug;
+use std::path::Path;
 use std::str::FromStr;

 use serde::{Deserialize, Deserializer, Serialize};
@@ -7,6 +8,11 @@ use crate::{Id, InvalidId, PackageId, Version};

 #[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize)]
 pub struct ImageId(Id);
+impl AsRef<Path> for ImageId {
+    fn as_ref(&self) -> &Path {
+        self.0.as_ref().as_ref()
+    }
+}
 impl std::fmt::Display for ImageId {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        write!(f, "{}", &self.0)
--- a/core/models/src/procedure_name.rs
+++ b/core/models/src/procedure_name.rs
@@ -4,54 +4,37 @@ use crate::{ActionId, HealthCheckId, PackageId};

 #[derive(Debug, Clone, Serialize, Deserialize)]
 pub enum ProcedureName {
-    Main, // Usually just run container
-    CreateBackup,
-    RestoreBackup,
+    StartMain,
+    StopMain,
    GetConfig,
    SetConfig,
-    Migration,
-    Properties,
-    LongRunning,
-    Check(PackageId),
-    AutoConfig(PackageId),
-    Health(HealthCheckId),
-    Action(ActionId),
-    Signal,
+    CreateBackup,
+    RestoreBackup,
+    ActionMetadata,
+    RunAction(ActionId),
+    GetAction(ActionId),
+    QueryDependency(ActionId),
+    UpdateDependency(ActionId),
+    Init,
+    Uninit,
 }

 impl ProcedureName {
-    pub fn docker_name(&self) -> Option<String> {
+    pub fn js_function_name(&self) -> String {
        match self {
-            ProcedureName::Main => None,
-            ProcedureName::LongRunning => None,
-            ProcedureName::CreateBackup => Some("CreateBackup".to_string()),
-            ProcedureName::RestoreBackup => Some("RestoreBackup".to_string()),
-            ProcedureName::GetConfig => Some("GetConfig".to_string()),
-            ProcedureName::SetConfig => Some("SetConfig".to_string()),
-            ProcedureName::Migration => Some("Migration".to_string()),
-            ProcedureName::Properties => Some(format!("Properties-{}", rand::random::<u64>())),
-            ProcedureName::Health(id) => Some(format!("{}Health", id)),
-            ProcedureName::Action(id) => Some(format!("{}Action", id)),
-            ProcedureName::Check(_) => None,
-            ProcedureName::AutoConfig(_) => None,
-            ProcedureName::Signal => None,
-        }
-    }
-    pub fn js_function_name(&self) -> Option<String> {
-        match self {
-            ProcedureName::Main => Some("/main".to_string()),
-            ProcedureName::LongRunning => None,
-            ProcedureName::CreateBackup => Some("/createBackup".to_string()),
-            ProcedureName::RestoreBackup => Some("/restoreBackup".to_string()),
-            ProcedureName::GetConfig => Some("/getConfig".to_string()),
-            ProcedureName::SetConfig => Some("/setConfig".to_string()),
-            ProcedureName::Migration => Some("/migration".to_string()),
-            ProcedureName::Properties => Some("/properties".to_string()),
-            ProcedureName::Health(id) => Some(format!("/health/{}", id)),
-            ProcedureName::Action(id) => Some(format!("/action/{}", id)),
-            ProcedureName::Check(id) => Some(format!("/dependencies/{}/check", id)),
-            ProcedureName::AutoConfig(id) => Some(format!("/dependencies/{}/autoConfigure", id)),
-            ProcedureName::Signal => Some("/handleSignal".to_string()),
+            ProcedureName::Init => "/init".to_string(),
+            ProcedureName::Uninit => "/uninit".to_string(),
+            ProcedureName::StartMain => "/main/start".to_string(),
+            ProcedureName::StopMain => "/main/stop".to_string(),
+            ProcedureName::SetConfig => "/config/set".to_string(),
+            ProcedureName::GetConfig => "/config/get".to_string(),
+            ProcedureName::CreateBackup => "/backup/create".to_string(),
+            ProcedureName::RestoreBackup => "/backup/restore".to_string(),
+            ProcedureName::ActionMetadata => "/actions/metadata".to_string(),
+            ProcedureName::RunAction(id) => format!("/actions/{}/run", id),
+            ProcedureName::GetAction(id) => format!("/actions/{}/get", id),
+            ProcedureName::QueryDependency(id) => format!("/dependencies/{}/query", id),
+            ProcedureName::UpdateDependency(id) => format!("/dependencies/{}/update", id),
        }
    }
 }