Feature/lxc container runtime (#2514)

* wip: static-server errors

* wip: fix wifi

* wip: Fix the service_effects

* wip: Fix cors in the middleware

* wip(chore): Auth clean up the lint.

* wip(fix): Vhost

* wip: continue manager refactor

Co-authored-by: J H <Blu-J@users.noreply.github.com>

* wip: service manager refactor

* wip: Some fixes

* wip(fix): Fix the lib.rs

* wip

* wip(fix): Logs

* wip: bins

* wip(innspect): Add in the inspect

* wip: config

* wip(fix): Diagnostic

* wip(fix): Dependencies

* wip: context

* wip(fix) Sorta auth

* wip: warnings

* wip(fix): registry/admin

* wip(fix) marketplace

* wip(fix) Some more converted and fixed with the linter and config

* wip: Working on the static server

* wip(fix)static server

* wip: Remove some asynnc

* wip: Something about the request and regular rpc

* wip: gut install

Co-authored-by: J H <Blu-J@users.noreply.github.com>

* wip: Convert the static server into the new system

* wip delete file

* test

* wip(fix) vhost does not need the with safe defaults

* wip: Adding in the wifi

* wip: Fix the developer and the verify

* wip: new install flow

Co-authored-by: J H <Blu-J@users.noreply.github.com>

* fix middleware

* wip

* wip: Fix the auth

* wip

* continue service refactor

* feature: Service get_config

* feat: Action

* wip: Fighting the great fight against the borrow checker

* wip: Remove an error in a file that I just need to deel with later

* chore: Add in some more lifetime stuff to the services

* wip: Install fix on lifetime

* cleanup

* wip: Deal with the borrow later

* more cleanup

* resolve borrowchecker errors

* wip(feat): add in the handler for the socket, for now

* wip(feat): Update the service_effect_handler::action

* chore: Add in the changes to make sure the from_service goes to context

* chore: Change the

* refactor service map

* fix references to service map

* fill out restore

* wip: Before I work on the store stuff

* fix backup module

* handle some warnings

* feat: add in the ui components on the rust side

* feature: Update the procedures

* chore: Update the js side of the main and a few of the others

* chore: Update the rpc listener to match the persistant container

* wip: Working on updating some things to have a better name

* wip(feat): Try and get the rpc to return the correct shape?

* lxc wip

* wip(feat): Try and get the rpc to return the correct shape?

* build for container runtime wip

* remove container-init

* fix build

* fix error

* chore: Update to work I suppose

* lxc wip

* remove docker module and feature

* download alpine squashfs automatically

* overlays effect

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* chore: Add the overlay effect

* feat: Add the mounter in the main

* chore: Convert to use the mounts, still need to work with the sandbox

* install fixes

* fix ssl

* fixes from testing

* implement tmpfile for upload

* wip

* misc fixes

* cleanup

* cleanup

* better progress reporting

* progress for sideload

* return real guid

* add devmode script

* fix lxc rootfs path

* fix percentage bar

* fix progress bar styling

* fix build for unstable

* tweaks

* label progress

* tweaks

* update progress more often

* make symlink in rpc_client

* make socket dir

* fix parent path

* add start-cli to container

* add echo and gitInfo commands

* wip: Add the init + errors

* chore: Add in the exit effect for the system

* chore: Change the type to null for failure to parse

* move sigterm timeout to stopping status

* update order

* chore: Update the return type

* remove dbg

* change the map error

* chore: Update the thing to capture id

* chore add some life changes

* chore: Update the loging

* chore: Update the package to run module

* us From for RpcError

* chore: Update to use import instead

* chore: update

* chore: Use require for the backup

* fix a default

* update the type that is wrong

* chore: Update the type of the manifest

* chore: Update to make null

* only symlink if not exists

* get rid of double result

* better debug info for ErrorCollection

* chore: Update effects

* chore: fix

* mount assets and volumes

* add exec instead of spawn

* fix mounting in image

* fix overlay mounts

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* misc fixes

* feat: Fix two

* fix: systemForEmbassy main

* chore: Fix small part of main loop

* chore: Modify the bundle

* merge

* fixMain loop"

* move tsc to makefile

* chore: Update the return types of the health check

* fix client

* chore: Convert the todo to use tsmatches

* add in the fixes for the seen and create the hack to allow demo

* chore: Update to include the systemForStartOs

* chore UPdate to the latest types from the expected outout

* fixes

* fix typo

* Don't emit if failure on tsc

* wip

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* add s9pk api

* add inspection

* add inspect manifest

* newline after display serializable

* fix squashfs in image name

* edit manifest

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* wait for response on repl

* ignore sig for now

* ignore sig for now

* re-enable sig verification

* fix

* wip

* env and chroot

* add profiling logs

* set uid & gid in squashfs to 100000

* set uid of sqfs to 100000

* fix mksquashfs args

* add env to compat

* fix

* re-add docker feature flag

* fix docker output format being stupid

* here be dragons

* chore: Add in the cross compiling for something

* fix npm link

* extract logs from container on exit

* chore: Update for testing

* add log capture to drop trait

* chore: add in the modifications that I make

* chore: Update small things for no updates

* chore: Update the types of something

* chore: Make main not complain

* idmapped mounts

* idmapped volumes

* re-enable kiosk

* chore: Add in some logging for the new system

* bring in start-sdk

* remove avahi

* chore: Update the deps

* switch to musl

* chore: Update the version of prettier

* chore: Organize'

* chore: Update some of the headers back to the standard of fetch

* fix musl build

* fix idmapped mounts

* fix cross build

* use cross compiler for correct arch

* feat: Add in the faked ssl stuff for the effects

* @dr_bonez Did a solution here

* chore: Something that DrBonez

* chore: up

* wip: We have a working server!!!

* wip

* uninstall

* wip

* tes

---------

Co-authored-by: J H <dragondef@gmail.com>
Co-authored-by: J H <Blu-J@users.noreply.github.com>
Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com>

core/startos/src/context/cli.rs

@@ -1,43 +1,37 @@
 use std::fs::File;
 use std::io::BufReader;
-use std::net::Ipv4Addr;
 use std::path::{Path, PathBuf};
 use std::sync::Arc;
 
-use clap::ArgMatches;
-use color_eyre::eyre::eyre;
 use cookie_store::{CookieStore, RawCookie};
 use josekit::jwk::Jwk;
+use once_cell::sync::OnceCell;
 use reqwest::Proxy;
 use reqwest_cookie_store::CookieStoreMutex;
 use rpc_toolkit::reqwest::{Client, Url};
-use rpc_toolkit::url::Host;
-use rpc_toolkit::Context;
-use serde::Deserialize;
+use rpc_toolkit::yajrc::RpcError;
+use rpc_toolkit::{call_remote_http, CallRemote, Context};
+use tokio::net::TcpStream;
+use tokio::runtime::Runtime;
+use tokio_tungstenite::{MaybeTlsStream, WebSocketStream};
 use tracing::instrument;
 
 use super::setup::CURRENT_SECRET;
+use crate::context::config::{local_config_path, ClientConfig};
+use crate::core::rpc_continuations::RequestGuid;
 use crate::middleware::auth::LOCAL_AUTH_COOKIE_PATH;
-use crate::util::config::{load_config_from_paths, local_config_path};
-use crate::ResultExt;
-
-#[derive(Debug, Default, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-pub struct CliContextConfig {
-    pub host: Option<Url>,
-    #[serde(deserialize_with = "crate::util::serde::deserialize_from_str_opt")]
-    #[serde(default)]
-    pub proxy: Option<Url>,
-    pub cookie_path: Option<PathBuf>,
-}
+use crate::prelude::*;
 
 #[derive(Debug)]
 pub struct CliContextSeed {
+    pub runtime: OnceCell<Runtime>,
     pub base_url: Url,
     pub rpc_url: Url,
     pub client: Client,
     pub cookie_store: Arc<CookieStoreMutex>,
     pub cookie_path: PathBuf,
+    pub developer_key_path: PathBuf,
+    pub developer_key: OnceCell<ed25519_dalek::SigningKey>,
 }
 impl Drop for CliContextSeed {
     fn drop(&mut self) {
@@ -60,42 +54,22 @@ impl Drop for CliContextSeed {
     }
 }
 
-const DEFAULT_HOST: Host<&'static str> = Host::Ipv4(Ipv4Addr::new(127, 0, 0, 1));
-const DEFAULT_PORT: u16 = 5959;
-
 #[derive(Debug, Clone)]
 pub struct CliContext(Arc<CliContextSeed>);
 impl CliContext {
     /// BLOCKING
     #[instrument(skip_all)]
-    pub fn init(matches: &ArgMatches) -> Result<Self, crate::Error> {
-        let local_config_path = local_config_path();
-        let base: CliContextConfig = load_config_from_paths(
-            matches
-                .values_of("config")
-                .into_iter()
-                .flatten()
-                .map(|p| Path::new(p))
-                .chain(local_config_path.as_deref().into_iter())
-                .chain(std::iter::once(Path::new(crate::util::config::CONFIG_PATH))),
-        )?;
-        let mut url = if let Some(host) = matches.value_of("host") {
-            host.parse()?
-        } else if let Some(host) = base.host {
+    pub fn init(config: ClientConfig) -> Result<Self, Error> {
+        let mut url = if let Some(host) = config.host {
             host
         } else {
             "http://localhost".parse()?
         };
-        let proxy = if let Some(proxy) = matches.value_of("proxy") {
-            Some(proxy.parse()?)
-        } else {
-            base.proxy
-        };
 
-        let cookie_path = base.cookie_path.unwrap_or_else(|| {
-            local_config_path
+        let cookie_path = config.cookie_path.unwrap_or_else(|| {
+            local_config_path()
                 .as_deref()
-                .unwrap_or_else(|| Path::new(crate::util::config::CONFIG_PATH))
+                .unwrap_or_else(|| Path::new(super::config::CONFIG_PATH))
                 .parent()
                 .unwrap_or(Path::new("/"))
                 .join(".cookies.json")
@@ -120,6 +94,7 @@ impl CliContext {
         }));
 
         Ok(CliContext(Arc::new(CliContextSeed {
+            runtime: OnceCell::new(),
             base_url: url.clone(),
             rpc_url: {
                 url.path_segments_mut()
@@ -131,7 +106,7 @@ impl CliContext {
             },
             client: {
                 let mut builder = Client::builder().cookie_provider(cookie_store.clone());
-                if let Some(proxy) = proxy {
+                if let Some(proxy) = config.proxy {
                     builder =
                         builder.proxy(Proxy::all(proxy).with_kind(crate::ErrorKind::ParseUrl)?)
                 }
@@ -139,8 +114,90 @@ impl CliContext {
             },
             cookie_store,
             cookie_path,
+            developer_key_path: config.developer_key_path.unwrap_or_else(|| {
+                local_config_path()
+                    .as_deref()
+                    .unwrap_or_else(|| Path::new(super::config::CONFIG_PATH))
+                    .parent()
+                    .unwrap_or(Path::new("/"))
+                    .join("developer.key.pem")
+            }),
+            developer_key: OnceCell::new(),
         })))
     }
+
+    /// BLOCKING
+    #[instrument(skip_all)]
+    pub fn developer_key(&self) -> Result<&ed25519_dalek::SigningKey, Error> {
+        self.developer_key.get_or_try_init(|| {
+            if !self.developer_key_path.exists() {
+                return Err(Error::new(eyre!("Developer Key does not exist! Please run `start-cli init` before running this command."), crate::ErrorKind::Uninitialized));
+            }
+            let pair = <ed25519::KeypairBytes as ed25519::pkcs8::DecodePrivateKey>::from_pkcs8_pem(
+                &std::fs::read_to_string(&self.developer_key_path)?,
+            )
+            .with_kind(crate::ErrorKind::Pem)?;
+            let secret = ed25519_dalek::SecretKey::try_from(&pair.secret_key[..]).map_err(|_| {
+                Error::new(
+                    eyre!("pkcs8 key is of incorrect length"),
+                    ErrorKind::OpenSsl,
+                )
+            })?;
+            Ok(secret.into())
+        })
+    }
+
+    pub async fn ws_continuation(
+        &self,
+        guid: RequestGuid,
+    ) -> Result<WebSocketStream<MaybeTlsStream<TcpStream>>, Error> {
+        let mut url = self.base_url.clone();
+        let ws_scheme = match url.scheme() {
+            "https" => "wss",
+            "http" => "ws",
+            _ => {
+                return Err(Error::new(
+                    eyre!("Cannot parse scheme from base URL"),
+                    crate::ErrorKind::ParseUrl,
+                )
+                .into())
+            }
+        };
+        url.set_scheme(ws_scheme)
+            .map_err(|_| Error::new(eyre!("Cannot set URL scheme"), crate::ErrorKind::ParseUrl))?;
+        url.path_segments_mut()
+            .map_err(|_| eyre!("Url cannot be base"))
+            .with_kind(crate::ErrorKind::ParseUrl)?
+            .push("ws")
+            .push("rpc")
+            .push(guid.as_ref());
+        let (stream, _) =
+                // base_url is "http://127.0.0.1/", with a trailing slash, so we don't put a leading slash in this path:
+                tokio_tungstenite::connect_async(url).await.with_kind(ErrorKind::Network)?;
+        Ok(stream)
+    }
+
+    pub async fn rest_continuation(
+        &self,
+        guid: RequestGuid,
+        body: reqwest::Body,
+        headers: reqwest::header::HeaderMap,
+    ) -> Result<reqwest::Response, Error> {
+        let mut url = self.base_url.clone();
+        url.path_segments_mut()
+            .map_err(|_| eyre!("Url cannot be base"))
+            .with_kind(crate::ErrorKind::ParseUrl)?
+            .push("rest")
+            .push("rpc")
+            .push(guid.as_ref());
+        self.client
+            .post(url)
+            .headers(headers)
+            .body(body)
+            .send()
+            .await
+            .with_kind(ErrorKind::Network)
+    }
 }
 impl AsRef<Jwk> for CliContext {
     fn as_ref(&self) -> &Jwk {
@@ -154,32 +211,33 @@ impl std::ops::Deref for CliContext {
     }
 }
 impl Context for CliContext {
-    fn protocol(&self) -> &str {
-        self.0.base_url.scheme()
-    }
-    fn host(&self) -> Host<&str> {
-        self.0.base_url.host().unwrap_or(DEFAULT_HOST)
-    }
-    fn port(&self) -> u16 {
-        self.0.base_url.port().unwrap_or(DEFAULT_PORT)
-    }
-    fn path(&self) -> &str {
-        self.0.rpc_url.path()
-    }
-    fn url(&self) -> Url {
-        self.0.rpc_url.clone()
-    }
-    fn client(&self) -> &Client {
-        &self.0.client
+    fn runtime(&self) -> tokio::runtime::Handle {
+        self.runtime
+            .get_or_init(|| {
+                tokio::runtime::Builder::new_multi_thread()
+                    .enable_all()
+                    .build()
+                    .unwrap()
+            })
+            .handle()
+            .clone()
     }
 }
-/// When we had an empty proxy the system wasn't working like it used to, which allowed empty proxy
+#[async_trait::async_trait]
+impl CallRemote for CliContext {
+    async fn call_remote(&self, method: &str, params: Value) -> Result<Value, RpcError> {
+        call_remote_http(&self.client, self.rpc_url.clone(), method, params).await
+    }
+}
+
 #[test]
-fn test_cli_proxy_empty() {
-    serde_yaml::from_str::<CliContextConfig>(
-        "
-        bind_rpc:
-    ",
-    )
-    .unwrap();
+fn test() {
+    let ctx = CliContext::init(ClientConfig::default()).unwrap();
+    ctx.runtime().block_on(async {
+        reqwest::Client::new()
+            .get("http://example.com")
+            .send()
+            .await
+            .unwrap();
+    });
 }

core/startos/src/context/config.rs

@@ -0,0 +1,175 @@
+use std::fs::File;
+use std::net::SocketAddr;
+use std::path::{Path, PathBuf};
+
+use clap::Parser;
+use patch_db::json_ptr::JsonPointer;
+use reqwest::Url;
+use serde::de::DeserializeOwned;
+use serde::{Deserialize, Serialize};
+use sqlx::postgres::PgConnectOptions;
+use sqlx::PgPool;
+
+use crate::account::AccountInfo;
+use crate::db::model::Database;
+use crate::disk::OsPartitionInfo;
+use crate::init::init_postgres;
+use crate::prelude::*;
+use crate::util::serde::IoFormat;
+
+pub const DEVICE_CONFIG_PATH: &str = "/media/embassy/config/config.yaml"; // "/media/startos/config/config.yaml";
+pub const CONFIG_PATH: &str = "/etc/startos/config.yaml";
+pub const CONFIG_PATH_LOCAL: &str = ".startos/config.yaml";
+
+pub fn local_config_path() -> Option<PathBuf> {
+    if let Ok(home) = std::env::var("HOME") {
+        Some(Path::new(&home).join(CONFIG_PATH_LOCAL))
+    } else {
+        None
+    }
+}
+
+pub trait ContextConfig: DeserializeOwned + Default {
+    fn next(&mut self) -> Option<PathBuf>;
+    fn merge_with(&mut self, other: Self);
+    fn from_path(path: impl AsRef<Path>) -> Result<Self, Error> {
+        let format: IoFormat = path
+            .as_ref()
+            .extension()
+            .and_then(|s| s.to_str())
+            .map(|f| f.parse())
+            .transpose()?
+            .unwrap_or_default();
+        format.from_reader(File::open(path)?)
+    }
+    fn load_path_rec(&mut self, path: Option<impl AsRef<Path>>) -> Result<(), Error> {
+        if let Some(path) = path.filter(|p| p.as_ref().exists()) {
+            let mut other = Self::from_path(path)?;
+            let path = other.next();
+            self.merge_with(other);
+            self.load_path_rec(path)?;
+        }
+        Ok(())
+    }
+}
+
+#[derive(Debug, Default, Deserialize, Serialize, Parser)]
+#[serde(rename_all = "kebab-case")]
+#[command(rename_all = "kebab-case")]
+pub struct ClientConfig {
+    #[arg(short = 'c', long = "config")]
+    pub config: Option<PathBuf>,
+    #[arg(short = 'h', long = "host")]
+    pub host: Option<Url>,
+    #[arg(short = 'p', long = "proxy")]
+    pub proxy: Option<Url>,
+    #[arg(long = "cookie-path")]
+    pub cookie_path: Option<PathBuf>,
+    #[arg(long = "developer-key-path")]
+    pub developer_key_path: Option<PathBuf>,
+}
+impl ContextConfig for ClientConfig {
+    fn next(&mut self) -> Option<PathBuf> {
+        self.config.take()
+    }
+    fn merge_with(&mut self, other: Self) {
+        self.host = self.host.take().or(other.host);
+        self.proxy = self.proxy.take().or(other.proxy);
+        self.cookie_path = self.cookie_path.take().or(other.cookie_path);
+    }
+}
+impl ClientConfig {
+    pub fn load(mut self) -> Result<Self, Error> {
+        let path = self.next();
+        self.load_path_rec(path)?;
+        self.load_path_rec(local_config_path())?;
+        self.load_path_rec(Some(CONFIG_PATH))?;
+        Ok(self)
+    }
+}
+
+#[derive(Debug, Clone, Default, Deserialize, Serialize, Parser)]
+#[serde(rename_all = "kebab-case")]
+#[command(rename_all = "kebab-case")]
+pub struct ServerConfig {
+    #[arg(short = 'c', long = "config")]
+    pub config: Option<PathBuf>,
+    #[arg(long = "wifi-interface")]
+    pub wifi_interface: Option<String>,
+    #[arg(long = "ethernet-interface")]
+    pub ethernet_interface: Option<String>,
+    #[arg(skip)]
+    pub os_partitions: Option<OsPartitionInfo>,
+    #[arg(long = "bind-rpc")]
+    pub bind_rpc: Option<SocketAddr>,
+    #[arg(long = "tor-control")]
+    pub tor_control: Option<SocketAddr>,
+    #[arg(long = "tor-socks")]
+    pub tor_socks: Option<SocketAddr>,
+    #[arg(long = "dns-bind")]
+    pub dns_bind: Option<Vec<SocketAddr>>,
+    #[arg(long = "revision-cache-size")]
+    pub revision_cache_size: Option<usize>,
+    #[arg(short = 'd', long = "datadir")]
+    pub datadir: Option<PathBuf>,
+    #[arg(long = "disable-encryption")]
+    pub disable_encryption: Option<bool>,
+}
+impl ContextConfig for ServerConfig {
+    fn next(&mut self) -> Option<PathBuf> {
+        self.config.take()
+    }
+    fn merge_with(&mut self, other: Self) {
+        self.wifi_interface = self.wifi_interface.take().or(other.wifi_interface);
+        self.ethernet_interface = self.ethernet_interface.take().or(other.ethernet_interface);
+        self.os_partitions = self.os_partitions.take().or(other.os_partitions);
+        self.bind_rpc = self.bind_rpc.take().or(other.bind_rpc);
+        self.tor_control = self.tor_control.take().or(other.tor_control);
+        self.tor_socks = self.tor_socks.take().or(other.tor_socks);
+        self.dns_bind = self.dns_bind.take().or(other.dns_bind);
+        self.revision_cache_size = self
+            .revision_cache_size
+            .take()
+            .or(other.revision_cache_size);
+        self.datadir = self.datadir.take().or(other.datadir);
+        self.disable_encryption = self.disable_encryption.take().or(other.disable_encryption);
+    }
+}
+
+impl ServerConfig {
+    pub fn load(mut self) -> Result<Self, Error> {
+        let path = self.next();
+        self.load_path_rec(path)?;
+        self.load_path_rec(Some(DEVICE_CONFIG_PATH))?;
+        self.load_path_rec(Some(CONFIG_PATH))?;
+        Ok(self)
+    }
+    pub fn datadir(&self) -> &Path {
+        self.datadir
+            .as_deref()
+            .unwrap_or_else(|| Path::new("/embassy-data"))
+    }
+    pub async fn db(&self, account: &AccountInfo) -> Result<PatchDb, Error> {
+        let db_path = self.datadir().join("main").join("embassy.db");
+        let db = PatchDb::open(&db_path)
+            .await
+            .with_ctx(|_| (crate::ErrorKind::Filesystem, db_path.display().to_string()))?;
+        if !db.exists(&<JsonPointer>::default()).await {
+            db.put(&<JsonPointer>::default(), &Database::init(account))
+                .await?;
+        }
+        Ok(db)
+    }
+    #[instrument(skip_all)]
+    pub async fn secret_store(&self) -> Result<PgPool, Error> {
+        init_postgres(self.datadir()).await?;
+        let secret_store =
+            PgPool::connect_with(PgConnectOptions::new().database("secrets").username("root"))
+                .await?;
+        sqlx::migrate!()
+            .run(&secret_store)
+            .await
+            .with_kind(crate::ErrorKind::Database)?;
+        Ok(secret_store)
+    }
+}

core/startos/src/context/diagnostic.rs

@@ -1,47 +1,16 @@
 use std::ops::Deref;
-use std::path::{Path, PathBuf};
+use std::path::PathBuf;
 use std::sync::Arc;
 
 use rpc_toolkit::yajrc::RpcError;
 use rpc_toolkit::Context;
-use serde::Deserialize;
 use tokio::sync::broadcast::Sender;
 use tracing::instrument;
 
+use crate::context::config::ServerConfig;
 use crate::shutdown::Shutdown;
-use crate::util::config::load_config_from_paths;
 use crate::Error;
 
-#[derive(Debug, Default, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-pub struct DiagnosticContextConfig {
-    pub datadir: Option<PathBuf>,
-}
-impl DiagnosticContextConfig {
-    #[instrument(skip_all)]
-    pub async fn load<P: AsRef<Path> + Send + 'static>(path: Option<P>) -> Result<Self, Error> {
-        tokio::task::spawn_blocking(move || {
-            load_config_from_paths(
-                path.as_ref()
-                    .into_iter()
-                    .map(|p| p.as_ref())
-                    .chain(std::iter::once(Path::new(
-                        crate::util::config::DEVICE_CONFIG_PATH,
-                    )))
-                    .chain(std::iter::once(Path::new(crate::util::config::CONFIG_PATH))),
-            )
-        })
-        .await
-        .unwrap()
-    }
-
-    pub fn datadir(&self) -> &Path {
-        self.datadir
-            .as_deref()
-            .unwrap_or_else(|| Path::new("/embassy-data"))
-    }
-}
-
 pub struct DiagnosticContextSeed {
     pub datadir: PathBuf,
     pub shutdown: Sender<Option<Shutdown>>,
@@ -53,20 +22,18 @@ pub struct DiagnosticContextSeed {
 pub struct DiagnosticContext(Arc<DiagnosticContextSeed>);
 impl DiagnosticContext {
     #[instrument(skip_all)]
-    pub async fn init<P: AsRef<Path> + Send + 'static>(
-        path: Option<P>,
+    pub fn init(
+        config: &ServerConfig,
         disk_guid: Option<Arc<String>>,
         error: Error,
     ) -> Result<Self, Error> {
         tracing::error!("Error: {}: Starting diagnostic UI", error);
         tracing::debug!("{:?}", error);
 
-        let cfg = DiagnosticContextConfig::load(path).await?;
-
         let (shutdown, _) = tokio::sync::broadcast::channel(1);
 
         Ok(Self(Arc::new(DiagnosticContextSeed {
-            datadir: cfg.datadir().to_owned(),
+            datadir: config.datadir().to_owned(),
             shutdown,
             disk_guid,
             error: Arc::new(error.into()),

core/startos/src/context/install.rs

@@ -1,35 +1,13 @@
 use std::ops::Deref;
-use std::path::Path;
 use std::sync::Arc;
 
 use rpc_toolkit::Context;
-use serde::Deserialize;
 use tokio::sync::broadcast::Sender;
 use tracing::instrument;
 
 use crate::net::utils::find_eth_iface;
-use crate::util::config::load_config_from_paths;
 use crate::Error;
 
-#[derive(Debug, Default, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-pub struct InstallContextConfig {}
-impl InstallContextConfig {
-    #[instrument(skip_all)]
-    pub async fn load<P: AsRef<Path> + Send + 'static>(path: Option<P>) -> Result<Self, Error> {
-        tokio::task::spawn_blocking(move || {
-            load_config_from_paths(
-                path.as_ref()
-                    .into_iter()
-                    .map(|p| p.as_ref())
-                    .chain(std::iter::once(Path::new(crate::util::config::CONFIG_PATH))),
-            )
-        })
-        .await
-        .unwrap()
-    }
-}
-
 pub struct InstallContextSeed {
     pub ethernet_interface: String,
     pub shutdown: Sender<()>,
@@ -39,8 +17,7 @@ pub struct InstallContextSeed {
 pub struct InstallContext(Arc<InstallContextSeed>);
 impl InstallContext {
     #[instrument(skip_all)]
-    pub async fn init<P: AsRef<Path> + Send + 'static>(path: Option<P>) -> Result<Self, Error> {
-        let _cfg = InstallContextConfig::load(path.as_ref().map(|p| p.as_ref().to_owned())).await?;
+    pub async fn init() -> Result<Self, Error> {
         let (shutdown, _) = tokio::sync::broadcast::channel(1);
         Ok(Self(Arc::new(InstallContextSeed {
             ethernet_interface: find_eth_iface().await?,

core/startos/src/context/mod.rs

@@ -1,44 +1,12 @@
 pub mod cli;
+pub mod config;
 pub mod diagnostic;
 pub mod install;
 pub mod rpc;
-pub mod sdk;
 pub mod setup;
 
 pub use cli::CliContext;
 pub use diagnostic::DiagnosticContext;
 pub use install::InstallContext;
 pub use rpc::RpcContext;
-pub use sdk::SdkContext;
 pub use setup::SetupContext;
-
-impl From<CliContext> for () {
-    fn from(_: CliContext) -> Self {
-        ()
-    }
-}
-impl From<DiagnosticContext> for () {
-    fn from(_: DiagnosticContext) -> Self {
-        ()
-    }
-}
-impl From<RpcContext> for () {
-    fn from(_: RpcContext) -> Self {
-        ()
-    }
-}
-impl From<SdkContext> for () {
-    fn from(_: SdkContext) -> Self {
-        ()
-    }
-}
-impl From<SetupContext> for () {
-    fn from(_: SetupContext) -> Self {
-        ()
-    }
-}
-impl From<InstallContext> for () {
-    fn from(_: InstallContext) -> Self {
-        ()
-    }
-}

core/startos/src/context/rpc.rs

@@ -1,19 +1,16 @@
 use std::collections::BTreeMap;
 use std::net::{Ipv4Addr, SocketAddr, SocketAddrV4};
 use std::ops::Deref;
-use std::path::{Path, PathBuf};
+use std::path::PathBuf;
 use std::sync::atomic::{AtomicBool, Ordering};
 use std::sync::Arc;
 use std::time::Duration;
 
-use helpers::to_tmp_path;
+use imbl_value::InternedString;
 use josekit::jwk::Jwk;
-use patch_db::json_ptr::JsonPointer;
 use patch_db::PatchDb;
-use reqwest::{Client, Proxy, Url};
+use reqwest::{Client, Proxy};
 use rpc_toolkit::Context;
-use serde::Deserialize;
-use sqlx::postgres::PgConnectOptions;
 use sqlx::PgPool;
 use tokio::sync::{broadcast, oneshot, Mutex, RwLock};
 use tokio::time::Instant;
@@ -21,87 +18,26 @@ use tracing::instrument;
 
 use super::setup::CURRENT_SECRET;
 use crate::account::AccountInfo;
-use crate::core::rpc_continuations::{RequestGuid, RestHandler, RpcContinuation};
-use crate::db::model::{CurrentDependents, Database, PackageDataEntryMatchModelRef};
+use crate::context::config::ServerConfig;
+use crate::core::rpc_continuations::{RequestGuid, RestHandler, RpcContinuation, WebSocketHandler};
+use crate::db::model::CurrentDependents;
 use crate::db::prelude::PatchDbExt;
 use crate::dependencies::compute_dependency_config_errs;
 use crate::disk::OsPartitionInfo;
-use crate::init::{check_time_is_synchronized, init_postgres};
-use crate::install::cleanup::{cleanup_failed, uninstall};
-use crate::manager::ManagerMap;
+use crate::init::check_time_is_synchronized;
+use crate::lxc::{LxcContainer, LxcManager};
 use crate::middleware::auth::HashSessionToken;
 use crate::net::net_controller::NetController;
 use crate::net::ssl::{root_ca_start_time, SslManager};
+use crate::net::utils::find_eth_iface;
 use crate::net::wifi::WpaCli;
 use crate::notifications::NotificationManager;
+use crate::prelude::*;
+use crate::service::ServiceMap;
 use crate::shutdown::Shutdown;
 use crate::status::MainStatus;
 use crate::system::get_mem_info;
-use crate::util::config::load_config_from_paths;
 use crate::util::lshw::{lshw, LshwDevice};
-use crate::{Error, ErrorKind, ResultExt};
-
-#[derive(Debug, Default, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-pub struct RpcContextConfig {
-    pub wifi_interface: Option<String>,
-    pub ethernet_interface: String,
-    pub os_partitions: OsPartitionInfo,
-    pub migration_batch_rows: Option<usize>,
-    pub migration_prefetch_rows: Option<usize>,
-    pub bind_rpc: Option<SocketAddr>,
-    pub tor_control: Option<SocketAddr>,
-    pub tor_socks: Option<SocketAddr>,
-    pub dns_bind: Option<Vec<SocketAddr>>,
-    pub revision_cache_size: Option<usize>,
-    pub datadir: Option<PathBuf>,
-    pub log_server: Option<Url>,
-}
-impl RpcContextConfig {
-    pub async fn load<P: AsRef<Path> + Send + 'static>(path: Option<P>) -> Result<Self, Error> {
-        tokio::task::spawn_blocking(move || {
-            load_config_from_paths(
-                path.as_ref()
-                    .into_iter()
-                    .map(|p| p.as_ref())
-                    .chain(std::iter::once(Path::new(
-                        crate::util::config::DEVICE_CONFIG_PATH,
-                    )))
-                    .chain(std::iter::once(Path::new(crate::util::config::CONFIG_PATH))),
-            )
-        })
-        .await
-        .unwrap()
-    }
-    pub fn datadir(&self) -> &Path {
-        self.datadir
-            .as_deref()
-            .unwrap_or_else(|| Path::new("/embassy-data"))
-    }
-    pub async fn db(&self, account: &AccountInfo) -> Result<PatchDb, Error> {
-        let db_path = self.datadir().join("main").join("embassy.db");
-        let db = PatchDb::open(&db_path)
-            .await
-            .with_ctx(|_| (crate::ErrorKind::Filesystem, db_path.display().to_string()))?;
-        if !db.exists(&<JsonPointer>::default()).await {
-            db.put(&<JsonPointer>::default(), &Database::init(account))
-                .await?;
-        }
-        Ok(db)
-    }
-    #[instrument(skip_all)]
-    pub async fn secret_store(&self) -> Result<PgPool, Error> {
-        init_postgres(self.datadir()).await?;
-        let secret_store =
-            PgPool::connect_with(PgConnectOptions::new().database("secrets").username("root"))
-                .await?;
-        sqlx::migrate!()
-            .run(&secret_store)
-            .await
-            .with_kind(crate::ErrorKind::Database)?;
-        Ok(secret_store)
-    }
-}
 
 pub struct RpcContextSeed {
     is_closed: AtomicBool,
@@ -114,11 +50,12 @@ pub struct RpcContextSeed {
     pub secret_store: PgPool,
     pub account: RwLock<AccountInfo>,
     pub net_controller: Arc<NetController>,
-    pub managers: ManagerMap,
+    pub services: ServiceMap,
     pub metrics_cache: RwLock<Option<crate::system::Metrics>>,
     pub shutdown: broadcast::Sender<Option<Shutdown>>,
     pub tor_socks: SocketAddr,
     pub notification_manager: NotificationManager,
+    pub lxc_manager: Arc<LxcManager>,
     pub open_authed_websockets: Mutex<BTreeMap<HashSessionToken, Vec<oneshot::Sender<()>>>>,
     pub rpc_stream_continuations: Mutex<BTreeMap<RequestGuid, RpcContinuation>>,
     pub wifi_manager: Option<Arc<RwLock<WpaCli>>>,
@@ -126,6 +63,11 @@ pub struct RpcContextSeed {
     pub client: Client,
     pub hardware: Hardware,
     pub start_time: Instant,
+    pub dev: Dev,
+}
+
+pub struct Dev {
+    pub lxc: Mutex<BTreeMap<InternedString, LxcContainer>>,
 }
 
 pub struct Hardware {
@@ -137,28 +79,26 @@ pub struct Hardware {
 pub struct RpcContext(Arc<RpcContextSeed>);
 impl RpcContext {
     #[instrument(skip_all)]
-    pub async fn init<P: AsRef<Path> + Send + Sync + 'static>(
-        cfg_path: Option<P>,
-        disk_guid: Arc<String>,
-    ) -> Result<Self, Error> {
-        let base = RpcContextConfig::load(cfg_path).await?;
+    pub async fn init(config: &ServerConfig, disk_guid: Arc<String>) -> Result<Self, Error> {
         tracing::info!("Loaded Config");
-        let tor_proxy = base.tor_socks.unwrap_or(SocketAddr::V4(SocketAddrV4::new(
+        let tor_proxy = config.tor_socks.unwrap_or(SocketAddr::V4(SocketAddrV4::new(
             Ipv4Addr::new(127, 0, 0, 1),
             9050,
         )));
         let (shutdown, _) = tokio::sync::broadcast::channel(1);
-        let secret_store = base.secret_store().await?;
+        let secret_store = config.secret_store().await?;
         tracing::info!("Opened Pg DB");
         let account = AccountInfo::load(&secret_store).await?;
-        let db = base.db(&account).await?;
+        let db = config.db(&account).await?;
         tracing::info!("Opened PatchDB");
         let net_controller = Arc::new(
             NetController::init(
-                base.tor_control
+                config
+                    .tor_control
                     .unwrap_or(SocketAddr::from(([127, 0, 0, 1], 9051))),
                 tor_proxy,
-                base.dns_bind
+                config
+                    .dns_bind
                     .as_deref()
                     .unwrap_or(&[SocketAddr::from(([127, 0, 0, 1], 53))]),
                 SslManager::new(&account, root_ca_start_time().await?)?,
@@ -168,7 +108,7 @@ impl RpcContext {
             .await?,
         );
         tracing::info!("Initialized Net Controller");
-        let managers = ManagerMap::default();
+        let services = ServiceMap::default();
         let metrics_cache = RwLock::<Option<crate::system::Metrics>>::new(None);
         let notification_manager = NotificationManager::new(secret_store.clone());
         tracing::info!("Initialized Notification Manager");
@@ -190,24 +130,35 @@ impl RpcContext {
 
         let seed = Arc::new(RpcContextSeed {
             is_closed: AtomicBool::new(false),
-            datadir: base.datadir().to_path_buf(),
-            os_partitions: base.os_partitions,
-            wifi_interface: base.wifi_interface.clone(),
-            ethernet_interface: base.ethernet_interface,
+            datadir: config.datadir().to_path_buf(),
+            os_partitions: config.os_partitions.clone().ok_or_else(|| {
+                Error::new(
+                    eyre!("OS Partition Information Missing"),
+                    ErrorKind::Filesystem,
+                )
+            })?,
+            wifi_interface: config.wifi_interface.clone(),
+            ethernet_interface: if let Some(eth) = config.ethernet_interface.clone() {
+                eth
+            } else {
+                find_eth_iface().await?
+            },
             disk_guid,
             db,
             secret_store,
             account: RwLock::new(account),
             net_controller,
-            managers,
+            services,
             metrics_cache,
             shutdown,
             tor_socks: tor_proxy,
             notification_manager,
+            lxc_manager: Arc::new(LxcManager::new()),
             open_authed_websockets: Mutex::new(BTreeMap::new()),
             rpc_stream_continuations: Mutex::new(BTreeMap::new()),
-            wifi_manager: base
+            wifi_manager: config
                 .wifi_interface
+                .clone()
                 .map(|i| Arc::new(RwLock::new(WpaCli::init(i)))),
             current_secret: Arc::new(
                 Jwk::generate_ec_key(josekit::jwk::alg::ec::EcCurve::P256).map_err(|e| {
@@ -231,6 +182,9 @@ impl RpcContext {
                 .with_kind(crate::ErrorKind::ParseUrl)?,
             hardware: Hardware { devices, ram },
             start_time: Instant::now(),
+            dev: Dev {
+                lxc: Mutex::new(BTreeMap::new()),
+            },
         });
 
         let res = Self(seed.clone());
@@ -241,7 +195,7 @@ impl RpcContext {
 
     #[instrument(skip_all)]
     pub async fn shutdown(self) -> Result<(), Error> {
-        self.managers.empty().await?;
+        self.services.shutdown_all().await?;
         self.secret_store.close().await;
         self.is_closed.store(true, Ordering::SeqCst);
         tracing::info!("RPC Context is shutdown");
@@ -293,70 +247,11 @@ impl RpcContext {
             })
             .await?;
 
-        let peek = self.db.peek().await;
-
-        for (package_id, package) in peek.as_package_data().as_entries()?.into_iter() {
-            let action = match package.as_match() {
-                PackageDataEntryMatchModelRef::Installing(_)
-                | PackageDataEntryMatchModelRef::Restoring(_)
-                | PackageDataEntryMatchModelRef::Updating(_) => {
-                    cleanup_failed(self, &package_id).await
-                }
-                PackageDataEntryMatchModelRef::Removing(_) => {
-                    uninstall(
-                        self,
-                        self.secret_store.acquire().await?.as_mut(),
-                        &package_id,
-                    )
-                    .await
-                }
-                PackageDataEntryMatchModelRef::Installed(m) => {
-                    let version = m.as_manifest().as_version().clone().de()?;
-                    let volumes = m.as_manifest().as_volumes().de()?;
-                    for (volume_id, volume_info) in &*volumes {
-                        let tmp_path = to_tmp_path(volume_info.path_for(
-                            &self.datadir,
-                            &package_id,
-                            &version,
-                            volume_id,
-                        ))
-                        .with_kind(ErrorKind::Filesystem)?;
-                        if tokio::fs::metadata(&tmp_path).await.is_ok() {
-                            tokio::fs::remove_dir_all(&tmp_path).await?;
-                        }
-                    }
-                    Ok(())
-                }
-                _ => continue,
-            };
-            if let Err(e) = action {
-                tracing::error!("Failed to clean up package {}: {}", package_id, e);
-                tracing::debug!("{:?}", e);
-            }
-        }
-        let peek = self
-            .db
-            .mutate(|v| {
-                for (_, pde) in v.as_package_data_mut().as_entries_mut()? {
-                    let status = pde
-                        .expect_as_installed_mut()?
-                        .as_installed_mut()
-                        .as_status_mut()
-                        .as_main_mut();
-                    let running = status.clone().de()?.running();
-                    status.ser(&if running {
-                        MainStatus::Starting
-                    } else {
-                        MainStatus::Stopped
-                    })?;
-                }
-                Ok(v.clone())
-            })
-            .await?;
-        self.managers.init(self.clone(), peek.clone()).await?;
+        self.services.init(&self).await?;
         tracing::info!("Initialized Package Managers");
 
         let mut all_dependency_config_errs = BTreeMap::new();
+        let peek = self.db.peek().await;
         for (package_id, package) in peek.as_package_data().as_entries()?.into_iter() {
             let package = package.clone();
             if let Some(current_dependencies) = package
@@ -419,33 +314,30 @@ impl RpcContext {
             .insert(guid, handler);
     }
 
-    pub async fn get_continuation_handler(&self, guid: &RequestGuid) -> Option<RestHandler> {
+    pub async fn get_ws_continuation_handler(
+        &self,
+        guid: &RequestGuid,
+    ) -> Option<WebSocketHandler> {
         let mut continuations = self.rpc_stream_continuations.lock().await;
-        if let Some(cont) = continuations.remove(guid) {
-            cont.into_handler().await
-        } else {
-            None
-        }
-    }
-
-    pub async fn get_ws_continuation_handler(&self, guid: &RequestGuid) -> Option<RestHandler> {
-        let continuations = self.rpc_stream_continuations.lock().await;
-        if matches!(continuations.get(guid), Some(RpcContinuation::WebSocket(_))) {
-            drop(continuations);
-            self.get_continuation_handler(guid).await
-        } else {
-            None
+        if !matches!(continuations.get(guid), Some(RpcContinuation::WebSocket(_))) {
+            return None;
         }
+        let Some(RpcContinuation::WebSocket(x)) = continuations.remove(guid) else {
+            return None;
+        };
+        x.get().await
     }
 
     pub async fn get_rest_continuation_handler(&self, guid: &RequestGuid) -> Option<RestHandler> {
-        let continuations = self.rpc_stream_continuations.lock().await;
-        if matches!(continuations.get(guid), Some(RpcContinuation::Rest(_))) {
-            drop(continuations);
-            self.get_continuation_handler(guid).await
-        } else {
-            None
+        let mut continuations: tokio::sync::MutexGuard<'_, BTreeMap<RequestGuid, RpcContinuation>> =
+            self.rpc_stream_continuations.lock().await;
+        if !matches!(continuations.get(guid), Some(RpcContinuation::Rest(_))) {
+            return None;
         }
+        let Some(RpcContinuation::Rest(x)) = continuations.remove(guid) else {
+            return None;
+        };
+        x.get().await
     }
 }
 impl AsRef<Jwk> for RpcContext {

core/startos/src/context/sdk.rs

@@ -8,13 +8,6 @@ use serde::Deserialize;
 use tracing::instrument;
 
 use crate::prelude::*;
-use crate::util::config::{load_config_from_paths, local_config_path};
-
-#[derive(Debug, Default, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-pub struct SdkContextConfig {
-    pub developer_key_path: Option<PathBuf>,
-}
 
 #[derive(Debug)]
 pub struct SdkContextSeed {
@@ -26,7 +19,7 @@ pub struct SdkContext(Arc<SdkContextSeed>);
 impl SdkContext {
     /// BLOCKING
     #[instrument(skip_all)]
-    pub fn init(matches: &ArgMatches) -> Result<Self, crate::Error> {
+    pub fn init(config: ) -> Result<Self, crate::Error> {
         let local_config_path = local_config_path();
         let base: SdkContextConfig = load_config_from_paths(
             matches
@@ -48,24 +41,7 @@ impl SdkContext {
             }),
         })))
     }
-    /// BLOCKING
-    #[instrument(skip_all)]
-    pub fn developer_key(&self) -> Result<ed25519_dalek::SigningKey, Error> {
-        if !self.developer_key_path.exists() {
-            return Err(Error::new(eyre!("Developer Key does not exist! Please run `start-sdk init` before running this command."), crate::ErrorKind::Uninitialized));
-        }
-        let pair = <ed25519::KeypairBytes as ed25519::pkcs8::DecodePrivateKey>::from_pkcs8_pem(
-            &std::fs::read_to_string(&self.developer_key_path)?,
-        )
-        .with_kind(crate::ErrorKind::Pem)?;
-        let secret = ed25519_dalek::SecretKey::try_from(&pair.secret_key[..]).map_err(|_| {
-            Error::new(
-                eyre!("pkcs8 key is of incorrect length"),
-                ErrorKind::OpenSsl,
-            )
-        })?;
-        Ok(secret.into())
-    }
+    
 }
 impl std::ops::Deref for SdkContext {
     type Target = SdkContextSeed;

core/startos/src/context/setup.rs

@@ -1,5 +1,5 @@
 use std::ops::Deref;
-use std::path::{Path, PathBuf};
+use std::path::PathBuf;
 use std::sync::Arc;
 
 use josekit::jwk::Jwk;
@@ -15,12 +15,12 @@ use tokio::sync::RwLock;
 use tracing::instrument;
 
 use crate::account::AccountInfo;
+use crate::context::config::ServerConfig;
 use crate::db::model::Database;
 use crate::disk::OsPartitionInfo;
 use crate::init::init_postgres;
+use crate::prelude::*;
 use crate::setup::SetupStatus;
-use crate::util::config::load_config_from_paths;
-use crate::{Error, ResultExt};
 
 lazy_static::lazy_static! {
     pub static ref CURRENT_SECRET: Jwk = Jwk::generate_ec_key(josekit::jwk::alg::ec::EcCurve::P256).unwrap_or_else(|e| {
@@ -38,45 +38,9 @@ pub struct SetupResult {
     pub root_ca: String,
 }
 
-#[derive(Debug, Default, Deserialize)]
-#[serde(rename_all = "kebab-case")]
-pub struct SetupContextConfig {
-    pub os_partitions: OsPartitionInfo,
-    pub migration_batch_rows: Option<usize>,
-    pub migration_prefetch_rows: Option<usize>,
-    pub datadir: Option<PathBuf>,
-    #[serde(default)]
-    pub disable_encryption: bool,
-}
-impl SetupContextConfig {
-    #[instrument(skip_all)]
-    pub async fn load<P: AsRef<Path> + Send + 'static>(path: Option<P>) -> Result<Self, Error> {
-        tokio::task::spawn_blocking(move || {
-            load_config_from_paths(
-                path.as_ref()
-                    .into_iter()
-                    .map(|p| p.as_ref())
-                    .chain(std::iter::once(Path::new(
-                        crate::util::config::DEVICE_CONFIG_PATH,
-                    )))
-                    .chain(std::iter::once(Path::new(crate::util::config::CONFIG_PATH))),
-            )
-        })
-        .await
-        .unwrap()
-    }
-    pub fn datadir(&self) -> &Path {
-        self.datadir
-            .as_deref()
-            .unwrap_or_else(|| Path::new("/embassy-data"))
-    }
-}
-
 pub struct SetupContextSeed {
+    pub config: ServerConfig,
     pub os_partitions: OsPartitionInfo,
-    pub config_path: Option<PathBuf>,
-    pub migration_batch_rows: usize,
-    pub migration_prefetch_rows: usize,
     pub disable_encryption: bool,
     pub shutdown: Sender<()>,
     pub datadir: PathBuf,
@@ -96,16 +60,18 @@ impl AsRef<Jwk> for SetupContextSeed {
 pub struct SetupContext(Arc<SetupContextSeed>);
 impl SetupContext {
     #[instrument(skip_all)]
-    pub async fn init<P: AsRef<Path> + Send + 'static>(path: Option<P>) -> Result<Self, Error> {
-        let cfg = SetupContextConfig::load(path.as_ref().map(|p| p.as_ref().to_owned())).await?;
+    pub fn init(config: &ServerConfig) -> Result<Self, Error> {
         let (shutdown, _) = tokio::sync::broadcast::channel(1);
-        let datadir = cfg.datadir().to_owned();
+        let datadir = config.datadir().to_owned();
         Ok(Self(Arc::new(SetupContextSeed {
-            os_partitions: cfg.os_partitions,
-            config_path: path.as_ref().map(|p| p.as_ref().to_owned()),
-            migration_batch_rows: cfg.migration_batch_rows.unwrap_or(25000),
-            migration_prefetch_rows: cfg.migration_prefetch_rows.unwrap_or(100_000),
-            disable_encryption: cfg.disable_encryption,
+            config: config.clone(),
+            os_partitions: config.os_partitions.clone().ok_or_else(|| {
+                Error::new(
+                    eyre!("missing required configuration: `os-partitions`"),
+                    ErrorKind::NotFound,
+                )
+            })?,
+            disable_encryption: config.disable_encryption.unwrap_or(false),
             shutdown,
             datadir,
             selected_v2_drive: RwLock::new(None),