Feat/js long running (#1879)

* wip: combining the streams * chore: Testing locally * chore: Fix some lint * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * wip: making the mananger create * wip: Working on trying to make the long running docker container command * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * feat: Use the long running feature in the manager * remove recovered services and drop reordering feature (#1829) * wip: Need to get the initial docker command running? * chore: Add in the new procedure for the docker. * feat: Get the system to finally run long * wip: Added the command inserter to the docker persistance * wip: Added the command inserter to the docker persistance * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * remove recovered services and drop reordering feature (#1829) * chore: Convert the migration to use receipt. (#1842) * feat: remove ionic storage (#1839) * feat: remove ionic storage * grayscal when disconncted, rename local storage service for clarity * remove storage from package lock * update patchDB Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> * update patchDB * feat: Move the run_command into the js * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * remove recovered services and drop reordering feature (#1829) * chore: Convert the migration to use receipt. (#1842) * feat: remove ionic storage (#1839) * feat: remove ionic storage * grayscal when disconncted, rename local storage service for clarity * remove storage from package lock * update patchDB Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> * update patch DB * chore: Change the error catching for the long running to try all * Feat/community marketplace (#1790) * add community marketplace * Update embassy-mock-api.service.ts * expect ui/marketplace to be undefined * possible undefined from getpackage * fix marketplace pages * rework marketplace infrastructure * fix bugs Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> * WIP: Fix the build, needed to move around creation of exec * wip: Working on solving why there is a missing end. * fix: make `shared` module independent of `config.js` (#1870) * feat: Add in the kill and timeout * feat: Get the run to actually work. * chore: Add when/ why/ where comments * feat: Convert inject main to use exec main. * Fix: Ability to stop services * wip: long running js main * feat: Kill for the main * Fix * fix: Fix the build for x86 * wip: Working on changes * wip: Working on trying to kill js * fix: Testing for slow * feat: Test that the new manifest works * chore: Try and fix build? * chore: Fix? the build * chore: Fix the long input dies and never restarts * build improvements * no workdir * fix: Architecture for long running * chore: Fix and remove the docker inject * chore: Undo the changes to the kiosk mode * fix: Remove the it from the prod build * fix: Start issue * fix: The compat build * chore: Add in the conditional compilation again for the missing impl * chore: Change to aux * chore: Remove the aux for now * chore: Add some documentation to docker container Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> Co-authored-by: Alex Inkin <alexander@inkin.ru>
2026-03-30 04:01:58 +00:00 · 2022-10-25 17:18:49 -06:00
parent 26d2152a36
commit 2642ec85e5
46 changed files with 2466 additions and 1050 deletions
--- a/libs/js_engine/src/lib.rs
+++ b/libs/js_engine/src/lib.rs
@@ -1,3 +1,5 @@
+use std::collections::BTreeMap;
+use std::future::Future;
 use std::path::{Path, PathBuf};
 use std::pin::Pin;
 use std::sync::Arc;
@@ -9,11 +11,12 @@ use deno_core::{
    resolve_import, Extension, JsRuntime, ModuleLoader, ModuleSource, ModuleSourceFuture,
    ModuleSpecifier, ModuleType, OpDecl, RuntimeOptions, Snapshot,
 };
-use helpers::{script_dir, NonDetachingJoinHandle};
-use models::{PackageId, ProcedureName, Version, VolumeId};
+use helpers::{script_dir, SingleThreadJoinHandle};
+use models::{ExecCommand, PackageId, ProcedureName, TermCommand, Version, VolumeId};
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
 use tokio::io::AsyncReadExt;
+use tokio::sync::Mutex;

 pub trait PathForVolumeId: Send + Sync {
    fn path_for(
@@ -80,6 +83,8 @@ const SNAPSHOT_BYTES: &[u8] = include_bytes!("./artifacts/JS_SNAPSHOT.bin");

 #[cfg(target_arch = "aarch64")]
 const SNAPSHOT_BYTES: &[u8] = include_bytes!("./artifacts/ARM_JS_SNAPSHOT.bin");
+type WaitFns = Arc<Mutex<BTreeMap<u32, Pin<Box<dyn Future<Output = ResultType>>>>>>;
+
 #[derive(Clone)]
 struct JsContext {
    sandboxed: bool,
@@ -90,8 +95,17 @@ struct JsContext {
    volumes: Arc<dyn PathForVolumeId>,
    input: Value,
    variable_args: Vec<serde_json::Value>,
+    command_inserter: ExecCommand,
+    term_command: TermCommand,
+    wait_fns: WaitFns,
+}
+#[derive(Debug, Clone, serde::Deserialize, serde::Serialize)]
+#[serde(rename_all = "kebab-case")]
+enum ResultType {
+    Error(String),
+    ErrorCode(i32, String),
+    Result(serde_json::Value),
 }
-
 #[derive(Clone, Default)]
 struct AnswerState(std::sync::Arc<deno_core::parking_lot::Mutex<Value>>);

@@ -162,6 +176,7 @@ impl ModuleLoader for ModsLoader {
        })
    }
 }
+
 pub struct JsExecutionEnvironment {
    sandboxed: bool,
    base_directory: PathBuf,
@@ -169,6 +184,8 @@ pub struct JsExecutionEnvironment {
    package_id: PackageId,
    version: Version,
    volumes: Arc<dyn PathForVolumeId>,
+    command_inserter: ExecCommand,
+    term_command: TermCommand,
 }

 impl JsExecutionEnvironment {
@@ -177,7 +194,9 @@ impl JsExecutionEnvironment {
        package_id: &PackageId,
        version: &Version,
        volumes: Box<dyn PathForVolumeId>,
-    ) -> Result<Self, (JsError, String)> {
+        command_inserter: ExecCommand,
+        term_command: TermCommand,
+    ) -> Result<JsExecutionEnvironment, (JsError, String)> {
        let data_dir = data_directory.as_ref();
        let base_directory = data_dir;
        let js_code = JsCode({
@@ -203,13 +222,15 @@ impl JsExecutionEnvironment {
            };
            buffer
        });
-        Ok(Self {
+        Ok(JsExecutionEnvironment {
            base_directory: base_directory.to_owned(),
            module_loader: ModsLoader { code: js_code },
            package_id: package_id.clone(),
            version: version.clone(),
            volumes: volumes.into(),
            sandboxed: false,
+            command_inserter,
+            term_command,
        })
    }
    pub fn read_only_effects(mut self) -> Self {
@@ -234,12 +255,9 @@ impl JsExecutionEnvironment {
                ));
            }
        };
-        let safer_handle: NonDetachingJoinHandle<_> =
-            tokio::task::spawn_blocking(move || self.execute(procedure_name, input, variable_args))
-                .into();
-        let output = safer_handle
-            .await
-            .map_err(|err| (JsError::Tokio, format!("Tokio gave us the error: {}", err)))??;
+        let safer_handle =
+            SingleThreadJoinHandle::new(move || self.execute(procedure_name, input, variable_args));
+        let output = safer_handle.await?;
        match serde_json::from_value(output.clone()) {
            Ok(x) => Ok(x),
            Err(err) => {
@@ -275,11 +293,15 @@ impl JsExecutionEnvironment {
            fns::get_variable_args::decl(),
            fns::set_value::decl(),
            fns::is_sandboxed::decl(),
+            fns::start_command::decl(),
+            fns::wait_command::decl(),
+            fns::sleep::decl(),
+            fns::term_command::decl(),
        ]
    }

-    fn execute(
-        &self,
+    async fn execute(
+        self,
        procedure_name: ProcedureName,
        input: Value,
        variable_args: Vec<serde_json::Value>,
@@ -304,6 +326,9 @@ impl JsExecutionEnvironment {
            sandboxed: self.sandboxed,
            input,
            variable_args,
+            command_inserter: self.command_inserter.clone(),
+            term_command: self.term_command.clone(),
+            wait_fns: Default::default(),
        };
        let ext = Extension::builder()
            .ops(Self::declarations())
@@ -321,25 +346,25 @@ impl JsExecutionEnvironment {
            startup_snapshot: Some(Snapshot::Static(SNAPSHOT_BYTES)),
            ..Default::default()
        };
-        let mut runtime = JsRuntime::new(runtime_options);
+        let runtime = Arc::new(Mutex::new(JsRuntime::new(runtime_options)));

        let future = async move {
            let mod_id = runtime
+                .lock()
+                .await
                .load_main_module(&"file:///loadModule.js".parse().unwrap(), None)
                .await?;
-            let evaluated = runtime.mod_evaluate(mod_id);
-            let res = runtime.run_event_loop(false).await;
+            let evaluated = runtime.lock().await.mod_evaluate(mod_id);
+            let res = runtime.lock().await.run_event_loop(false).await;
            res?;
            evaluated.await??;
            Ok::<_, AnyError>(())
        };

-        tokio::runtime::Handle::current()
-            .block_on(future)
-            .map_err(|e| {
-                tracing::debug!("{:?}", e);
-                (JsError::Javascript, format!("{}", e))
-            })?;
+        future.await.map_err(|e| {
+            tracing::debug!("{:?}", e);
+            (JsError::Javascript, format!("{}", e))
+        })?;

        let answer = answer_state.0.lock().clone();
        Ok(answer)
@@ -348,23 +373,24 @@ impl JsExecutionEnvironment {

 /// Note: Make sure that we have the assumption that all these methods are callable at any time, and all call restrictions should be in rust
 mod fns {
-    use std::cell::RefCell;
    use std::collections::BTreeMap;
    use std::convert::TryFrom;
    use std::os::unix::fs::MetadataExt;
    use std::path::{Path, PathBuf};
    use std::rc::Rc;
+    use std::{cell::RefCell, time::Duration};

    use deno_core::anyhow::{anyhow, bail};
    use deno_core::error::AnyError;
    use deno_core::*;
+    use embassy_container_init::RpcId;
    use helpers::{to_tmp_path, AtomicFile};
-    use models::VolumeId;
+    use models::{TermCommand, VolumeId};
    use serde_json::Value;
    use tokio::io::AsyncWriteExt;

    use super::{AnswerState, JsContext};
-    use crate::{system_time_as_unix_ms, MetadataJs};
+    use crate::{system_time_as_unix_ms, MetadataJs, ResultType};

    #[derive(serde::Serialize, serde::Deserialize, Debug, Clone, Default)]
    struct FetchOptions {
@@ -386,10 +412,13 @@ mod fns {
        url: url::Url,
        options: Option<FetchOptions>,
    ) -> Result<FetchResponse, AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
+        let sandboxed = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            ctx.sandboxed
+        };

-        if ctx.sandboxed {
+        if sandboxed {
            bail!("Will not run fetch in sandboxed mode");
        }

@@ -432,7 +461,7 @@ mod fns {
            body: response.text().await.ok(),
        };

-        return Ok(fetch_response);
+        Ok(fetch_response)
    }

    #[op]
@@ -441,12 +470,13 @@ mod fns {
        volume_id: VolumeId,
        path_in: PathBuf,
    ) -> Result<String, AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
-        let volume_path = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
+        let volume_path = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            ctx.volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?
+        };
        //get_path_for in volume.rs
        let new_file = volume_path.join(path_in);
        if !is_subset(&volume_path, &new_file).await? {
@@ -465,12 +495,13 @@ mod fns {
        volume_id: VolumeId,
        path_in: PathBuf,
    ) -> Result<MetadataJs, AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
-        let volume_path = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
+        let volume_path = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            ctx.volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?
+        };
        //get_path_for in volume.rs
        let new_file = volume_path.join(path_in);
        if !is_subset(&volume_path, &new_file).await? {
@@ -517,13 +548,16 @@ mod fns {
        path_in: PathBuf,
        write: String,
    ) -> Result<(), AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
-        let volume_path = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
-        if ctx.volumes.readonly(&volume_id) {
+        let (volumes, volume_path) = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            let volume_path = ctx
+                .volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
+            (ctx.volumes.clone(), volume_path)
+        };
+        if volumes.readonly(&volume_id) {
            bail!("Volume {} is readonly", volume_id);
        }

@@ -566,17 +600,20 @@ mod fns {
        dst_volume: VolumeId,
        dst_path: PathBuf,
    ) -> Result<(), AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
-        let volume_path = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &src_volume)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", src_volume))?;
-        let volume_path_out = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &dst_volume)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", dst_volume))?;
-        if ctx.volumes.readonly(&dst_volume) {
+        let (volumes, volume_path, volume_path_out) = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            let volume_path = ctx
+                .volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &src_volume)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", src_volume))?;
+            let volume_path_out = ctx
+                .volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &dst_volume)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", dst_volume))?;
+            (ctx.volumes.clone(), volume_path, volume_path_out)
+        };
+        if volumes.readonly(&dst_volume) {
            bail!("Volume {} is readonly", dst_volume);
        }

@@ -614,13 +651,16 @@ mod fns {
        volume_id: VolumeId,
        path_in: PathBuf,
    ) -> Result<(), AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
-        let volume_path = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
-        if ctx.volumes.readonly(&volume_id) {
+        let (volumes, volume_path) = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            let volume_path = ctx
+                .volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
+            (ctx.volumes.clone(), volume_path)
+        };
+        if volumes.readonly(&volume_id) {
            bail!("Volume {} is readonly", volume_id);
        }
        let new_file = volume_path.join(path_in);
@@ -641,13 +681,16 @@ mod fns {
        volume_id: VolumeId,
        path_in: PathBuf,
    ) -> Result<(), AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
-        let volume_path = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
-        if ctx.volumes.readonly(&volume_id) {
+        let (volumes, volume_path) = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            let volume_path = ctx
+                .volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
+            (ctx.volumes.clone(), volume_path)
+        };
+        if volumes.readonly(&volume_id) {
            bail!("Volume {} is readonly", volume_id);
        }
        let new_file = volume_path.join(path_in);
@@ -668,13 +711,16 @@ mod fns {
        volume_id: VolumeId,
        path_in: PathBuf,
    ) -> Result<(), AnyError> {
-        let state = state.borrow();
-        let ctx: &JsContext = state.borrow();
-        let volume_path = ctx
-            .volumes
-            .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
-            .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
-        if ctx.volumes.readonly(&volume_id) {
+        let (volumes, volume_path) = {
+            let state = state.borrow();
+            let ctx: &JsContext = state.borrow();
+            let volume_path = ctx
+                .volumes
+                .path_for(&ctx.datadir, &ctx.package_id, &ctx.version, &volume_id)
+                .ok_or_else(|| anyhow!("There is no {} in volumes", volume_id))?;
+            (ctx.volumes.clone(), volume_path)
+        };
+        if volumes.readonly(&volume_id) {
            bail!("Volume {} is readonly", volume_id);
        }
        let new_file = volume_path.join(path_in);
@@ -777,6 +823,102 @@ mod fns {
        Ok(ctx.sandboxed)
    }

+    #[op]
+    async fn term_command(state: Rc<RefCell<OpState>>, id: u32) -> Result<(), AnyError> {
+        let term_command_impl: TermCommand = {
+            let state = state.borrow();
+            let ctx = state.borrow::<JsContext>();
+            ctx.term_command.clone()
+        };
+        if let Err(err) = term_command_impl(embassy_container_init::RpcId::UInt(id)).await {
+            bail!("{}", err);
+        }
+        Ok(())
+    }
+
+    #[op]
+    async fn start_command(
+        state: Rc<RefCell<OpState>>,
+        command: String,
+        args: Vec<String>,
+        timeout: Option<u64>,
+    ) -> Result<u32, AnyError> {
+        use embassy_container_init::Output;
+        let (command_inserter, wait_fns) = {
+            let state = state.borrow();
+            let ctx = state.borrow::<JsContext>();
+            (ctx.command_inserter.clone(), ctx.wait_fns.clone())
+        };
+
+        let (sender, mut receiver) = tokio::sync::mpsc::unbounded_channel::<Output>();
+        let id = match command_inserter(
+            command,
+            args.into_iter().collect(),
+            sender,
+            timeout.map(std::time::Duration::from_millis),
+        )
+        .await
+        {
+            Err(err) => bail!(err),
+            Ok(RpcId::UInt(a)) => a,
+        };
+
+        let wait = async move {
+            let mut answer = String::new();
+            let mut command_error = String::new();
+            let mut status: Option<i32> = None;
+            while let Some(output) = receiver.recv().await {
+                match output {
+                    Output::Line(value) => {
+                        answer.push_str(&value);
+                        answer.push('\n');
+                    }
+                    Output::Error(error) => {
+                        command_error.push_str(&error);
+                        command_error.push('\n');
+                    }
+                    Output::Done(error_code) => {
+                        status = error_code;
+                        break;
+                    }
+                }
+            }
+            if !command_error.is_empty() {
+                if let Some(status) = status {
+                    return ResultType::ErrorCode(status, command_error);
+                }
+
+                return ResultType::Error(command_error);
+            }
+
+            ResultType::Result(serde_json::Value::String(answer))
+        };
+        wait_fns.lock().await.insert(id, Box::pin(wait));
+        Ok(id)
+    }
+
+    #[op]
+    async fn wait_command(state: Rc<RefCell<OpState>>, id: u32) -> Result<ResultType, AnyError> {
+        let wait_fns = {
+            let state = state.borrow();
+            let ctx = state.borrow::<JsContext>();
+            ctx.wait_fns.clone()
+        };
+
+        let found_future = match wait_fns.lock().await.remove(&id) {
+            Some(a) => a,
+            None => bail!("No future for id {id}, could have been removed already"),
+        };
+
+        Ok(found_future.await)
+    }
+    #[op]
+    async fn sleep(time_ms: u64) -> Result<(), AnyError> {
+        tokio::time::sleep(Duration::from_millis(time_ms)).await;
+
+        Ok(())
+    }
+
    /// We need to make sure that during the file accessing, we don't reach beyond our scope of control
    async fn is_subset(
        parent: impl AsRef<Path>,