mirror of
https://github.com/Start9Labs/start-os.git
synced 2026-03-30 04:01:58 +00:00
Feat/js long running (#1879)
* wip: combining the streams * chore: Testing locally * chore: Fix some lint * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * wip: making the mananger create * wip: Working on trying to make the long running docker container command * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * feat: Use the long running feature in the manager * remove recovered services and drop reordering feature (#1829) * wip: Need to get the initial docker command running? * chore: Add in the new procedure for the docker. * feat: Get the system to finally run long * wip: Added the command inserter to the docker persistance * wip: Added the command inserter to the docker persistance * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * remove recovered services and drop reordering feature (#1829) * chore: Convert the migration to use receipt. (#1842) * feat: remove ionic storage (#1839) * feat: remove ionic storage * grayscal when disconncted, rename local storage service for clarity * remove storage from package lock * update patchDB Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> * update patchDB * feat: Move the run_command into the js * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * remove recovered services and drop reordering feature (#1829) * chore: Convert the migration to use receipt. (#1842) * feat: remove ionic storage (#1839) * feat: remove ionic storage * grayscal when disconncted, rename local storage service for clarity * remove storage from package lock * update patchDB Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> * update patch DB * chore: Change the error catching for the long running to try all * Feat/community marketplace (#1790) * add community marketplace * Update embassy-mock-api.service.ts * expect ui/marketplace to be undefined * possible undefined from getpackage * fix marketplace pages * rework marketplace infrastructure * fix bugs Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> * WIP: Fix the build, needed to move around creation of exec * wip: Working on solving why there is a missing end. * fix: make `shared` module independent of `config.js` (#1870) * feat: Add in the kill and timeout * feat: Get the run to actually work. * chore: Add when/ why/ where comments * feat: Convert inject main to use exec main. * Fix: Ability to stop services * wip: long running js main * feat: Kill for the main * Fix * fix: Fix the build for x86 * wip: Working on changes * wip: Working on trying to kill js * fix: Testing for slow * feat: Test that the new manifest works * chore: Try and fix build? * chore: Fix? the build * chore: Fix the long input dies and never restarts * build improvements * no workdir * fix: Architecture for long running * chore: Fix and remove the docker inject * chore: Undo the changes to the kiosk mode * fix: Remove the it from the prod build * fix: Start issue * fix: The compat build * chore: Add in the conditional compilation again for the missing impl * chore: Change to aux * chore: Remove the aux for now * chore: Add some documentation to docker container Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> Co-authored-by: Alex Inkin <alexander@inkin.ru>
This commit is contained in:
J M
@@ -9,14 +9,19 @@ use async_stream::stream;
|
||||
use bollard::container::RemoveContainerOptions;
|
||||
use color_eyre::eyre::eyre;
|
||||
use color_eyre::Report;
|
||||
use embassy_container_init::{InputJsonRpc, OutputJsonRpc};
|
||||
use futures::future::Either as EitherFuture;
|
||||
use futures::TryStreamExt;
|
||||
use futures::{Stream, StreamExt, TryFutureExt, TryStreamExt};
|
||||
use helpers::NonDetachingJoinHandle;
|
||||
use nix::sys::signal;
|
||||
use nix::unistd::Pid;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde::{de::DeserializeOwned, Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
use tokio::io::{AsyncBufRead, AsyncBufReadExt, BufReader};
|
||||
use tokio::{
|
||||
io::{AsyncBufRead, AsyncBufReadExt, BufReader},
|
||||
process::Child,
|
||||
sync::mpsc::UnboundedReceiver,
|
||||
};
|
||||
use tracing::instrument;
|
||||
|
||||
use super::ProcedureName;
|
||||
@@ -41,6 +46,17 @@ lazy_static::lazy_static! {
|
||||
};
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Deserialize, Serialize, patch_db::HasModel)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub struct DockerContainers {
|
||||
pub main: DockerContainer,
|
||||
// #[serde(default)]
|
||||
// pub aux: BTreeMap<String, DockerContainer>,
|
||||
}
|
||||
|
||||
/// This is like the docker procedures of the past designs,
|
||||
/// but this time all the entrypoints and args are not
|
||||
/// part of this struct by choice. Used for the times that we are creating our own entry points
|
||||
#[derive(Clone, Debug, Deserialize, Serialize, patch_db::HasModel)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub struct DockerContainer {
|
||||
@@ -49,6 +65,10 @@ pub struct DockerContainer {
|
||||
pub mounts: BTreeMap<VolumeId, PathBuf>,
|
||||
#[serde(default)]
|
||||
pub shm_size_mb: Option<usize>, // TODO: use postfix sizing? like 1k vs 1m vs 1g
|
||||
#[serde(default)]
|
||||
pub sigterm_timeout: Option<SerdeDuration>,
|
||||
#[serde(default)]
|
||||
pub system: bool,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Deserialize, Serialize)]
|
||||
@@ -70,7 +90,7 @@ pub struct DockerProcedure {
|
||||
pub shm_size_mb: Option<usize>, // TODO: use postfix sizing? like 1k vs 1m vs 1g
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Deserialize, Serialize)]
|
||||
#[derive(Clone, Debug, Deserialize, Serialize, Default)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub struct DockerInject {
|
||||
#[serde(default)]
|
||||
@@ -83,26 +103,42 @@ pub struct DockerInject {
|
||||
#[serde(default)]
|
||||
pub sigterm_timeout: Option<SerdeDuration>,
|
||||
}
|
||||
|
||||
impl From<(&DockerContainer, &DockerInject)> for DockerProcedure {
|
||||
fn from((container, injectable): (&DockerContainer, &DockerInject)) -> Self {
|
||||
impl DockerProcedure {
|
||||
pub fn main_docker_procedure(
|
||||
container: &DockerContainer,
|
||||
injectable: &DockerInject,
|
||||
) -> DockerProcedure {
|
||||
DockerProcedure {
|
||||
image: container.image.clone(),
|
||||
system: injectable.system.clone(),
|
||||
system: injectable.system,
|
||||
entrypoint: injectable.entrypoint.clone(),
|
||||
args: injectable.args.clone(),
|
||||
mounts: container.mounts.clone(),
|
||||
io_format: injectable.io_format.clone(),
|
||||
sigterm_timeout: injectable.sigterm_timeout.clone(),
|
||||
shm_size_mb: container.shm_size_mb.clone(),
|
||||
io_format: injectable.io_format,
|
||||
sigterm_timeout: injectable.sigterm_timeout,
|
||||
shm_size_mb: container.shm_size_mb,
|
||||
}
|
||||
}
|
||||
#[cfg(feature = "js_engine")]
|
||||
pub fn main_docker_procedure_js(
|
||||
container: &DockerContainer,
|
||||
_procedure: &super::js_scripts::JsProcedure,
|
||||
) -> DockerProcedure {
|
||||
DockerProcedure {
|
||||
image: container.image.clone(),
|
||||
system: container.system,
|
||||
entrypoint: "sleep".to_string(),
|
||||
args: Vec::new(),
|
||||
mounts: container.mounts.clone(),
|
||||
io_format: None,
|
||||
sigterm_timeout: container.sigterm_timeout,
|
||||
shm_size_mb: container.shm_size_mb,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl DockerProcedure {
|
||||
pub fn validate(
|
||||
&self,
|
||||
eos_version: &Version,
|
||||
_eos_version: &Version,
|
||||
volumes: &Volumes,
|
||||
image_ids: &BTreeSet<ImageId>,
|
||||
expected_io: bool,
|
||||
@@ -116,10 +152,8 @@ impl DockerProcedure {
|
||||
if !SYSTEM_IMAGES.contains(&self.image) {
|
||||
color_eyre::eyre::bail!("unknown system image: {}", self.image);
|
||||
}
|
||||
} else {
|
||||
if !image_ids.contains(&self.image) {
|
||||
color_eyre::eyre::bail!("image for {} not contained in package", self.image);
|
||||
}
|
||||
} else if !image_ids.contains(&self.image) {
|
||||
color_eyre::eyre::bail!("image for {} not contained in package", self.image);
|
||||
}
|
||||
if expected_io && self.io_format.is_none() {
|
||||
color_eyre::eyre::bail!("expected io-format");
|
||||
@@ -128,7 +162,7 @@ impl DockerProcedure {
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, input))]
|
||||
pub async fn execute<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
pub async fn execute<I: Serialize, O: DeserializeOwned>(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
@@ -217,7 +251,7 @@ impl DockerProcedure {
|
||||
handle
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| eyre!("Can't takeout stout"))
|
||||
.ok_or_else(|| eyre!("Can't takeout stdout in execute"))
|
||||
.with_kind(crate::ErrorKind::Docker)?,
|
||||
);
|
||||
let output = NonDetachingJoinHandle::from(tokio::spawn(async move {
|
||||
@@ -307,25 +341,83 @@ impl DockerProcedure {
|
||||
)
|
||||
}
|
||||
|
||||
/// We created a new exec runner, where we are going to be passing the commands for it to run.
|
||||
/// Idea is that we are going to send it command and get the inputs be filtered back from the manager.
|
||||
/// Then we could in theory run commands without the cost of running the docker exec which is known to have
|
||||
/// a dely of > 200ms which is not acceptable.
|
||||
#[instrument(skip(ctx, input))]
|
||||
pub async fn inject<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
pub async fn long_running_execute<S>(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
name: ProcedureName,
|
||||
volumes: &Volumes,
|
||||
input: S,
|
||||
) -> Result<LongRunning, Error>
|
||||
where
|
||||
S: Stream<Item = InputJsonRpc> + Send + 'static,
|
||||
{
|
||||
let name = name.docker_name();
|
||||
let name: Option<&str> = name.as_deref();
|
||||
let container_name = Self::container_name(pkg_id, name);
|
||||
|
||||
let mut cmd = LongRunning::setup_long_running_docker_cmd(
|
||||
self,
|
||||
ctx,
|
||||
&container_name,
|
||||
volumes,
|
||||
pkg_id,
|
||||
pkg_version,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let mut handle = cmd.spawn().with_kind(crate::ErrorKind::Docker)?;
|
||||
let input_handle = LongRunning::spawn_input_handle(&mut handle, input)?
|
||||
.map_err(|e| eyre!("Input Handle Error: {e:?}"));
|
||||
|
||||
let (output, output_handle) = LongRunning::spawn_output_handle(&mut handle)?;
|
||||
let output_handle = output_handle.map_err(|e| eyre!("Output Handle Error: {e:?}"));
|
||||
let err_handle = LongRunning::spawn_error_handle(&mut handle)?
|
||||
.map_err(|e| eyre!("Err Handle Error: {e:?}"));
|
||||
|
||||
let running_output = NonDetachingJoinHandle::from(tokio::spawn(async move {
|
||||
if let Err(err) = tokio::select!(
|
||||
x = handle.wait().map_err(|e| eyre!("Runtime error: {e:?}")) => x.map(|_| ()),
|
||||
x = err_handle => x.map(|_| ()),
|
||||
x = output_handle => x.map(|_| ()),
|
||||
x = input_handle => x.map(|_| ())
|
||||
) {
|
||||
tracing::debug!("{:?}", err);
|
||||
tracing::error!("Join error");
|
||||
}
|
||||
}));
|
||||
|
||||
Ok(LongRunning {
|
||||
output,
|
||||
running_output,
|
||||
})
|
||||
}
|
||||
|
||||
#[instrument(skip(_ctx, input))]
|
||||
pub async fn inject<I: Serialize, O: DeserializeOwned>(
|
||||
&self,
|
||||
_ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
name: ProcedureName,
|
||||
volumes: &Volumes,
|
||||
input: Option<I>,
|
||||
timeout: Option<Duration>,
|
||||
) -> Result<Result<O, (i32, String)>, Error> {
|
||||
let name = name.docker_name();
|
||||
let name: Option<&str> = name.as_ref().map(|x| &**x);
|
||||
let name: Option<&str> = name.as_deref();
|
||||
let mut cmd = tokio::process::Command::new("docker");
|
||||
|
||||
tracing::debug!("{:?} is exec", name);
|
||||
cmd.arg("exec");
|
||||
|
||||
cmd.args(self.docker_args_inject(ctx, pkg_id, pkg_version).await?);
|
||||
cmd.args(self.docker_args_inject(pkg_id).await?);
|
||||
let input_buf = if let (Some(input), Some(format)) = (&input, &self.io_format) {
|
||||
cmd.stdin(std::process::Stdio::piped());
|
||||
Some(format.to_vec(input)?)
|
||||
@@ -372,7 +464,7 @@ impl DockerProcedure {
|
||||
handle
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| eyre!("Can't takeout stout"))
|
||||
.ok_or_else(|| eyre!("Can't takeout stdout in inject"))
|
||||
.with_kind(crate::ErrorKind::Docker)?,
|
||||
);
|
||||
let output = NonDetachingJoinHandle::from(tokio::spawn(async move {
|
||||
@@ -463,7 +555,7 @@ impl DockerProcedure {
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, input))]
|
||||
pub async fn sandboxed<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
pub async fn sandboxed<I: Serialize, O: DeserializeOwned>(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
@@ -513,7 +605,7 @@ impl DockerProcedure {
|
||||
handle
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| eyre!("Can't takeout stout"))
|
||||
.ok_or_else(|| eyre!("Can't takeout stdout in sandboxed"))
|
||||
.with_kind(crate::ErrorKind::Docker)?,
|
||||
);
|
||||
let output = NonDetachingJoinHandle::from(tokio::spawn(async move {
|
||||
@@ -607,7 +699,7 @@ impl DockerProcedure {
|
||||
continue;
|
||||
};
|
||||
let src = volume.path_for(&ctx.datadir, pkg_id, pkg_version, volume_id);
|
||||
if let Err(e) = tokio::fs::metadata(&src).await {
|
||||
if let Err(_e) = tokio::fs::metadata(&src).await {
|
||||
tokio::fs::create_dir_all(&src).await?;
|
||||
}
|
||||
res.push(OsStr::new("--mount").into());
|
||||
@@ -626,7 +718,6 @@ impl DockerProcedure {
|
||||
res.push(OsString::from(format!("{}m", shm_size_mb)).into());
|
||||
}
|
||||
res.push(OsStr::new("--interactive").into());
|
||||
|
||||
res.push(OsStr::new("--log-driver=journald").into());
|
||||
res.push(OsStr::new("--entrypoint").into());
|
||||
res.push(OsStr::new(&self.entrypoint).into());
|
||||
@@ -649,12 +740,7 @@ impl DockerProcedure {
|
||||
+ self.args.len(), // [ARG...]
|
||||
)
|
||||
}
|
||||
async fn docker_args_inject(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
) -> Result<Vec<Cow<'_, OsStr>>, Error> {
|
||||
async fn docker_args_inject(&self, pkg_id: &PackageId) -> Result<Vec<Cow<'_, OsStr>>, Error> {
|
||||
let mut res = self.new_docker_args();
|
||||
if let Some(shm_size_mb) = self.shm_size_mb {
|
||||
res.push(OsStr::new("--shm-size").into());
|
||||
@@ -693,6 +779,215 @@ impl<T> RingVec<T> {
|
||||
}
|
||||
}
|
||||
|
||||
/// This is created when we wanted a long running docker executor that we could send commands to and get the responses back.
|
||||
/// We wanted a long running since we want to be able to have the equivelent to the docker execute without the heavy costs of 400 + ms time lag.
|
||||
/// Also the long running let's us have the ability to start/ end the services quicker.
|
||||
pub struct LongRunning {
|
||||
pub output: UnboundedReceiver<OutputJsonRpc>,
|
||||
pub running_output: NonDetachingJoinHandle<()>,
|
||||
}
|
||||
|
||||
impl LongRunning {
|
||||
async fn setup_long_running_docker_cmd(
|
||||
docker: &DockerProcedure,
|
||||
ctx: &RpcContext,
|
||||
container_name: &str,
|
||||
volumes: &Volumes,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
) -> Result<tokio::process::Command, Error> {
|
||||
tracing::error!("BLUJ setup_long_running_docker_cmd {container_name}");
|
||||
const INIT_EXEC: &str = "/start9/embassy_container_init";
|
||||
const BIND_LOCATION: &str = "/usr/lib/embassy/container";
|
||||
tracing::trace!("setup_long_running_docker_cmd");
|
||||
|
||||
LongRunning::cleanup_previous_container(ctx, container_name).await?;
|
||||
|
||||
let image_architecture = {
|
||||
let mut cmd = tokio::process::Command::new("docker");
|
||||
cmd.arg("image")
|
||||
.arg("inspect")
|
||||
.arg("--format")
|
||||
.arg("'{{.Architecture}}'");
|
||||
|
||||
if docker.system {
|
||||
cmd.arg(docker.image.for_package(SYSTEM_PACKAGE_ID, None));
|
||||
} else {
|
||||
cmd.arg(docker.image.for_package(pkg_id, Some(pkg_version)));
|
||||
}
|
||||
let arch = String::from_utf8(cmd.output().await?.stdout)?;
|
||||
arch.replace('\'', "").trim().to_string()
|
||||
};
|
||||
|
||||
let mut cmd = tokio::process::Command::new("docker");
|
||||
cmd.arg("run")
|
||||
.arg("--network=start9")
|
||||
.arg(format!("--add-host=embassy:{}", Ipv4Addr::from(HOST_IP)))
|
||||
.arg("--mount")
|
||||
.arg(format!("type=bind,src={BIND_LOCATION},dst=/start9"))
|
||||
.arg("--name")
|
||||
.arg(&container_name)
|
||||
.arg(format!("--hostname={}", &container_name))
|
||||
.arg("--entrypoint")
|
||||
.arg(format!("{INIT_EXEC}.{image_architecture}"))
|
||||
.arg("-i")
|
||||
.arg("--rm");
|
||||
|
||||
for (volume_id, dst) in &docker.mounts {
|
||||
let volume = if let Some(v) = volumes.get(volume_id) {
|
||||
v
|
||||
} else {
|
||||
continue;
|
||||
};
|
||||
let src = volume.path_for(&ctx.datadir, pkg_id, pkg_version, volume_id);
|
||||
if let Err(_e) = tokio::fs::metadata(&src).await {
|
||||
tokio::fs::create_dir_all(&src).await?;
|
||||
}
|
||||
cmd.arg("--mount").arg(format!(
|
||||
"type=bind,src={},dst={}{}",
|
||||
src.display(),
|
||||
dst.display(),
|
||||
if volume.readonly() { ",readonly" } else { "" }
|
||||
));
|
||||
}
|
||||
if let Some(shm_size_mb) = docker.shm_size_mb {
|
||||
cmd.arg("--shm-size").arg(format!("{}m", shm_size_mb));
|
||||
}
|
||||
cmd.arg("--log-driver=journald");
|
||||
if docker.system {
|
||||
cmd.arg(docker.image.for_package(SYSTEM_PACKAGE_ID, None));
|
||||
} else {
|
||||
cmd.arg(docker.image.for_package(pkg_id, Some(pkg_version)));
|
||||
}
|
||||
cmd.stdout(std::process::Stdio::piped());
|
||||
cmd.stderr(std::process::Stdio::piped());
|
||||
cmd.stdin(std::process::Stdio::piped());
|
||||
Ok(cmd)
|
||||
}
|
||||
|
||||
async fn cleanup_previous_container(
|
||||
ctx: &RpcContext,
|
||||
container_name: &str,
|
||||
) -> Result<(), Error> {
|
||||
match ctx
|
||||
.docker
|
||||
.remove_container(
|
||||
container_name,
|
||||
Some(RemoveContainerOptions {
|
||||
v: false,
|
||||
force: true,
|
||||
link: false,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(())
|
||||
| Err(bollard::errors::Error::DockerResponseServerError {
|
||||
status_code: 404, // NOT FOUND
|
||||
..
|
||||
}) => Ok(()),
|
||||
Err(e) => Err(e)?,
|
||||
}
|
||||
}
|
||||
fn spawn_input_handle<S>(
|
||||
handle: &mut Child,
|
||||
input: S,
|
||||
) -> Result<NonDetachingJoinHandle<()>, Error>
|
||||
where
|
||||
S: Stream<Item = InputJsonRpc> + Send + 'static,
|
||||
{
|
||||
use tokio::io::AsyncWriteExt;
|
||||
let mut stdin = handle
|
||||
.stdin
|
||||
.take()
|
||||
.ok_or_else(|| eyre!("Can't takeout stdin"))
|
||||
.with_kind(crate::ErrorKind::Docker)?;
|
||||
let handle = NonDetachingJoinHandle::from(tokio::spawn(async move {
|
||||
let input = input;
|
||||
tokio::pin!(input);
|
||||
while let Some(input) = input.next().await {
|
||||
let input = match serde_json::to_string(&input) {
|
||||
Ok(a) => a,
|
||||
Err(e) => {
|
||||
tracing::debug!("{:?}", e);
|
||||
tracing::error!("Docker Input Serialization issue");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
if let Err(e) = stdin.write_all(format!("{input}\n").as_bytes()).await {
|
||||
tracing::debug!("{:?}", e);
|
||||
tracing::error!("Docker Input issue");
|
||||
return;
|
||||
}
|
||||
}
|
||||
}));
|
||||
Ok(handle)
|
||||
}
|
||||
fn spawn_error_handle(handle: &mut Child) -> Result<NonDetachingJoinHandle<()>, Error> {
|
||||
let id = handle.id();
|
||||
let mut output = tokio::io::BufReader::new(
|
||||
handle
|
||||
.stderr
|
||||
.take()
|
||||
.ok_or_else(|| eyre!("Can't takeout stderr"))
|
||||
.with_kind(crate::ErrorKind::Docker)?,
|
||||
)
|
||||
.lines();
|
||||
Ok(NonDetachingJoinHandle::from(tokio::spawn(async move {
|
||||
while let Ok(Some(line)) = output.next_line().await {
|
||||
tracing::debug!("{:?}", id);
|
||||
tracing::error!("Error from long running container");
|
||||
tracing::error!("{}", line);
|
||||
}
|
||||
})))
|
||||
}
|
||||
|
||||
fn spawn_output_handle(
|
||||
handle: &mut Child,
|
||||
) -> Result<(UnboundedReceiver<OutputJsonRpc>, NonDetachingJoinHandle<()>), Error> {
|
||||
let mut output = tokio::io::BufReader::new(
|
||||
handle
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| eyre!("Can't takeout stdout for long running"))
|
||||
.with_kind(crate::ErrorKind::Docker)?,
|
||||
)
|
||||
.lines();
|
||||
let (sender, receiver) = tokio::sync::mpsc::unbounded_channel::<OutputJsonRpc>();
|
||||
Ok((
|
||||
receiver,
|
||||
NonDetachingJoinHandle::from(tokio::spawn(async move {
|
||||
loop {
|
||||
let next = output.next_line().await;
|
||||
let next = match next {
|
||||
Ok(Some(a)) => a,
|
||||
Ok(None) => {
|
||||
tracing::error!("The docker pipe is closed?");
|
||||
break;
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::debug!("{:?}", e);
|
||||
tracing::error!("Output from docker, killing");
|
||||
break;
|
||||
}
|
||||
};
|
||||
let next = match serde_json::from_str(&next) {
|
||||
Ok(a) => a,
|
||||
Err(_e) => {
|
||||
tracing::trace!("Could not decode output from long running binary");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
if let Err(e) = sender.send(next) {
|
||||
tracing::debug!("{:?}", e);
|
||||
tracing::error!("Could no longer send output");
|
||||
break;
|
||||
}
|
||||
}
|
||||
})),
|
||||
))
|
||||
}
|
||||
}
|
||||
async fn buf_reader_to_lines(
|
||||
reader: impl AsyncBufRead + Unpin,
|
||||
limit: impl Into<Option<usize>>,
|
||||
@@ -756,6 +1051,7 @@ async fn max_by_lines(
|
||||
}
|
||||
MaxByLines::Done(answer)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
pub use js_engine::JsError;
|
||||
use js_engine::{JsExecutionEnvironment, PathForVolumeId};
|
||||
use models::VolumeId;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use models::{ExecCommand, TermCommand};
|
||||
use serde::{de::DeserializeOwned, Deserialize, Serialize};
|
||||
use tracing::instrument;
|
||||
|
||||
use super::ProcedureName;
|
||||
@@ -52,8 +54,8 @@ impl JsProcedure {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[instrument(skip(directory, input))]
|
||||
pub async fn execute<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
#[instrument(skip(directory, input, exec_command, term_command))]
|
||||
pub async fn execute<I: Serialize, O: DeserializeOwned>(
|
||||
&self,
|
||||
directory: &PathBuf,
|
||||
pkg_id: &PackageId,
|
||||
@@ -62,6 +64,8 @@ impl JsProcedure {
|
||||
volumes: &Volumes,
|
||||
input: Option<I>,
|
||||
timeout: Option<Duration>,
|
||||
exec_command: ExecCommand,
|
||||
term_command: TermCommand,
|
||||
) -> Result<Result<O, (i32, String)>, Error> {
|
||||
Ok(async move {
|
||||
let running_action = JsExecutionEnvironment::load_from_package(
|
||||
@@ -69,6 +73,8 @@ impl JsProcedure {
|
||||
pkg_id,
|
||||
pkg_version,
|
||||
Box::new(volumes.clone()),
|
||||
exec_command,
|
||||
term_command,
|
||||
)
|
||||
.await?
|
||||
.run_action(name, input, self.args.clone());
|
||||
@@ -86,7 +92,7 @@ impl JsProcedure {
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, input))]
|
||||
pub async fn sandboxed<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
pub async fn sandboxed<I: Serialize, O: DeserializeOwned>(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
@@ -102,6 +108,12 @@ impl JsProcedure {
|
||||
pkg_id,
|
||||
pkg_version,
|
||||
Box::new(volumes.clone()),
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async { Err("Can't run commands in sandox mode".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
)
|
||||
.await?
|
||||
.read_only_effects()
|
||||
@@ -120,7 +132,7 @@ impl JsProcedure {
|
||||
}
|
||||
}
|
||||
|
||||
fn unwrap_known_error<O: for<'de> Deserialize<'de>>(
|
||||
fn unwrap_known_error<O: DeserializeOwned>(
|
||||
error_value: ErrorValue,
|
||||
) -> Result<O, (JsError, String)> {
|
||||
match error_value {
|
||||
@@ -181,6 +193,10 @@ async fn js_action_execute() {
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
@@ -236,6 +252,10 @@ async fn js_action_execute_error() {
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
@@ -280,11 +300,70 @@ async fn js_action_fetch() {
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn js_test_slow() {
|
||||
let js_action = JsProcedure { args: vec![] };
|
||||
let path: PathBuf = "test/js_action_execute/"
|
||||
.parse::<PathBuf>()
|
||||
.unwrap()
|
||||
.canonicalize()
|
||||
.unwrap();
|
||||
let package_id = "test-package".parse().unwrap();
|
||||
let package_version: Version = "0.3.0.3".parse().unwrap();
|
||||
let name = ProcedureName::Action("slow".parse().unwrap());
|
||||
let volumes: Volumes = serde_json::from_value(serde_json::json!({
|
||||
"main": {
|
||||
"type": "data"
|
||||
},
|
||||
"compat": {
|
||||
"type": "assets"
|
||||
},
|
||||
"filebrowser" :{
|
||||
"package-id": "filebrowser",
|
||||
"path": "data",
|
||||
"readonly": true,
|
||||
"type": "pointer",
|
||||
"volume-id": "main",
|
||||
}
|
||||
}))
|
||||
.unwrap();
|
||||
let input: Option<serde_json::Value> = None;
|
||||
let timeout = Some(Duration::from_secs(10));
|
||||
tracing::debug!("testing start");
|
||||
tokio::select! {
|
||||
a = js_action
|
||||
.execute::<serde_json::Value, serde_json::Value>(
|
||||
&path,
|
||||
&package_id,
|
||||
&package_version,
|
||||
name,
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
=> {a
|
||||
.unwrap()
|
||||
.unwrap();},
|
||||
_ = tokio::time::sleep(Duration::from_secs(1)) => ()
|
||||
}
|
||||
tracing::debug!("testing end should");
|
||||
tokio::time::sleep(Duration::from_secs(2)).await;
|
||||
tracing::debug!("Done");
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn js_action_var_arg() {
|
||||
let js_action = JsProcedure {
|
||||
@@ -325,6 +404,10 @@ async fn js_action_var_arg() {
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
@@ -369,6 +452,10 @@ async fn js_action_test_rename() {
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
@@ -413,6 +500,10 @@ async fn js_action_test_deep_dir() {
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
@@ -456,6 +547,10 @@ async fn js_action_test_deep_dir_escape() {
|
||||
&volumes,
|
||||
input,
|
||||
timeout,
|
||||
Arc::new(|_, _, _, _| {
|
||||
Box::pin(async move { Err("Can't run commands in test".to_string()) })
|
||||
}),
|
||||
Arc::new(|_| Box::pin(async move { Err("Can't run commands in test".to_string()) })),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
use std::collections::BTreeSet;
|
||||
use std::time::Duration;
|
||||
|
||||
use color_eyre::eyre::{bail, eyre};
|
||||
use color_eyre::eyre::eyre;
|
||||
use patch_db::HasModel;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde::{de::DeserializeOwned, Deserialize, Serialize};
|
||||
use tracing::instrument;
|
||||
|
||||
use self::docker::{DockerContainer, DockerInject, DockerProcedure};
|
||||
use self::docker::{DockerContainers, DockerProcedure};
|
||||
use crate::context::RpcContext;
|
||||
use crate::id::ImageId;
|
||||
use crate::s9pk::manifest::PackageId;
|
||||
use crate::util::Version;
|
||||
use crate::volume::Volumes;
|
||||
use crate::Error;
|
||||
use crate::{Error, ErrorKind};
|
||||
|
||||
pub mod docker;
|
||||
#[cfg(feature = "js_engine")]
|
||||
@@ -26,7 +26,6 @@ pub use models::ProcedureName;
|
||||
#[serde(tag = "type")]
|
||||
pub enum PackageProcedure {
|
||||
Docker(DockerProcedure),
|
||||
DockerInject(DockerInject),
|
||||
|
||||
#[cfg(feature = "js_engine")]
|
||||
Script(js_scripts::JsProcedure),
|
||||
@@ -43,7 +42,7 @@ impl PackageProcedure {
|
||||
#[instrument]
|
||||
pub fn validate(
|
||||
&self,
|
||||
container: &Option<DockerContainer>,
|
||||
container: &Option<DockerContainers>,
|
||||
eos_version: &Version,
|
||||
volumes: &Volumes,
|
||||
image_ids: &BTreeSet<ImageId>,
|
||||
@@ -53,25 +52,15 @@ impl PackageProcedure {
|
||||
PackageProcedure::Docker(action) => {
|
||||
action.validate(eos_version, volumes, image_ids, expected_io)
|
||||
}
|
||||
PackageProcedure::DockerInject(injectable) => {
|
||||
let container = match container {
|
||||
None => bail!("For the docker injectable procedure, a container must be exist on the config"),
|
||||
Some(container) => container,
|
||||
} ;
|
||||
let docker_procedure: DockerProcedure = (container, injectable).into();
|
||||
docker_procedure.validate(eos_version, volumes, image_ids, expected_io)
|
||||
}
|
||||
|
||||
#[cfg(feature = "js_engine")]
|
||||
PackageProcedure::Script(action) => action.validate(volumes),
|
||||
}
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, input, container))]
|
||||
pub async fn execute<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
#[instrument(skip(ctx, input))]
|
||||
pub async fn execute<I: Serialize, O: DeserializeOwned + 'static>(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
container: &Option<DockerContainer>,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
name: ProcedureName,
|
||||
@@ -86,18 +75,36 @@ impl PackageProcedure {
|
||||
.execute(ctx, pkg_id, pkg_version, name, volumes, input, timeout)
|
||||
.await
|
||||
}
|
||||
PackageProcedure::DockerInject(injectable) => {
|
||||
let container = match container {
|
||||
None => return Err(Error::new(eyre!("For the docker injectable procedure, a container must be exist on the config"), crate::ErrorKind::Action)),
|
||||
Some(container) => container,
|
||||
} ;
|
||||
let docker_procedure: DockerProcedure = (container, injectable).into();
|
||||
docker_procedure
|
||||
.inject(ctx, pkg_id, pkg_version, name, volumes, input, timeout)
|
||||
.await
|
||||
}
|
||||
#[cfg(feature = "js_engine")]
|
||||
PackageProcedure::Script(procedure) => {
|
||||
let exec_command = match ctx
|
||||
.managers
|
||||
.get(&(pkg_id.clone(), pkg_version.clone()))
|
||||
.await
|
||||
{
|
||||
None => {
|
||||
return Err(Error::new(
|
||||
eyre!("No manager found for {}", pkg_id),
|
||||
ErrorKind::NotFound,
|
||||
))
|
||||
}
|
||||
Some(x) => x,
|
||||
}
|
||||
.exec_command();
|
||||
let term_command = match ctx
|
||||
.managers
|
||||
.get(&(pkg_id.clone(), pkg_version.clone()))
|
||||
.await
|
||||
{
|
||||
None => {
|
||||
return Err(Error::new(
|
||||
eyre!("No manager found for {}", pkg_id),
|
||||
ErrorKind::NotFound,
|
||||
))
|
||||
}
|
||||
Some(x) => x,
|
||||
}
|
||||
.term_command();
|
||||
procedure
|
||||
.execute(
|
||||
&ctx.datadir,
|
||||
@@ -107,17 +114,18 @@ impl PackageProcedure {
|
||||
volumes,
|
||||
input,
|
||||
timeout,
|
||||
exec_command,
|
||||
term_command,
|
||||
)
|
||||
.await
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, input, container))]
|
||||
pub async fn inject<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
#[instrument(skip(ctx, input))]
|
||||
pub async fn inject<I: Serialize, O: DeserializeOwned + 'static>(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
container: &Option<DockerContainer>,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
name: ProcedureName,
|
||||
@@ -125,25 +133,42 @@ impl PackageProcedure {
|
||||
input: Option<I>,
|
||||
timeout: Option<Duration>,
|
||||
) -> Result<Result<O, (i32, String)>, Error> {
|
||||
tracing::trace!("Procedure inject {} {} - {:?}", self, pkg_id, name);
|
||||
match self {
|
||||
PackageProcedure::Docker(procedure) => {
|
||||
procedure
|
||||
.inject(ctx, pkg_id, pkg_version, name, volumes, input, timeout)
|
||||
.await
|
||||
}
|
||||
PackageProcedure::DockerInject(injectable) => {
|
||||
let container = match container {
|
||||
None => return Err(Error::new(eyre!("For the docker injectable procedure, a container must be exist on the config"), crate::ErrorKind::Action)),
|
||||
Some(container) => container,
|
||||
} ;
|
||||
let docker_procedure: DockerProcedure = (container, injectable).into();
|
||||
docker_procedure
|
||||
.inject(ctx, pkg_id, pkg_version, name, volumes, input, timeout)
|
||||
.await
|
||||
}
|
||||
#[cfg(feature = "js_engine")]
|
||||
PackageProcedure::Script(procedure) => {
|
||||
let exec_command = match ctx
|
||||
.managers
|
||||
.get(&(pkg_id.clone(), pkg_version.clone()))
|
||||
.await
|
||||
{
|
||||
None => {
|
||||
return Err(Error::new(
|
||||
eyre!("No manager found for {}", pkg_id),
|
||||
ErrorKind::NotFound,
|
||||
))
|
||||
}
|
||||
Some(x) => x,
|
||||
}
|
||||
.exec_command();
|
||||
let term_command = match ctx
|
||||
.managers
|
||||
.get(&(pkg_id.clone(), pkg_version.clone()))
|
||||
.await
|
||||
{
|
||||
None => {
|
||||
return Err(Error::new(
|
||||
eyre!("No manager found for {}", pkg_id),
|
||||
ErrorKind::NotFound,
|
||||
))
|
||||
}
|
||||
Some(x) => x,
|
||||
}
|
||||
.term_command();
|
||||
procedure
|
||||
.execute(
|
||||
&ctx.datadir,
|
||||
@@ -153,15 +178,17 @@ impl PackageProcedure {
|
||||
volumes,
|
||||
input,
|
||||
timeout,
|
||||
exec_command,
|
||||
term_command,
|
||||
)
|
||||
.await
|
||||
}
|
||||
}
|
||||
}
|
||||
#[instrument(skip(ctx, input))]
|
||||
pub async fn sandboxed<I: Serialize, O: for<'de> Deserialize<'de>>(
|
||||
pub async fn sandboxed<I: Serialize, O: DeserializeOwned>(
|
||||
&self,
|
||||
container: &Option<DockerContainer>,
|
||||
container: &Option<DockerContainers>,
|
||||
ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
@@ -177,16 +204,6 @@ impl PackageProcedure {
|
||||
.sandboxed(ctx, pkg_id, pkg_version, volumes, input, timeout)
|
||||
.await
|
||||
}
|
||||
PackageProcedure::DockerInject(injectable) => {
|
||||
let container = match container {
|
||||
None => return Err(Error::new(eyre!("For the docker injectable procedure, a container must be exist on the config"), crate::ErrorKind::Action)),
|
||||
Some(container) => container,
|
||||
} ;
|
||||
let docker_procedure: DockerProcedure = (container, injectable).into();
|
||||
docker_procedure
|
||||
.sandboxed(ctx, pkg_id, pkg_version, volumes, input, timeout)
|
||||
.await
|
||||
}
|
||||
#[cfg(feature = "js_engine")]
|
||||
PackageProcedure::Script(procedure) => {
|
||||
procedure
|
||||
@@ -200,7 +217,6 @@ impl PackageProcedure {
|
||||
impl std::fmt::Display for PackageProcedure {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
PackageProcedure::DockerInject(_) => write!(f, "Docker Injectable")?,
|
||||
PackageProcedure::Docker(_) => write!(f, "Docker")?,
|
||||
#[cfg(feature = "js_engine")]
|
||||
PackageProcedure::Script(_) => write!(f, "JS")?,
|
||||
@@ -208,6 +224,7 @@ impl std::fmt::Display for PackageProcedure {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct NoOutput;
|
||||
impl<'de> Deserialize<'de> for NoOutput {
|
||||
|
||||
Reference in New Issue
Block a user