mirror of
https://github.com/Start9Labs/start-os.git
synced 2026-04-01 21:13:09 +00:00
Feat/js long running (#1879)
* wip: combining the streams * chore: Testing locally * chore: Fix some lint * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * wip: making the mananger create * wip: Working on trying to make the long running docker container command * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * feat: Use the long running feature in the manager * remove recovered services and drop reordering feature (#1829) * wip: Need to get the initial docker command running? * chore: Add in the new procedure for the docker. * feat: Get the system to finally run long * wip: Added the command inserter to the docker persistance * wip: Added the command inserter to the docker persistance * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * remove recovered services and drop reordering feature (#1829) * chore: Convert the migration to use receipt. (#1842) * feat: remove ionic storage (#1839) * feat: remove ionic storage * grayscal when disconncted, rename local storage service for clarity * remove storage from package lock * update patchDB Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> * update patchDB * feat: Move the run_command into the js * Feat/long running (#1676) * feat: Start the long running container * feat: Long running docker, running, stoping, and uninstalling * feat: Just make the folders that we would like to mount. * fix: Uninstall not working * chore: remove some logging * feat: Smarter cleanup * feat: Wait for start * wip: Need to kill * chore: Remove the bad tracing * feat: Stopping the long running processes without killing the long running * Mino Feat: Change the Manifest To have a new type (#1736) * Add build-essential to README.md (#1716) Update README.md * write image to sparse-aware archive format (#1709) * fix: Add modification to the max_user_watches (#1695) * fix: Add modification to the max_user_watches * chore: Move to initialization * [Feat] follow logs (#1714) * tail logs * add cli * add FE * abstract http to shared * batch new logs * file download for logs * fix modal error when no config Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: BluJ <mogulslayer@gmail.com> * Update README.md (#1728) * fix build for patch-db client for consistency (#1722) * fix cli install (#1720) * highlight instructions if not viewed (#1731) * wip: * [ ] Fix the build (dependencies:634 map for option) * fix: Cargo build * fix: Long running wasn't starting * fix: uninstall works Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * chore: Fix a dbg! * chore: Make the commands of the docker-inject do inject instead of exec * chore: Fix compile mistake * chore: Change to use simpler Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> * remove recovered services and drop reordering feature (#1829) * chore: Convert the migration to use receipt. (#1842) * feat: remove ionic storage (#1839) * feat: remove ionic storage * grayscal when disconncted, rename local storage service for clarity * remove storage from package lock * update patchDB Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> * update patch DB * chore: Change the error catching for the long running to try all * Feat/community marketplace (#1790) * add community marketplace * Update embassy-mock-api.service.ts * expect ui/marketplace to be undefined * possible undefined from getpackage * fix marketplace pages * rework marketplace infrastructure * fix bugs Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> * WIP: Fix the build, needed to move around creation of exec * wip: Working on solving why there is a missing end. * fix: make `shared` module independent of `config.js` (#1870) * feat: Add in the kill and timeout * feat: Get the run to actually work. * chore: Add when/ why/ where comments * feat: Convert inject main to use exec main. * Fix: Ability to stop services * wip: long running js main * feat: Kill for the main * Fix * fix: Fix the build for x86 * wip: Working on changes * wip: Working on trying to kill js * fix: Testing for slow * feat: Test that the new manifest works * chore: Try and fix build? * chore: Fix? the build * chore: Fix the long input dies and never restarts * build improvements * no workdir * fix: Architecture for long running * chore: Fix and remove the docker inject * chore: Undo the changes to the kiosk mode * fix: Remove the it from the prod build * fix: Start issue * fix: The compat build * chore: Add in the conditional compilation again for the missing impl * chore: Change to aux * chore: Remove the aux for now * chore: Add some documentation to docker container Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com> Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <matthewonthemoon@gmail.com> Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com> Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com> Co-authored-by: Alex Inkin <alexander@inkin.ru>
This commit is contained in:
J M
@@ -10,7 +10,7 @@ use super::{Config, ConfigSpec};
|
||||
use crate::context::RpcContext;
|
||||
use crate::dependencies::Dependencies;
|
||||
use crate::id::ImageId;
|
||||
use crate::procedure::docker::DockerContainer;
|
||||
use crate::procedure::docker::DockerContainers;
|
||||
use crate::procedure::{PackageProcedure, ProcedureName};
|
||||
use crate::s9pk::manifest::PackageId;
|
||||
use crate::status::health_check::HealthCheckId;
|
||||
@@ -34,7 +34,7 @@ impl ConfigActions {
|
||||
#[instrument]
|
||||
pub fn validate(
|
||||
&self,
|
||||
container: &Option<DockerContainer>,
|
||||
container: &Option<DockerContainers>,
|
||||
eos_version: &Version,
|
||||
volumes: &Volumes,
|
||||
image_ids: &BTreeSet<ImageId>,
|
||||
@@ -51,7 +51,6 @@ impl ConfigActions {
|
||||
pub async fn get(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
container: &Option<DockerContainer>,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
volumes: &Volumes,
|
||||
@@ -59,7 +58,6 @@ impl ConfigActions {
|
||||
self.get
|
||||
.execute(
|
||||
ctx,
|
||||
container,
|
||||
pkg_id,
|
||||
pkg_version,
|
||||
ProcedureName::GetConfig,
|
||||
@@ -77,7 +75,6 @@ impl ConfigActions {
|
||||
pub async fn set(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
container: &Option<DockerContainer>,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
dependencies: &Dependencies,
|
||||
@@ -88,7 +85,6 @@ impl ConfigActions {
|
||||
.set
|
||||
.execute(
|
||||
ctx,
|
||||
container,
|
||||
pkg_id,
|
||||
pkg_version,
|
||||
ProcedureName::SetConfig,
|
||||
|
||||
@@ -21,7 +21,7 @@ use crate::dependencies::{
|
||||
DependencyErrors, DependencyReceipt, TaggedDependencyError, TryHealReceipts,
|
||||
};
|
||||
use crate::install::cleanup::{remove_from_current_dependents_lists, UpdateDependencyReceipts};
|
||||
use crate::procedure::docker::DockerContainer;
|
||||
use crate::procedure::docker::DockerContainers;
|
||||
use crate::s9pk::manifest::{Manifest, PackageId};
|
||||
use crate::util::display_none;
|
||||
use crate::util::serde::{display_serializable, parse_stdin_deserializable, IoFormat};
|
||||
@@ -168,7 +168,6 @@ pub struct ConfigGetReceipts {
|
||||
manifest_volumes: LockReceipt<crate::volume::Volumes, ()>,
|
||||
manifest_version: LockReceipt<crate::util::Version, ()>,
|
||||
manifest_config: LockReceipt<Option<ConfigActions>, ()>,
|
||||
docker_container: LockReceipt<DockerContainer, String>,
|
||||
}
|
||||
|
||||
impl ConfigGetReceipts {
|
||||
@@ -204,19 +203,11 @@ impl ConfigGetReceipts {
|
||||
.map(|x| x.manifest().config())
|
||||
.make_locker(LockType::Write)
|
||||
.add_to_keys(locks);
|
||||
let docker_container = crate::db::DatabaseModel::new()
|
||||
.package_data()
|
||||
.star()
|
||||
.installed()
|
||||
.and_then(|x| x.manifest().container())
|
||||
.make_locker(LockType::Write)
|
||||
.add_to_keys(locks);
|
||||
move |skeleton_key| {
|
||||
Ok(Self {
|
||||
manifest_volumes: manifest_volumes.verify(skeleton_key)?,
|
||||
manifest_version: manifest_version.verify(skeleton_key)?,
|
||||
manifest_config: manifest_config.verify(skeleton_key)?,
|
||||
docker_container: docker_container.verify(skeleton_key)?,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -239,11 +230,9 @@ pub async fn get(
|
||||
.await?
|
||||
.ok_or_else(|| Error::new(eyre!("{} has no config", id), crate::ErrorKind::NotFound))?;
|
||||
|
||||
let container = receipts.docker_container.get(&mut db, &id).await?;
|
||||
|
||||
let volumes = receipts.manifest_volumes.get(&mut db).await?;
|
||||
let version = receipts.manifest_version.get(&mut db).await?;
|
||||
action.get(&ctx, &container, &id, &version, &volumes).await
|
||||
action.get(&ctx, &id, &version, &volumes).await
|
||||
}
|
||||
|
||||
#[command(
|
||||
@@ -286,7 +275,7 @@ pub struct ConfigReceipts {
|
||||
pub current_dependencies: LockReceipt<CurrentDependencies, String>,
|
||||
dependency_errors: LockReceipt<DependencyErrors, String>,
|
||||
manifest_dependencies_config: LockReceipt<DependencyConfig, (String, String)>,
|
||||
docker_container: LockReceipt<DockerContainer, String>,
|
||||
docker_containers: LockReceipt<DockerContainers, String>,
|
||||
}
|
||||
|
||||
impl ConfigReceipts {
|
||||
@@ -391,11 +380,11 @@ impl ConfigReceipts {
|
||||
.and_then(|x| x.manifest().dependencies().star().config())
|
||||
.make_locker(LockType::Write)
|
||||
.add_to_keys(locks);
|
||||
let docker_container = crate::db::DatabaseModel::new()
|
||||
let docker_containers = crate::db::DatabaseModel::new()
|
||||
.package_data()
|
||||
.star()
|
||||
.installed()
|
||||
.and_then(|x| x.manifest().container())
|
||||
.and_then(|x| x.manifest().containers())
|
||||
.make_locker(LockType::Write)
|
||||
.add_to_keys(locks);
|
||||
|
||||
@@ -417,7 +406,7 @@ impl ConfigReceipts {
|
||||
current_dependencies: current_dependencies.verify(skeleton_key)?,
|
||||
dependency_errors: dependency_errors.verify(skeleton_key)?,
|
||||
manifest_dependencies_config: manifest_dependencies_config.verify(skeleton_key)?,
|
||||
docker_container: docker_container.verify(skeleton_key)?,
|
||||
docker_containers: docker_containers.verify(skeleton_key)?,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -509,8 +498,6 @@ pub fn configure_rec<'a, Db: DbHandle>(
|
||||
receipts: &'a ConfigReceipts,
|
||||
) -> BoxFuture<'a, Result<(), Error>> {
|
||||
async move {
|
||||
let container = receipts.docker_container.get(db, &id).await?;
|
||||
let container = &container;
|
||||
// fetch data from db
|
||||
let action = receipts
|
||||
.config_actions
|
||||
@@ -534,7 +521,7 @@ pub fn configure_rec<'a, Db: DbHandle>(
|
||||
let ConfigRes {
|
||||
config: old_config,
|
||||
spec,
|
||||
} = action.get(ctx, container, id, &version, &volumes).await?;
|
||||
} = action.get(ctx, id, &version, &volumes).await?;
|
||||
|
||||
// determine new config to use
|
||||
let mut config = if let Some(config) = config.or_else(|| old_config.clone()) {
|
||||
@@ -602,15 +589,7 @@ pub fn configure_rec<'a, Db: DbHandle>(
|
||||
let signal = if !dry_run {
|
||||
// run config action
|
||||
let res = action
|
||||
.set(
|
||||
ctx,
|
||||
container,
|
||||
id,
|
||||
&version,
|
||||
&dependencies,
|
||||
&volumes,
|
||||
&config,
|
||||
)
|
||||
.set(ctx, id, &version, &dependencies, &volumes, &config)
|
||||
.await?;
|
||||
|
||||
// track dependencies with no pointers
|
||||
@@ -702,7 +681,7 @@ pub fn configure_rec<'a, Db: DbHandle>(
|
||||
.unwrap_or_default();
|
||||
let next = Value::Object(config.clone());
|
||||
for (dependent, dep_info) in dependents.0.iter().filter(|(dep_id, _)| dep_id != &id) {
|
||||
let dependent_container = receipts.docker_container.get(db, &dependent).await?;
|
||||
let dependent_container = receipts.docker_containers.get(db, &dependent).await?;
|
||||
let dependent_container = &dependent_container;
|
||||
// check if config passes dependent check
|
||||
if let Some(cfg) = receipts
|
||||
|
||||
@@ -25,7 +25,6 @@ use super::{Config, MatchError, NoMatchWithPath, TimeoutError, TypeOf};
|
||||
use crate::config::ConfigurationError;
|
||||
use crate::context::RpcContext;
|
||||
use crate::net::interface::InterfaceId;
|
||||
use crate::procedure::docker::DockerContainer;
|
||||
use crate::s9pk::manifest::{Manifest, PackageId};
|
||||
use crate::Error;
|
||||
|
||||
@@ -1883,7 +1882,6 @@ pub struct ConfigPointerReceipts {
|
||||
manifest_volumes: LockReceipt<crate::volume::Volumes, String>,
|
||||
manifest_version: LockReceipt<crate::util::Version, String>,
|
||||
config_actions: LockReceipt<super::action::ConfigActions, String>,
|
||||
docker_container: LockReceipt<DockerContainer, String>,
|
||||
}
|
||||
|
||||
impl ConfigPointerReceipts {
|
||||
@@ -1920,20 +1918,12 @@ impl ConfigPointerReceipts {
|
||||
.and_then(|x| x.manifest().config())
|
||||
.make_locker(LockType::Read)
|
||||
.add_to_keys(locks);
|
||||
let docker_container = crate::db::DatabaseModel::new()
|
||||
.package_data()
|
||||
.star()
|
||||
.installed()
|
||||
.and_then(|x| x.manifest().container())
|
||||
.make_locker(LockType::Write)
|
||||
.add_to_keys(locks);
|
||||
move |skeleton_key| {
|
||||
Ok(Self {
|
||||
interface_addresses_receipt: interface_addresses_receipt(skeleton_key)?,
|
||||
manifest_volumes: manifest_volumes.verify(skeleton_key)?,
|
||||
config_actions: config_actions.verify(skeleton_key)?,
|
||||
manifest_version: manifest_version.verify(skeleton_key)?,
|
||||
docker_container: docker_container.verify(skeleton_key)?,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1963,12 +1953,11 @@ impl ConfigPointer {
|
||||
let version = receipts.manifest_version.get(db, id).await.ok().flatten();
|
||||
let cfg_actions = receipts.config_actions.get(db, id).await.ok().flatten();
|
||||
let volumes = receipts.manifest_volumes.get(db, id).await.ok().flatten();
|
||||
let container = receipts.docker_container.get(db, id).await.ok().flatten();
|
||||
if let (Some(version), Some(cfg_actions), Some(volumes)) =
|
||||
(&version, &cfg_actions, &volumes)
|
||||
{
|
||||
let cfg_res = cfg_actions
|
||||
.get(ctx, &container, &self.package_id, version, volumes)
|
||||
.get(ctx, &self.package_id, version, volumes)
|
||||
.await
|
||||
.map_err(|e| ConfigurationError::SystemError(e))?;
|
||||
if let Some(cfg) = cfg_res.config {
|
||||
|
||||
Reference in New Issue
Block a user