mirror of
https://github.com/Start9Labs/start-os.git
synced 2026-04-01 21:13:09 +00:00
Feature/lxc container runtime (#2562)
* wip(fix): Dependencies * wip: context * wip(fix) Sorta auth * wip: warnings * wip(fix): registry/admin * wip(fix) marketplace * wip(fix) Some more converted and fixed with the linter and config * wip: Working on the static server * wip(fix)static server * wip: Remove some asynnc * wip: Something about the request and regular rpc * wip: gut install Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: Convert the static server into the new system * wip delete file * test * wip(fix) vhost does not need the with safe defaults * wip: Adding in the wifi * wip: Fix the developer and the verify * wip: new install flow Co-authored-by: J H <Blu-J@users.noreply.github.com> * fix middleware * wip * wip: Fix the auth * wip * continue service refactor * feature: Service get_config * feat: Action * wip: Fighting the great fight against the borrow checker * wip: Remove an error in a file that I just need to deel with later * chore: Add in some more lifetime stuff to the services * wip: Install fix on lifetime * cleanup * wip: Deal with the borrow later * more cleanup * resolve borrowchecker errors * wip(feat): add in the handler for the socket, for now * wip(feat): Update the service_effect_handler::action * chore: Add in the changes to make sure the from_service goes to context * chore: Change the * refactor service map * fix references to service map * fill out restore * wip: Before I work on the store stuff * fix backup module * handle some warnings * feat: add in the ui components on the rust side * feature: Update the procedures * chore: Update the js side of the main and a few of the others * chore: Update the rpc listener to match the persistant container * wip: Working on updating some things to have a better name * wip(feat): Try and get the rpc to return the correct shape? * lxc wip * wip(feat): Try and get the rpc to return the correct shape? * build for container runtime wip * remove container-init * fix build * fix error * chore: Update to work I suppose * lxc wip * remove docker module and feature * download alpine squashfs automatically * overlays effect Co-authored-by: Jade <Blu-J@users.noreply.github.com> * chore: Add the overlay effect * feat: Add the mounter in the main * chore: Convert to use the mounts, still need to work with the sandbox * install fixes * fix ssl * fixes from testing * implement tmpfile for upload * wip * misc fixes * cleanup * cleanup * better progress reporting * progress for sideload * return real guid * add devmode script * fix lxc rootfs path * fix percentage bar * fix progress bar styling * fix build for unstable * tweaks * label progress * tweaks * update progress more often * make symlink in rpc_client * make socket dir * fix parent path * add start-cli to container * add echo and gitInfo commands * wip: Add the init + errors * chore: Add in the exit effect for the system * chore: Change the type to null for failure to parse * move sigterm timeout to stopping status * update order * chore: Update the return type * remove dbg * change the map error * chore: Update the thing to capture id * chore add some life changes * chore: Update the loging * chore: Update the package to run module * us From for RpcError * chore: Update to use import instead * chore: update * chore: Use require for the backup * fix a default * update the type that is wrong * chore: Update the type of the manifest * chore: Update to make null * only symlink if not exists * get rid of double result * better debug info for ErrorCollection * chore: Update effects * chore: fix * mount assets and volumes * add exec instead of spawn * fix mounting in image * fix overlay mounts Co-authored-by: Jade <Blu-J@users.noreply.github.com> * misc fixes * feat: Fix two * fix: systemForEmbassy main * chore: Fix small part of main loop * chore: Modify the bundle * merge * fixMain loop" * move tsc to makefile * chore: Update the return types of the health check * fix client * chore: Convert the todo to use tsmatches * add in the fixes for the seen and create the hack to allow demo * chore: Update to include the systemForStartOs * chore UPdate to the latest types from the expected outout * fixes * fix typo * Don't emit if failure on tsc * wip Co-authored-by: Jade <Blu-J@users.noreply.github.com> * add s9pk api * add inspection * add inspect manifest * newline after display serializable * fix squashfs in image name * edit manifest Co-authored-by: Jade <Blu-J@users.noreply.github.com> * wait for response on repl * ignore sig for now * ignore sig for now * re-enable sig verification * fix * wip * env and chroot * add profiling logs * set uid & gid in squashfs to 100000 * set uid of sqfs to 100000 * fix mksquashfs args * add env to compat * fix * re-add docker feature flag * fix docker output format being stupid * here be dragons * chore: Add in the cross compiling for something * fix npm link * extract logs from container on exit * chore: Update for testing * add log capture to drop trait * chore: add in the modifications that I make * chore: Update small things for no updates * chore: Update the types of something * chore: Make main not complain * idmapped mounts * idmapped volumes * re-enable kiosk * chore: Add in some logging for the new system * bring in start-sdk * remove avahi * chore: Update the deps * switch to musl * chore: Update the version of prettier * chore: Organize' * chore: Update some of the headers back to the standard of fetch * fix musl build * fix idmapped mounts * fix cross build * use cross compiler for correct arch * feat: Add in the faked ssl stuff for the effects * @dr_bonez Did a solution here * chore: Something that DrBonez * chore: up * wip: We have a working server!!! * wip * uninstall * wip * tes * misc fixes * fix cli * replace interface with host * chore: Fix the types in some ts files * chore: quick update for the system for embassy to update the types * replace br-start9 with lxcbr0 * split patchdb into public/private * chore: Add changes for config set * Feat: Adding some debugging for the errors * wip: Working on getting the set config to work * chore: Update and fix the small issue with the deserialization * lightning, masked, schemeOverride, invert host-iface relationship * feat: Add in the changes for just the sdk * feat: Add in the changes for the new effects I suppose for now --------- Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com> Co-authored-by: J H <Blu-J@users.noreply.github.com> Co-authored-by: J H <dragondef@gmail.com> Co-authored-by: Matt Hill <mattnine@protonmail.com>
This commit is contained in:
Aiden McClelland
@@ -11,7 +11,7 @@ use clap::Parser;
|
||||
use futures::{FutureExt, StreamExt};
|
||||
use http::header::COOKIE;
|
||||
use http::HeaderMap;
|
||||
use patch_db::json_ptr::JsonPointer;
|
||||
use patch_db::json_ptr::{JsonPointer, ROOT};
|
||||
use patch_db::{Dump, Revision};
|
||||
use rpc_toolkit::yajrc::RpcError;
|
||||
use rpc_toolkit::{command, from_fn_async, CallRemote, HandlerExt, ParentHandler};
|
||||
@@ -25,13 +25,17 @@ use crate::middleware::auth::{HasValidSession, HashSessionToken};
|
||||
use crate::prelude::*;
|
||||
use crate::util::serde::{apply_expr, HandlerExtSerde};
|
||||
|
||||
lazy_static::lazy_static! {
|
||||
static ref PUBLIC: JsonPointer = "/public".parse().unwrap();
|
||||
}
|
||||
|
||||
#[instrument(skip_all)]
|
||||
async fn ws_handler(
|
||||
ctx: RpcContext,
|
||||
session: Option<(HasValidSession, HashSessionToken)>,
|
||||
mut stream: WebSocket,
|
||||
) -> Result<(), Error> {
|
||||
let (dump, sub) = ctx.db.dump_and_sub().await;
|
||||
let (dump, sub) = ctx.db.dump_and_sub(PUBLIC.clone()).await;
|
||||
|
||||
if let Some((session, token)) = session {
|
||||
let kill = subscribe_to_session_kill(&ctx, token).await;
|
||||
@@ -181,7 +185,7 @@ pub enum RevisionsRes {
|
||||
#[instrument(skip_all)]
|
||||
async fn cli_dump(ctx: CliContext, DumpParams { path }: DumpParams) -> Result<Dump, RpcError> {
|
||||
let dump = if let Some(path) = path {
|
||||
PatchDb::open(path).await?.dump().await
|
||||
PatchDb::open(path).await?.dump(&ROOT).await
|
||||
} else {
|
||||
from_value::<Dump>(ctx.call_remote("db.dump", imbl_value::json!({})).await?)?
|
||||
};
|
||||
@@ -201,7 +205,7 @@ pub struct DumpParams {
|
||||
// display(display_serializable)
|
||||
// )]
|
||||
pub async fn dump(ctx: RpcContext, _: DumpParams) -> Result<Dump, Error> {
|
||||
Ok(ctx.db.dump().await)
|
||||
Ok(ctx.db.dump(&*PUBLIC).await)
|
||||
}
|
||||
|
||||
#[instrument(skip_all)]
|
||||
|
||||
@@ -7,7 +7,7 @@ use imbl_value::InternedString;
|
||||
use ipnet::{Ipv4Net, Ipv6Net};
|
||||
use isocountry::CountryCode;
|
||||
use itertools::Itertools;
|
||||
use models::{DataUrl, HealthCheckId, InterfaceId, PackageId};
|
||||
use models::{DataUrl, HealthCheckId, HostId, PackageId};
|
||||
use openssl::hash::MessageDigest;
|
||||
use patch_db::json_ptr::JsonPointer;
|
||||
use patch_db::{HasModel, Value};
|
||||
@@ -16,7 +16,6 @@ use serde::{Deserialize, Serialize};
|
||||
use ssh_key::public::Ed25519PublicKey;
|
||||
|
||||
use crate::account::AccountInfo;
|
||||
use crate::config::spec::PackagePointerSpec;
|
||||
use crate::net::utils::{get_iface_ipv4_addr, get_iface_ipv6_addr};
|
||||
use crate::prelude::*;
|
||||
use crate::progress::FullProgress;
|
||||
@@ -30,72 +29,85 @@ use crate::{ARCH, PLATFORM};
|
||||
#[derive(Debug, Deserialize, Serialize, HasModel)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[model = "Model<Self>"]
|
||||
// #[macro_debug]
|
||||
pub struct Database {
|
||||
pub server_info: ServerInfo,
|
||||
pub package_data: AllPackageData,
|
||||
pub ui: Value,
|
||||
pub public: Public,
|
||||
pub private: (), // TODO
|
||||
}
|
||||
impl Database {
|
||||
pub fn init(account: &AccountInfo) -> Self {
|
||||
let lan_address = account.hostname.lan_address().parse().unwrap();
|
||||
Database {
|
||||
server_info: ServerInfo {
|
||||
arch: get_arch(),
|
||||
platform: get_platform(),
|
||||
id: account.server_id.clone(),
|
||||
version: Current::new().semver().into(),
|
||||
hostname: account.hostname.no_dot_host_name(),
|
||||
last_backup: None,
|
||||
last_wifi_region: None,
|
||||
eos_version_compat: Current::new().compat().clone(),
|
||||
lan_address,
|
||||
tor_address: format!("https://{}", account.key.tor_address())
|
||||
.parse()
|
||||
.unwrap(),
|
||||
ip_info: BTreeMap::new(),
|
||||
status_info: ServerStatus {
|
||||
backup_progress: None,
|
||||
updated: false,
|
||||
update_progress: None,
|
||||
shutting_down: false,
|
||||
restarting: false,
|
||||
},
|
||||
wifi: WifiInfo {
|
||||
ssids: Vec::new(),
|
||||
connected: None,
|
||||
selected: None,
|
||||
},
|
||||
unread_notification_count: 0,
|
||||
connection_addresses: ConnectionAddresses {
|
||||
tor: Vec::new(),
|
||||
clearnet: Vec::new(),
|
||||
},
|
||||
password_hash: account.password.clone(),
|
||||
pubkey: ssh_key::PublicKey::from(Ed25519PublicKey::from(&account.key.ssh_key()))
|
||||
public: Public {
|
||||
server_info: ServerInfo {
|
||||
arch: get_arch(),
|
||||
platform: get_platform(),
|
||||
id: account.server_id.clone(),
|
||||
version: Current::new().semver().into(),
|
||||
hostname: account.hostname.no_dot_host_name(),
|
||||
last_backup: None,
|
||||
last_wifi_region: None,
|
||||
eos_version_compat: Current::new().compat().clone(),
|
||||
lan_address,
|
||||
tor_address: format!("https://{}", account.key.tor_address())
|
||||
.parse()
|
||||
.unwrap(),
|
||||
ip_info: BTreeMap::new(),
|
||||
status_info: ServerStatus {
|
||||
backup_progress: None,
|
||||
updated: false,
|
||||
update_progress: None,
|
||||
shutting_down: false,
|
||||
restarting: false,
|
||||
},
|
||||
wifi: WifiInfo {
|
||||
ssids: Vec::new(),
|
||||
connected: None,
|
||||
selected: None,
|
||||
},
|
||||
unread_notification_count: 0,
|
||||
connection_addresses: ConnectionAddresses {
|
||||
tor: Vec::new(),
|
||||
clearnet: Vec::new(),
|
||||
},
|
||||
password_hash: account.password.clone(),
|
||||
pubkey: ssh_key::PublicKey::from(Ed25519PublicKey::from(
|
||||
&account.key.ssh_key(),
|
||||
))
|
||||
.to_openssh()
|
||||
.unwrap(),
|
||||
ca_fingerprint: account
|
||||
.root_ca_cert
|
||||
.digest(MessageDigest::sha256())
|
||||
.unwrap()
|
||||
.iter()
|
||||
.map(|x| format!("{x:X}"))
|
||||
.join(":"),
|
||||
ntp_synced: false,
|
||||
zram: true,
|
||||
governor: None,
|
||||
ca_fingerprint: account
|
||||
.root_ca_cert
|
||||
.digest(MessageDigest::sha256())
|
||||
.unwrap()
|
||||
.iter()
|
||||
.map(|x| format!("{x:X}"))
|
||||
.join(":"),
|
||||
ntp_synced: false,
|
||||
zram: true,
|
||||
governor: None,
|
||||
},
|
||||
package_data: AllPackageData::default(),
|
||||
ui: serde_json::from_str(include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/../../web/patchdb-ui-seed.json"
|
||||
)))
|
||||
.unwrap(),
|
||||
},
|
||||
package_data: AllPackageData::default(),
|
||||
ui: serde_json::from_str(include_str!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/../../web/patchdb-ui-seed.json"
|
||||
)))
|
||||
.unwrap(),
|
||||
private: (), // TODO
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize, HasModel)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[model = "Model<Self>"]
|
||||
// #[macro_debug]
|
||||
pub struct Public {
|
||||
pub server_info: ServerInfo,
|
||||
pub package_data: AllPackageData,
|
||||
pub ui: Value,
|
||||
}
|
||||
|
||||
pub type DatabaseModel = Model<Database>;
|
||||
|
||||
fn get_arch() -> InternedString {
|
||||
@@ -532,14 +544,13 @@ pub struct StaticDependencyInfo {
|
||||
#[model = "Model<Self>"]
|
||||
pub struct CurrentDependencyInfo {
|
||||
#[serde(default)]
|
||||
pub pointers: BTreeSet<PackagePointerSpec>,
|
||||
pub health_checks: BTreeSet<HealthCheckId>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, Deserialize, Serialize)]
|
||||
pub struct InterfaceAddressMap(pub BTreeMap<InterfaceId, InterfaceAddresses>);
|
||||
pub struct InterfaceAddressMap(pub BTreeMap<HostId, InterfaceAddresses>);
|
||||
impl Map for InterfaceAddressMap {
|
||||
type Key = InterfaceId;
|
||||
type Key = HostId;
|
||||
type Value = InterfaceAddresses;
|
||||
}
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ use std::marker::PhantomData;
|
||||
use std::panic::UnwindSafe;
|
||||
|
||||
pub use imbl_value::Value;
|
||||
use patch_db::json_ptr::ROOT;
|
||||
use patch_db::value::InternedString;
|
||||
pub use patch_db::{HasModel, PatchDb};
|
||||
use serde::de::DeserializeOwned;
|
||||
@@ -42,7 +43,7 @@ pub trait PatchDbExt {
|
||||
#[async_trait::async_trait]
|
||||
impl PatchDbExt for PatchDb {
|
||||
async fn peek(&self) -> DatabaseModel {
|
||||
DatabaseModel::from(self.dump().await.value)
|
||||
DatabaseModel::from(self.dump(&ROOT).await.value)
|
||||
}
|
||||
async fn mutate<U: UnwindSafe + Send>(
|
||||
&self,
|
||||
|
||||
Reference in New Issue
Block a user