mirror of
https://github.com/Start9Labs/start-os.git
synced 2026-03-30 12:11:56 +00:00
Feature/lxc container runtime (#2514)
* wip: static-server errors * wip: fix wifi * wip: Fix the service_effects * wip: Fix cors in the middleware * wip(chore): Auth clean up the lint. * wip(fix): Vhost * wip: continue manager refactor Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: service manager refactor * wip: Some fixes * wip(fix): Fix the lib.rs * wip * wip(fix): Logs * wip: bins * wip(innspect): Add in the inspect * wip: config * wip(fix): Diagnostic * wip(fix): Dependencies * wip: context * wip(fix) Sorta auth * wip: warnings * wip(fix): registry/admin * wip(fix) marketplace * wip(fix) Some more converted and fixed with the linter and config * wip: Working on the static server * wip(fix)static server * wip: Remove some asynnc * wip: Something about the request and regular rpc * wip: gut install Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: Convert the static server into the new system * wip delete file * test * wip(fix) vhost does not need the with safe defaults * wip: Adding in the wifi * wip: Fix the developer and the verify * wip: new install flow Co-authored-by: J H <Blu-J@users.noreply.github.com> * fix middleware * wip * wip: Fix the auth * wip * continue service refactor * feature: Service get_config * feat: Action * wip: Fighting the great fight against the borrow checker * wip: Remove an error in a file that I just need to deel with later * chore: Add in some more lifetime stuff to the services * wip: Install fix on lifetime * cleanup * wip: Deal with the borrow later * more cleanup * resolve borrowchecker errors * wip(feat): add in the handler for the socket, for now * wip(feat): Update the service_effect_handler::action * chore: Add in the changes to make sure the from_service goes to context * chore: Change the * refactor service map * fix references to service map * fill out restore * wip: Before I work on the store stuff * fix backup module * handle some warnings * feat: add in the ui components on the rust side * feature: Update the procedures * chore: Update the js side of the main and a few of the others * chore: Update the rpc listener to match the persistant container * wip: Working on updating some things to have a better name * wip(feat): Try and get the rpc to return the correct shape? * lxc wip * wip(feat): Try and get the rpc to return the correct shape? * build for container runtime wip * remove container-init * fix build * fix error * chore: Update to work I suppose * lxc wip * remove docker module and feature * download alpine squashfs automatically * overlays effect Co-authored-by: Jade <Blu-J@users.noreply.github.com> * chore: Add the overlay effect * feat: Add the mounter in the main * chore: Convert to use the mounts, still need to work with the sandbox * install fixes * fix ssl * fixes from testing * implement tmpfile for upload * wip * misc fixes * cleanup * cleanup * better progress reporting * progress for sideload * return real guid * add devmode script * fix lxc rootfs path * fix percentage bar * fix progress bar styling * fix build for unstable * tweaks * label progress * tweaks * update progress more often * make symlink in rpc_client * make socket dir * fix parent path * add start-cli to container * add echo and gitInfo commands * wip: Add the init + errors * chore: Add in the exit effect for the system * chore: Change the type to null for failure to parse * move sigterm timeout to stopping status * update order * chore: Update the return type * remove dbg * change the map error * chore: Update the thing to capture id * chore add some life changes * chore: Update the loging * chore: Update the package to run module * us From for RpcError * chore: Update to use import instead * chore: update * chore: Use require for the backup * fix a default * update the type that is wrong * chore: Update the type of the manifest * chore: Update to make null * only symlink if not exists * get rid of double result * better debug info for ErrorCollection * chore: Update effects * chore: fix * mount assets and volumes * add exec instead of spawn * fix mounting in image * fix overlay mounts Co-authored-by: Jade <Blu-J@users.noreply.github.com> * misc fixes * feat: Fix two * fix: systemForEmbassy main * chore: Fix small part of main loop * chore: Modify the bundle * merge * fixMain loop" * move tsc to makefile * chore: Update the return types of the health check * fix client * chore: Convert the todo to use tsmatches * add in the fixes for the seen and create the hack to allow demo * chore: Update to include the systemForStartOs * chore UPdate to the latest types from the expected outout * fixes * fix typo * Don't emit if failure on tsc * wip Co-authored-by: Jade <Blu-J@users.noreply.github.com> * add s9pk api * add inspection * add inspect manifest * newline after display serializable * fix squashfs in image name * edit manifest Co-authored-by: Jade <Blu-J@users.noreply.github.com> * wait for response on repl * ignore sig for now * ignore sig for now * re-enable sig verification * fix * wip * env and chroot * add profiling logs * set uid & gid in squashfs to 100000 * set uid of sqfs to 100000 * fix mksquashfs args * add env to compat * fix * re-add docker feature flag * fix docker output format being stupid * here be dragons * chore: Add in the cross compiling for something * fix npm link * extract logs from container on exit * chore: Update for testing * add log capture to drop trait * chore: add in the modifications that I make * chore: Update small things for no updates * chore: Update the types of something * chore: Make main not complain * idmapped mounts * idmapped volumes * re-enable kiosk * chore: Add in some logging for the new system * bring in start-sdk * remove avahi * chore: Update the deps * switch to musl * chore: Update the version of prettier * chore: Organize' * chore: Update some of the headers back to the standard of fetch * fix musl build * fix idmapped mounts * fix cross build * use cross compiler for correct arch * feat: Add in the faked ssl stuff for the effects * @dr_bonez Did a solution here * chore: Something that DrBonez * chore: up * wip: We have a working server!!! * wip * uninstall * wip * tes --------- Co-authored-by: J H <dragondef@gmail.com> Co-authored-by: J H <Blu-J@users.noreply.github.com> Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com>
This commit is contained in:
Aiden McClelland
@@ -1,43 +1,37 @@
|
||||
use std::fs::File;
|
||||
use std::io::BufReader;
|
||||
use std::net::Ipv4Addr;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
|
||||
use clap::ArgMatches;
|
||||
use color_eyre::eyre::eyre;
|
||||
use cookie_store::{CookieStore, RawCookie};
|
||||
use josekit::jwk::Jwk;
|
||||
use once_cell::sync::OnceCell;
|
||||
use reqwest::Proxy;
|
||||
use reqwest_cookie_store::CookieStoreMutex;
|
||||
use rpc_toolkit::reqwest::{Client, Url};
|
||||
use rpc_toolkit::url::Host;
|
||||
use rpc_toolkit::Context;
|
||||
use serde::Deserialize;
|
||||
use rpc_toolkit::yajrc::RpcError;
|
||||
use rpc_toolkit::{call_remote_http, CallRemote, Context};
|
||||
use tokio::net::TcpStream;
|
||||
use tokio::runtime::Runtime;
|
||||
use tokio_tungstenite::{MaybeTlsStream, WebSocketStream};
|
||||
use tracing::instrument;
|
||||
|
||||
use super::setup::CURRENT_SECRET;
|
||||
use crate::context::config::{local_config_path, ClientConfig};
|
||||
use crate::core::rpc_continuations::RequestGuid;
|
||||
use crate::middleware::auth::LOCAL_AUTH_COOKIE_PATH;
|
||||
use crate::util::config::{load_config_from_paths, local_config_path};
|
||||
use crate::ResultExt;
|
||||
|
||||
#[derive(Debug, Default, Deserialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub struct CliContextConfig {
|
||||
pub host: Option<Url>,
|
||||
#[serde(deserialize_with = "crate::util::serde::deserialize_from_str_opt")]
|
||||
#[serde(default)]
|
||||
pub proxy: Option<Url>,
|
||||
pub cookie_path: Option<PathBuf>,
|
||||
}
|
||||
use crate::prelude::*;
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct CliContextSeed {
|
||||
pub runtime: OnceCell<Runtime>,
|
||||
pub base_url: Url,
|
||||
pub rpc_url: Url,
|
||||
pub client: Client,
|
||||
pub cookie_store: Arc<CookieStoreMutex>,
|
||||
pub cookie_path: PathBuf,
|
||||
pub developer_key_path: PathBuf,
|
||||
pub developer_key: OnceCell<ed25519_dalek::SigningKey>,
|
||||
}
|
||||
impl Drop for CliContextSeed {
|
||||
fn drop(&mut self) {
|
||||
@@ -60,42 +54,22 @@ impl Drop for CliContextSeed {
|
||||
}
|
||||
}
|
||||
|
||||
const DEFAULT_HOST: Host<&'static str> = Host::Ipv4(Ipv4Addr::new(127, 0, 0, 1));
|
||||
const DEFAULT_PORT: u16 = 5959;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CliContext(Arc<CliContextSeed>);
|
||||
impl CliContext {
|
||||
/// BLOCKING
|
||||
#[instrument(skip_all)]
|
||||
pub fn init(matches: &ArgMatches) -> Result<Self, crate::Error> {
|
||||
let local_config_path = local_config_path();
|
||||
let base: CliContextConfig = load_config_from_paths(
|
||||
matches
|
||||
.values_of("config")
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.map(|p| Path::new(p))
|
||||
.chain(local_config_path.as_deref().into_iter())
|
||||
.chain(std::iter::once(Path::new(crate::util::config::CONFIG_PATH))),
|
||||
)?;
|
||||
let mut url = if let Some(host) = matches.value_of("host") {
|
||||
host.parse()?
|
||||
} else if let Some(host) = base.host {
|
||||
pub fn init(config: ClientConfig) -> Result<Self, Error> {
|
||||
let mut url = if let Some(host) = config.host {
|
||||
host
|
||||
} else {
|
||||
"http://localhost".parse()?
|
||||
};
|
||||
let proxy = if let Some(proxy) = matches.value_of("proxy") {
|
||||
Some(proxy.parse()?)
|
||||
} else {
|
||||
base.proxy
|
||||
};
|
||||
|
||||
let cookie_path = base.cookie_path.unwrap_or_else(|| {
|
||||
local_config_path
|
||||
let cookie_path = config.cookie_path.unwrap_or_else(|| {
|
||||
local_config_path()
|
||||
.as_deref()
|
||||
.unwrap_or_else(|| Path::new(crate::util::config::CONFIG_PATH))
|
||||
.unwrap_or_else(|| Path::new(super::config::CONFIG_PATH))
|
||||
.parent()
|
||||
.unwrap_or(Path::new("/"))
|
||||
.join(".cookies.json")
|
||||
@@ -120,6 +94,7 @@ impl CliContext {
|
||||
}));
|
||||
|
||||
Ok(CliContext(Arc::new(CliContextSeed {
|
||||
runtime: OnceCell::new(),
|
||||
base_url: url.clone(),
|
||||
rpc_url: {
|
||||
url.path_segments_mut()
|
||||
@@ -131,7 +106,7 @@ impl CliContext {
|
||||
},
|
||||
client: {
|
||||
let mut builder = Client::builder().cookie_provider(cookie_store.clone());
|
||||
if let Some(proxy) = proxy {
|
||||
if let Some(proxy) = config.proxy {
|
||||
builder =
|
||||
builder.proxy(Proxy::all(proxy).with_kind(crate::ErrorKind::ParseUrl)?)
|
||||
}
|
||||
@@ -139,8 +114,90 @@ impl CliContext {
|
||||
},
|
||||
cookie_store,
|
||||
cookie_path,
|
||||
developer_key_path: config.developer_key_path.unwrap_or_else(|| {
|
||||
local_config_path()
|
||||
.as_deref()
|
||||
.unwrap_or_else(|| Path::new(super::config::CONFIG_PATH))
|
||||
.parent()
|
||||
.unwrap_or(Path::new("/"))
|
||||
.join("developer.key.pem")
|
||||
}),
|
||||
developer_key: OnceCell::new(),
|
||||
})))
|
||||
}
|
||||
|
||||
/// BLOCKING
|
||||
#[instrument(skip_all)]
|
||||
pub fn developer_key(&self) -> Result<&ed25519_dalek::SigningKey, Error> {
|
||||
self.developer_key.get_or_try_init(|| {
|
||||
if !self.developer_key_path.exists() {
|
||||
return Err(Error::new(eyre!("Developer Key does not exist! Please run `start-cli init` before running this command."), crate::ErrorKind::Uninitialized));
|
||||
}
|
||||
let pair = <ed25519::KeypairBytes as ed25519::pkcs8::DecodePrivateKey>::from_pkcs8_pem(
|
||||
&std::fs::read_to_string(&self.developer_key_path)?,
|
||||
)
|
||||
.with_kind(crate::ErrorKind::Pem)?;
|
||||
let secret = ed25519_dalek::SecretKey::try_from(&pair.secret_key[..]).map_err(|_| {
|
||||
Error::new(
|
||||
eyre!("pkcs8 key is of incorrect length"),
|
||||
ErrorKind::OpenSsl,
|
||||
)
|
||||
})?;
|
||||
Ok(secret.into())
|
||||
})
|
||||
}
|
||||
|
||||
pub async fn ws_continuation(
|
||||
&self,
|
||||
guid: RequestGuid,
|
||||
) -> Result<WebSocketStream<MaybeTlsStream<TcpStream>>, Error> {
|
||||
let mut url = self.base_url.clone();
|
||||
let ws_scheme = match url.scheme() {
|
||||
"https" => "wss",
|
||||
"http" => "ws",
|
||||
_ => {
|
||||
return Err(Error::new(
|
||||
eyre!("Cannot parse scheme from base URL"),
|
||||
crate::ErrorKind::ParseUrl,
|
||||
)
|
||||
.into())
|
||||
}
|
||||
};
|
||||
url.set_scheme(ws_scheme)
|
||||
.map_err(|_| Error::new(eyre!("Cannot set URL scheme"), crate::ErrorKind::ParseUrl))?;
|
||||
url.path_segments_mut()
|
||||
.map_err(|_| eyre!("Url cannot be base"))
|
||||
.with_kind(crate::ErrorKind::ParseUrl)?
|
||||
.push("ws")
|
||||
.push("rpc")
|
||||
.push(guid.as_ref());
|
||||
let (stream, _) =
|
||||
// base_url is "http://127.0.0.1/", with a trailing slash, so we don't put a leading slash in this path:
|
||||
tokio_tungstenite::connect_async(url).await.with_kind(ErrorKind::Network)?;
|
||||
Ok(stream)
|
||||
}
|
||||
|
||||
pub async fn rest_continuation(
|
||||
&self,
|
||||
guid: RequestGuid,
|
||||
body: reqwest::Body,
|
||||
headers: reqwest::header::HeaderMap,
|
||||
) -> Result<reqwest::Response, Error> {
|
||||
let mut url = self.base_url.clone();
|
||||
url.path_segments_mut()
|
||||
.map_err(|_| eyre!("Url cannot be base"))
|
||||
.with_kind(crate::ErrorKind::ParseUrl)?
|
||||
.push("rest")
|
||||
.push("rpc")
|
||||
.push(guid.as_ref());
|
||||
self.client
|
||||
.post(url)
|
||||
.headers(headers)
|
||||
.body(body)
|
||||
.send()
|
||||
.await
|
||||
.with_kind(ErrorKind::Network)
|
||||
}
|
||||
}
|
||||
impl AsRef<Jwk> for CliContext {
|
||||
fn as_ref(&self) -> &Jwk {
|
||||
@@ -154,32 +211,33 @@ impl std::ops::Deref for CliContext {
|
||||
}
|
||||
}
|
||||
impl Context for CliContext {
|
||||
fn protocol(&self) -> &str {
|
||||
self.0.base_url.scheme()
|
||||
}
|
||||
fn host(&self) -> Host<&str> {
|
||||
self.0.base_url.host().unwrap_or(DEFAULT_HOST)
|
||||
}
|
||||
fn port(&self) -> u16 {
|
||||
self.0.base_url.port().unwrap_or(DEFAULT_PORT)
|
||||
}
|
||||
fn path(&self) -> &str {
|
||||
self.0.rpc_url.path()
|
||||
}
|
||||
fn url(&self) -> Url {
|
||||
self.0.rpc_url.clone()
|
||||
}
|
||||
fn client(&self) -> &Client {
|
||||
&self.0.client
|
||||
fn runtime(&self) -> tokio::runtime::Handle {
|
||||
self.runtime
|
||||
.get_or_init(|| {
|
||||
tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.unwrap()
|
||||
})
|
||||
.handle()
|
||||
.clone()
|
||||
}
|
||||
}
|
||||
/// When we had an empty proxy the system wasn't working like it used to, which allowed empty proxy
|
||||
#[async_trait::async_trait]
|
||||
impl CallRemote for CliContext {
|
||||
async fn call_remote(&self, method: &str, params: Value) -> Result<Value, RpcError> {
|
||||
call_remote_http(&self.client, self.rpc_url.clone(), method, params).await
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_cli_proxy_empty() {
|
||||
serde_yaml::from_str::<CliContextConfig>(
|
||||
"
|
||||
bind_rpc:
|
||||
",
|
||||
)
|
||||
.unwrap();
|
||||
fn test() {
|
||||
let ctx = CliContext::init(ClientConfig::default()).unwrap();
|
||||
ctx.runtime().block_on(async {
|
||||
reqwest::Client::new()
|
||||
.get("http://example.com")
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
});
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user