mirror of
https://github.com/Start9Labs/start-os.git
synced 2026-03-30 20:14:49 +00:00
Feature/lxc container runtime (#2514)
* wip: static-server errors * wip: fix wifi * wip: Fix the service_effects * wip: Fix cors in the middleware * wip(chore): Auth clean up the lint. * wip(fix): Vhost * wip: continue manager refactor Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: service manager refactor * wip: Some fixes * wip(fix): Fix the lib.rs * wip * wip(fix): Logs * wip: bins * wip(innspect): Add in the inspect * wip: config * wip(fix): Diagnostic * wip(fix): Dependencies * wip: context * wip(fix) Sorta auth * wip: warnings * wip(fix): registry/admin * wip(fix) marketplace * wip(fix) Some more converted and fixed with the linter and config * wip: Working on the static server * wip(fix)static server * wip: Remove some asynnc * wip: Something about the request and regular rpc * wip: gut install Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: Convert the static server into the new system * wip delete file * test * wip(fix) vhost does not need the with safe defaults * wip: Adding in the wifi * wip: Fix the developer and the verify * wip: new install flow Co-authored-by: J H <Blu-J@users.noreply.github.com> * fix middleware * wip * wip: Fix the auth * wip * continue service refactor * feature: Service get_config * feat: Action * wip: Fighting the great fight against the borrow checker * wip: Remove an error in a file that I just need to deel with later * chore: Add in some more lifetime stuff to the services * wip: Install fix on lifetime * cleanup * wip: Deal with the borrow later * more cleanup * resolve borrowchecker errors * wip(feat): add in the handler for the socket, for now * wip(feat): Update the service_effect_handler::action * chore: Add in the changes to make sure the from_service goes to context * chore: Change the * refactor service map * fix references to service map * fill out restore * wip: Before I work on the store stuff * fix backup module * handle some warnings * feat: add in the ui components on the rust side * feature: Update the procedures * chore: Update the js side of the main and a few of the others * chore: Update the rpc listener to match the persistant container * wip: Working on updating some things to have a better name * wip(feat): Try and get the rpc to return the correct shape? * lxc wip * wip(feat): Try and get the rpc to return the correct shape? * build for container runtime wip * remove container-init * fix build * fix error * chore: Update to work I suppose * lxc wip * remove docker module and feature * download alpine squashfs automatically * overlays effect Co-authored-by: Jade <Blu-J@users.noreply.github.com> * chore: Add the overlay effect * feat: Add the mounter in the main * chore: Convert to use the mounts, still need to work with the sandbox * install fixes * fix ssl * fixes from testing * implement tmpfile for upload * wip * misc fixes * cleanup * cleanup * better progress reporting * progress for sideload * return real guid * add devmode script * fix lxc rootfs path * fix percentage bar * fix progress bar styling * fix build for unstable * tweaks * label progress * tweaks * update progress more often * make symlink in rpc_client * make socket dir * fix parent path * add start-cli to container * add echo and gitInfo commands * wip: Add the init + errors * chore: Add in the exit effect for the system * chore: Change the type to null for failure to parse * move sigterm timeout to stopping status * update order * chore: Update the return type * remove dbg * change the map error * chore: Update the thing to capture id * chore add some life changes * chore: Update the loging * chore: Update the package to run module * us From for RpcError * chore: Update to use import instead * chore: update * chore: Use require for the backup * fix a default * update the type that is wrong * chore: Update the type of the manifest * chore: Update to make null * only symlink if not exists * get rid of double result * better debug info for ErrorCollection * chore: Update effects * chore: fix * mount assets and volumes * add exec instead of spawn * fix mounting in image * fix overlay mounts Co-authored-by: Jade <Blu-J@users.noreply.github.com> * misc fixes * feat: Fix two * fix: systemForEmbassy main * chore: Fix small part of main loop * chore: Modify the bundle * merge * fixMain loop" * move tsc to makefile * chore: Update the return types of the health check * fix client * chore: Convert the todo to use tsmatches * add in the fixes for the seen and create the hack to allow demo * chore: Update to include the systemForStartOs * chore UPdate to the latest types from the expected outout * fixes * fix typo * Don't emit if failure on tsc * wip Co-authored-by: Jade <Blu-J@users.noreply.github.com> * add s9pk api * add inspection * add inspect manifest * newline after display serializable * fix squashfs in image name * edit manifest Co-authored-by: Jade <Blu-J@users.noreply.github.com> * wait for response on repl * ignore sig for now * ignore sig for now * re-enable sig verification * fix * wip * env and chroot * add profiling logs * set uid & gid in squashfs to 100000 * set uid of sqfs to 100000 * fix mksquashfs args * add env to compat * fix * re-add docker feature flag * fix docker output format being stupid * here be dragons * chore: Add in the cross compiling for something * fix npm link * extract logs from container on exit * chore: Update for testing * add log capture to drop trait * chore: add in the modifications that I make * chore: Update small things for no updates * chore: Update the types of something * chore: Make main not complain * idmapped mounts * idmapped volumes * re-enable kiosk * chore: Add in some logging for the new system * bring in start-sdk * remove avahi * chore: Update the deps * switch to musl * chore: Update the version of prettier * chore: Organize' * chore: Update some of the headers back to the standard of fetch * fix musl build * fix idmapped mounts * fix cross build * use cross compiler for correct arch * feat: Add in the faked ssl stuff for the effects * @dr_bonez Did a solution here * chore: Something that DrBonez * chore: up * wip: We have a working server!!! * wip * uninstall * wip * tes --------- Co-authored-by: J H <dragondef@gmail.com> Co-authored-by: J H <Blu-J@users.noreply.github.com> Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com>
This commit is contained in:
Aiden McClelland
@@ -4,14 +4,13 @@ use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
|
||||
use chrono::Utc;
|
||||
use clap::ArgMatches;
|
||||
use clap::Parser;
|
||||
use color_eyre::eyre::eyre;
|
||||
use helpers::AtomicFile;
|
||||
use imbl::OrdSet;
|
||||
use models::Version;
|
||||
use rpc_toolkit::command;
|
||||
use models::PackageId;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tokio::io::AsyncWriteExt;
|
||||
use tokio::sync::Mutex;
|
||||
use tracing::instrument;
|
||||
|
||||
use super::target::BackupTargetId;
|
||||
@@ -21,42 +20,37 @@ use crate::backup::os::OsBackup;
|
||||
use crate::backup::{BackupReport, ServerBackupReport};
|
||||
use crate::context::RpcContext;
|
||||
use crate::db::model::BackupProgress;
|
||||
use crate::db::package::get_packages;
|
||||
use crate::disk::mount::backup::BackupMountGuard;
|
||||
use crate::disk::mount::filesystem::ReadWrite;
|
||||
use crate::disk::mount::guard::TmpMountGuard;
|
||||
use crate::manager::BackupReturn;
|
||||
use crate::disk::mount::guard::{GenericMountGuard, TmpMountGuard};
|
||||
use crate::notifications::NotificationLevel;
|
||||
use crate::prelude::*;
|
||||
use crate::s9pk::manifest::PackageId;
|
||||
use crate::util::display_none;
|
||||
use crate::util::io::dir_copy;
|
||||
use crate::util::serde::IoFormat;
|
||||
use crate::version::VersionT;
|
||||
|
||||
fn parse_comma_separated(arg: &str, _: &ArgMatches) -> Result<OrdSet<PackageId>, Error> {
|
||||
arg.split(',')
|
||||
.map(|s| s.trim().parse::<PackageId>().map_err(Error::from))
|
||||
.collect()
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct BackupParams {
|
||||
target_id: BackupTargetId,
|
||||
#[arg(long = "old-password")]
|
||||
old_password: Option<crate::auth::PasswordType>,
|
||||
#[arg(long = "package-ids")]
|
||||
package_ids: Option<Vec<PackageId>>,
|
||||
password: crate::auth::PasswordType,
|
||||
}
|
||||
|
||||
#[command(rename = "create", display(display_none))]
|
||||
#[instrument(skip(ctx, old_password, password))]
|
||||
pub async fn backup_all(
|
||||
#[context] ctx: RpcContext,
|
||||
#[arg(rename = "target-id")] target_id: BackupTargetId,
|
||||
#[arg(rename = "old-password", long = "old-password")] old_password: Option<
|
||||
crate::auth::PasswordType,
|
||||
>,
|
||||
#[arg(
|
||||
rename = "package-ids",
|
||||
long = "package-ids",
|
||||
parse(parse_comma_separated)
|
||||
)]
|
||||
package_ids: Option<OrdSet<PackageId>>,
|
||||
#[arg] password: crate::auth::PasswordType,
|
||||
ctx: RpcContext,
|
||||
BackupParams {
|
||||
target_id,
|
||||
old_password,
|
||||
package_ids,
|
||||
password,
|
||||
}: BackupParams,
|
||||
) -> Result<(), Error> {
|
||||
let db = ctx.db.peek().await;
|
||||
let old_password_decrypted = old_password
|
||||
.as_ref()
|
||||
.unwrap_or(&password)
|
||||
@@ -73,20 +67,9 @@ pub async fn backup_all(
|
||||
)
|
||||
.await?;
|
||||
let package_ids = if let Some(ids) = package_ids {
|
||||
ids.into_iter()
|
||||
.flat_map(|package_id| {
|
||||
let version = db
|
||||
.as_package_data()
|
||||
.as_idx(&package_id)?
|
||||
.as_manifest()
|
||||
.as_version()
|
||||
.de()
|
||||
.ok()?;
|
||||
Some((package_id, version))
|
||||
})
|
||||
.collect()
|
||||
ids.into_iter().collect()
|
||||
} else {
|
||||
get_packages(db.clone())?.into_iter().collect()
|
||||
todo!("all installed packages");
|
||||
};
|
||||
if old_password.is_some() {
|
||||
backup_guard.change_password(&password)?;
|
||||
@@ -108,10 +91,7 @@ pub async fn backup_all(
|
||||
attempted: true,
|
||||
error: None,
|
||||
},
|
||||
packages: report
|
||||
.into_iter()
|
||||
.map(|((package_id, _), value)| (package_id, value))
|
||||
.collect(),
|
||||
packages: report,
|
||||
},
|
||||
None,
|
||||
)
|
||||
@@ -130,10 +110,7 @@ pub async fn backup_all(
|
||||
attempted: true,
|
||||
error: None,
|
||||
},
|
||||
packages: report
|
||||
.into_iter()
|
||||
.map(|((package_id, _), value)| (package_id, value))
|
||||
.collect(),
|
||||
packages: report,
|
||||
},
|
||||
None,
|
||||
)
|
||||
@@ -178,7 +155,7 @@ pub async fn backup_all(
|
||||
#[instrument(skip(db, packages))]
|
||||
async fn assure_backing_up(
|
||||
db: &PatchDb,
|
||||
packages: impl IntoIterator<Item = &(PackageId, Version)> + UnwindSafe + Send,
|
||||
packages: impl IntoIterator<Item = &PackageId> + UnwindSafe + Send,
|
||||
) -> Result<(), Error> {
|
||||
db.mutate(|v| {
|
||||
let backing_up = v
|
||||
@@ -205,7 +182,7 @@ async fn assure_backing_up(
|
||||
backing_up.ser(&Some(
|
||||
packages
|
||||
.into_iter()
|
||||
.map(|(x, _)| (x.clone(), BackupProgress { complete: false }))
|
||||
.map(|x| (x.clone(), BackupProgress { complete: false }))
|
||||
.collect(),
|
||||
))?;
|
||||
Ok(())
|
||||
@@ -217,62 +194,39 @@ async fn assure_backing_up(
|
||||
async fn perform_backup(
|
||||
ctx: &RpcContext,
|
||||
backup_guard: BackupMountGuard<TmpMountGuard>,
|
||||
package_ids: &OrdSet<(PackageId, Version)>,
|
||||
) -> Result<BTreeMap<(PackageId, Version), PackageBackupReport>, Error> {
|
||||
package_ids: &OrdSet<PackageId>,
|
||||
) -> Result<BTreeMap<PackageId, PackageBackupReport>, Error> {
|
||||
let mut backup_report = BTreeMap::new();
|
||||
let backup_guard = Arc::new(Mutex::new(backup_guard));
|
||||
let backup_guard = Arc::new(backup_guard);
|
||||
|
||||
for package_id in package_ids {
|
||||
let (response, _report) = match ctx
|
||||
.managers
|
||||
.get(package_id)
|
||||
.await
|
||||
.ok_or_else(|| Error::new(eyre!("Manager not found"), ErrorKind::InvalidRequest))?
|
||||
.backup(backup_guard.clone())
|
||||
.await
|
||||
{
|
||||
BackupReturn::Ran { report, res } => (res, report),
|
||||
BackupReturn::AlreadyRunning(report) => {
|
||||
backup_report.insert(package_id.clone(), report);
|
||||
continue;
|
||||
}
|
||||
BackupReturn::Error(error) => {
|
||||
tracing::warn!("Backup thread error");
|
||||
tracing::debug!("{error:?}");
|
||||
backup_report.insert(
|
||||
package_id.clone(),
|
||||
PackageBackupReport {
|
||||
error: Some("Backup thread error".to_owned()),
|
||||
},
|
||||
);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
backup_report.insert(
|
||||
package_id.clone(),
|
||||
PackageBackupReport {
|
||||
error: response.as_ref().err().map(|e| e.to_string()),
|
||||
},
|
||||
);
|
||||
|
||||
if let Ok(pkg_meta) = response {
|
||||
backup_guard
|
||||
.lock()
|
||||
.await
|
||||
.metadata
|
||||
.package_backups
|
||||
.insert(package_id.0.clone(), pkg_meta);
|
||||
for id in package_ids {
|
||||
if let Some(service) = &*ctx.services.get(id).await {
|
||||
backup_report.insert(
|
||||
id.clone(),
|
||||
PackageBackupReport {
|
||||
error: service
|
||||
.backup(backup_guard.package_backup(id))
|
||||
.await
|
||||
.err()
|
||||
.map(|e| e.to_string()),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let mut backup_guard = Arc::try_unwrap(backup_guard).map_err(|_| {
|
||||
Error::new(
|
||||
eyre!("leaked reference to BackupMountGuard"),
|
||||
ErrorKind::Incoherent,
|
||||
)
|
||||
})?;
|
||||
|
||||
let ui = ctx.db.peek().await.into_ui().de()?;
|
||||
|
||||
let mut os_backup_file = AtomicFile::new(
|
||||
backup_guard.lock().await.as_ref().join("os-backup.cbor"),
|
||||
None::<PathBuf>,
|
||||
)
|
||||
.await
|
||||
.with_kind(ErrorKind::Filesystem)?;
|
||||
let mut os_backup_file =
|
||||
AtomicFile::new(backup_guard.path().join("os-backup.cbor"), None::<PathBuf>)
|
||||
.await
|
||||
.with_kind(ErrorKind::Filesystem)?;
|
||||
os_backup_file
|
||||
.write_all(&IoFormat::Cbor.to_vec(&OsBackup {
|
||||
account: ctx.account.read().await.clone(),
|
||||
@@ -284,11 +238,11 @@ async fn perform_backup(
|
||||
.await
|
||||
.with_kind(ErrorKind::Filesystem)?;
|
||||
|
||||
let luks_folder_old = backup_guard.lock().await.as_ref().join("luks.old");
|
||||
let luks_folder_old = backup_guard.path().join("luks.old");
|
||||
if tokio::fs::metadata(&luks_folder_old).await.is_ok() {
|
||||
tokio::fs::remove_dir_all(&luks_folder_old).await?;
|
||||
}
|
||||
let luks_folder_bak = backup_guard.lock().await.as_ref().join("luks");
|
||||
let luks_folder_bak = backup_guard.path().join("luks");
|
||||
if tokio::fs::metadata(&luks_folder_bak).await.is_ok() {
|
||||
tokio::fs::rename(&luks_folder_bak, &luks_folder_old).await?;
|
||||
}
|
||||
@@ -298,14 +252,6 @@ async fn perform_backup(
|
||||
}
|
||||
|
||||
let timestamp = Some(Utc::now());
|
||||
let mut backup_guard = Arc::try_unwrap(backup_guard)
|
||||
.map_err(|_err| {
|
||||
Error::new(
|
||||
eyre!("Backup guard could not ensure that the others where dropped"),
|
||||
ErrorKind::Unknown,
|
||||
)
|
||||
})?
|
||||
.into_inner();
|
||||
|
||||
backup_guard.unencrypted_metadata.version = crate::version::Current::new().semver().into();
|
||||
backup_guard.unencrypted_metadata.full = true;
|
||||
|
||||
@@ -1,33 +1,16 @@
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use color_eyre::eyre::eyre;
|
||||
use helpers::AtomicFile;
|
||||
use models::{ImageId, OptionExt};
|
||||
use models::PackageId;
|
||||
use reqwest::Url;
|
||||
use rpc_toolkit::command;
|
||||
use rpc_toolkit::{from_fn_async, HandlerExt, ParentHandler};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tokio::fs::File;
|
||||
use tokio::io::AsyncWriteExt;
|
||||
use tracing::instrument;
|
||||
|
||||
use self::target::PackageBackupInfo;
|
||||
use crate::context::RpcContext;
|
||||
use crate::install::PKG_ARCHIVE_DIR;
|
||||
use crate::manager::manager_seed::ManagerSeed;
|
||||
use crate::context::CliContext;
|
||||
use crate::net::interface::InterfaceId;
|
||||
use crate::net::keys::Key;
|
||||
#[allow(unused_imports)]
|
||||
use crate::prelude::*;
|
||||
use crate::procedure::docker::DockerContainers;
|
||||
use crate::procedure::{NoOutput, PackageProcedure, ProcedureName};
|
||||
use crate::s9pk::manifest::PackageId;
|
||||
use crate::util::serde::{Base32, Base64, IoFormat};
|
||||
use crate::util::Version;
|
||||
use crate::version::{Current, VersionT};
|
||||
use crate::volume::{backup_dir, Volume, VolumeId, Volumes, BACKUP_DIR};
|
||||
use crate::{Error, ErrorKind, ResultExt};
|
||||
use crate::util::serde::{Base32, Base64};
|
||||
|
||||
pub mod backup_bulk;
|
||||
pub mod os;
|
||||
@@ -51,14 +34,16 @@ pub struct PackageBackupReport {
|
||||
pub error: Option<String>,
|
||||
}
|
||||
|
||||
#[command(subcommands(backup_bulk::backup_all, target::target))]
|
||||
pub fn backup() -> Result<(), Error> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[command(rename = "backup", subcommands(restore::restore_packages_rpc))]
|
||||
pub fn package_backup() -> Result<(), Error> {
|
||||
Ok(())
|
||||
// #[command(subcommands(backup_bulk::backup_all, target::target))]
|
||||
pub fn backup() -> ParentHandler {
|
||||
ParentHandler::new()
|
||||
.subcommand(
|
||||
"create",
|
||||
from_fn_async(backup_bulk::backup_all)
|
||||
.no_display()
|
||||
.with_remote_cli::<CliContext>(),
|
||||
)
|
||||
.subcommand("target", target::target())
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Serialize)]
|
||||
@@ -70,157 +55,3 @@ struct BackupMetadata {
|
||||
pub tor_keys: BTreeMap<InterfaceId, Base32<[u8; 64]>>, // DEPRECATED
|
||||
pub marketplace_url: Option<Url>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Deserialize, Serialize, HasModel)]
|
||||
#[model = "Model<Self>"]
|
||||
pub struct BackupActions {
|
||||
pub create: PackageProcedure,
|
||||
pub restore: PackageProcedure,
|
||||
}
|
||||
impl BackupActions {
|
||||
pub fn validate(
|
||||
&self,
|
||||
_container: &Option<DockerContainers>,
|
||||
eos_version: &Version,
|
||||
volumes: &Volumes,
|
||||
image_ids: &BTreeSet<ImageId>,
|
||||
) -> Result<(), Error> {
|
||||
self.create
|
||||
.validate(eos_version, volumes, image_ids, false)
|
||||
.with_ctx(|_| (crate::ErrorKind::ValidateS9pk, "Backup Create"))?;
|
||||
self.restore
|
||||
.validate(eos_version, volumes, image_ids, false)
|
||||
.with_ctx(|_| (crate::ErrorKind::ValidateS9pk, "Backup Restore"))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[instrument(skip_all)]
|
||||
pub async fn create(&self, seed: Arc<ManagerSeed>) -> Result<PackageBackupInfo, Error> {
|
||||
let manifest = &seed.manifest;
|
||||
let mut volumes = seed.manifest.volumes.to_readonly();
|
||||
let ctx = &seed.ctx;
|
||||
let pkg_id = &manifest.id;
|
||||
let pkg_version = &manifest.version;
|
||||
volumes.insert(VolumeId::Backup, Volume::Backup { readonly: false });
|
||||
let backup_dir = backup_dir(&manifest.id);
|
||||
if tokio::fs::metadata(&backup_dir).await.is_err() {
|
||||
tokio::fs::create_dir_all(&backup_dir).await?
|
||||
}
|
||||
self.create
|
||||
.execute::<(), NoOutput>(
|
||||
ctx,
|
||||
pkg_id,
|
||||
pkg_version,
|
||||
ProcedureName::CreateBackup,
|
||||
&volumes,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await?
|
||||
.map_err(|e| eyre!("{}", e.1))
|
||||
.with_kind(crate::ErrorKind::Backup)?;
|
||||
let (network_keys, tor_keys): (Vec<_>, Vec<_>) =
|
||||
Key::for_package(&ctx.secret_store, pkg_id)
|
||||
.await?
|
||||
.into_iter()
|
||||
.filter_map(|k| {
|
||||
let interface = k.interface().map(|(_, i)| i)?;
|
||||
Some((
|
||||
(interface.clone(), Base64(k.as_bytes())),
|
||||
(interface, Base32(k.tor_key().as_bytes())),
|
||||
))
|
||||
})
|
||||
.unzip();
|
||||
let marketplace_url = ctx
|
||||
.db
|
||||
.peek()
|
||||
.await
|
||||
.as_package_data()
|
||||
.as_idx(&pkg_id)
|
||||
.or_not_found(pkg_id)?
|
||||
.expect_as_installed()?
|
||||
.as_installed()
|
||||
.as_marketplace_url()
|
||||
.de()?;
|
||||
let tmp_path = Path::new(BACKUP_DIR)
|
||||
.join(pkg_id)
|
||||
.join(format!("{}.s9pk", pkg_id));
|
||||
let s9pk_path = ctx
|
||||
.datadir
|
||||
.join(PKG_ARCHIVE_DIR)
|
||||
.join(pkg_id)
|
||||
.join(pkg_version.as_str())
|
||||
.join(format!("{}.s9pk", pkg_id));
|
||||
let mut infile = File::open(&s9pk_path).await?;
|
||||
let mut outfile = AtomicFile::new(&tmp_path, None::<PathBuf>)
|
||||
.await
|
||||
.with_kind(ErrorKind::Filesystem)?;
|
||||
tokio::io::copy(&mut infile, &mut *outfile)
|
||||
.await
|
||||
.with_ctx(|_| {
|
||||
(
|
||||
crate::ErrorKind::Filesystem,
|
||||
format!("cp {} -> {}", s9pk_path.display(), tmp_path.display()),
|
||||
)
|
||||
})?;
|
||||
outfile.save().await.with_kind(ErrorKind::Filesystem)?;
|
||||
let timestamp = Utc::now();
|
||||
let metadata_path = Path::new(BACKUP_DIR).join(pkg_id).join("metadata.cbor");
|
||||
let mut outfile = AtomicFile::new(&metadata_path, None::<PathBuf>)
|
||||
.await
|
||||
.with_kind(ErrorKind::Filesystem)?;
|
||||
let network_keys = network_keys.into_iter().collect();
|
||||
let tor_keys = tor_keys.into_iter().collect();
|
||||
outfile
|
||||
.write_all(&IoFormat::Cbor.to_vec(&BackupMetadata {
|
||||
timestamp,
|
||||
network_keys,
|
||||
tor_keys,
|
||||
marketplace_url,
|
||||
})?)
|
||||
.await?;
|
||||
outfile.save().await.with_kind(ErrorKind::Filesystem)?;
|
||||
Ok(PackageBackupInfo {
|
||||
os_version: Current::new().semver().into(),
|
||||
title: manifest.title.clone(),
|
||||
version: pkg_version.clone(),
|
||||
timestamp,
|
||||
})
|
||||
}
|
||||
|
||||
#[instrument(skip_all)]
|
||||
pub async fn restore(
|
||||
&self,
|
||||
ctx: &RpcContext,
|
||||
pkg_id: &PackageId,
|
||||
pkg_version: &Version,
|
||||
volumes: &Volumes,
|
||||
) -> Result<Option<Url>, Error> {
|
||||
let mut volumes = volumes.clone();
|
||||
volumes.insert(VolumeId::Backup, Volume::Backup { readonly: true });
|
||||
self.restore
|
||||
.execute::<(), NoOutput>(
|
||||
ctx,
|
||||
pkg_id,
|
||||
pkg_version,
|
||||
ProcedureName::RestoreBackup,
|
||||
&volumes,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await?
|
||||
.map_err(|e| eyre!("{}", e.1))
|
||||
.with_kind(crate::ErrorKind::Restore)?;
|
||||
let metadata_path = Path::new(BACKUP_DIR).join(pkg_id).join("metadata.cbor");
|
||||
let metadata: BackupMetadata = IoFormat::Cbor.from_slice(
|
||||
&tokio::fs::read(&metadata_path).await.with_ctx(|_| {
|
||||
(
|
||||
crate::ErrorKind::Filesystem,
|
||||
metadata_path.display().to_string(),
|
||||
)
|
||||
})?,
|
||||
)?;
|
||||
|
||||
Ok(metadata.marketplace_url)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,55 +1,46 @@
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::Path;
|
||||
use std::sync::atomic::Ordering;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use clap::ArgMatches;
|
||||
use futures::future::BoxFuture;
|
||||
use futures::{stream, FutureExt, StreamExt};
|
||||
use clap::Parser;
|
||||
use futures::{stream, StreamExt};
|
||||
use models::PackageId;
|
||||
use openssl::x509::X509;
|
||||
use rpc_toolkit::command;
|
||||
use sqlx::Connection;
|
||||
use tokio::fs::File;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use torut::onion::OnionAddressV3;
|
||||
use tracing::instrument;
|
||||
|
||||
use super::target::BackupTargetId;
|
||||
use crate::backup::os::OsBackup;
|
||||
use crate::backup::BackupMetadata;
|
||||
use crate::context::rpc::RpcContextConfig;
|
||||
use crate::context::{RpcContext, SetupContext};
|
||||
use crate::db::model::{PackageDataEntry, PackageDataEntryRestoring, StaticFiles};
|
||||
use crate::disk::mount::backup::{BackupMountGuard, PackageBackupMountGuard};
|
||||
use crate::disk::mount::backup::BackupMountGuard;
|
||||
use crate::disk::mount::filesystem::ReadWrite;
|
||||
use crate::disk::mount::guard::TmpMountGuard;
|
||||
use crate::disk::mount::guard::{GenericMountGuard, TmpMountGuard};
|
||||
use crate::hostname::Hostname;
|
||||
use crate::init::init;
|
||||
use crate::install::progress::InstallProgress;
|
||||
use crate::install::{download_install_s9pk, PKG_PUBLIC_DIR};
|
||||
use crate::notifications::NotificationLevel;
|
||||
use crate::prelude::*;
|
||||
use crate::s9pk::manifest::{Manifest, PackageId};
|
||||
use crate::s9pk::reader::S9pkReader;
|
||||
use crate::setup::SetupStatus;
|
||||
use crate::util::display_none;
|
||||
use crate::util::io::dir_size;
|
||||
use crate::s9pk::S9pk;
|
||||
use crate::service::service_map::DownloadInstallFuture;
|
||||
use crate::util::serde::IoFormat;
|
||||
use crate::volume::{backup_dir, BACKUP_DIR, PKG_VOLUME_DIR};
|
||||
|
||||
fn parse_comma_separated(arg: &str, _: &ArgMatches) -> Result<Vec<PackageId>, Error> {
|
||||
arg.split(',')
|
||||
.map(|s| s.trim().parse().map_err(Error::from))
|
||||
.collect()
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct RestorePackageParams {
|
||||
pub ids: Vec<PackageId>,
|
||||
pub target_id: BackupTargetId,
|
||||
pub password: String,
|
||||
}
|
||||
|
||||
#[command(rename = "restore", display(display_none))]
|
||||
// TODO dr Why doesn't anything use this
|
||||
// #[command(rename = "restore", display(display_none))]
|
||||
#[instrument(skip(ctx, password))]
|
||||
pub async fn restore_packages_rpc(
|
||||
#[context] ctx: RpcContext,
|
||||
#[arg(parse(parse_comma_separated))] ids: Vec<PackageId>,
|
||||
#[arg(rename = "target-id")] target_id: BackupTargetId,
|
||||
#[arg] password: String,
|
||||
ctx: RpcContext,
|
||||
RestorePackageParams {
|
||||
ids,
|
||||
target_id,
|
||||
password,
|
||||
}: RestorePackageParams,
|
||||
) -> Result<(), Error> {
|
||||
let fs = target_id
|
||||
.load(ctx.secret_store.acquire().await?.as_mut())
|
||||
@@ -57,114 +48,25 @@ pub async fn restore_packages_rpc(
|
||||
let backup_guard =
|
||||
BackupMountGuard::mount(TmpMountGuard::mount(&fs, ReadWrite).await?, &password).await?;
|
||||
|
||||
let (backup_guard, tasks, _) = restore_packages(&ctx, backup_guard, ids).await?;
|
||||
let tasks = restore_packages(&ctx, backup_guard, ids).await?;
|
||||
|
||||
tokio::spawn(async move {
|
||||
stream::iter(tasks.into_iter().map(|x| (x, ctx.clone())))
|
||||
.for_each_concurrent(5, |(res, ctx)| async move {
|
||||
match res.await {
|
||||
(Ok(_), _) => (),
|
||||
(Err(err), package_id) => {
|
||||
if let Err(err) = ctx
|
||||
.notification_manager
|
||||
.notify(
|
||||
ctx.db.clone(),
|
||||
Some(package_id.clone()),
|
||||
NotificationLevel::Error,
|
||||
"Restoration Failure".to_string(),
|
||||
format!("Error restoring package {}: {}", package_id, err),
|
||||
(),
|
||||
None,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::error!("Failed to notify: {}", err);
|
||||
tracing::debug!("{:?}", err);
|
||||
};
|
||||
tracing::error!("Error restoring package {}: {}", package_id, err);
|
||||
stream::iter(tasks)
|
||||
.for_each_concurrent(5, |(id, res)| async move {
|
||||
match async { res.await?.await }.await {
|
||||
Ok(_) => (),
|
||||
Err(err) => {
|
||||
tracing::error!("Error restoring package {}: {}", id, err);
|
||||
tracing::debug!("{:?}", err);
|
||||
}
|
||||
}
|
||||
})
|
||||
.await;
|
||||
if let Err(e) = backup_guard.unmount().await {
|
||||
tracing::error!("Error unmounting backup drive: {}", e);
|
||||
tracing::debug!("{:?}", e);
|
||||
}
|
||||
});
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn approximate_progress(
|
||||
rpc_ctx: &RpcContext,
|
||||
progress: &mut ProgressInfo,
|
||||
) -> Result<(), Error> {
|
||||
for (id, size) in &mut progress.target_volume_size {
|
||||
let dir = rpc_ctx.datadir.join(PKG_VOLUME_DIR).join(id).join("data");
|
||||
if tokio::fs::metadata(&dir).await.is_err() {
|
||||
*size = 0;
|
||||
} else {
|
||||
*size = dir_size(&dir, None).await?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn approximate_progress_loop(
|
||||
ctx: &SetupContext,
|
||||
rpc_ctx: &RpcContext,
|
||||
mut starting_info: ProgressInfo,
|
||||
) {
|
||||
loop {
|
||||
if let Err(e) = approximate_progress(rpc_ctx, &mut starting_info).await {
|
||||
tracing::error!("Failed to approximate restore progress: {}", e);
|
||||
tracing::debug!("{:?}", e);
|
||||
} else {
|
||||
*ctx.setup_status.write().await = Some(Ok(starting_info.flatten()));
|
||||
}
|
||||
tokio::time::sleep(Duration::from_secs(1)).await;
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct ProgressInfo {
|
||||
package_installs: BTreeMap<PackageId, Arc<InstallProgress>>,
|
||||
src_volume_size: BTreeMap<PackageId, u64>,
|
||||
target_volume_size: BTreeMap<PackageId, u64>,
|
||||
}
|
||||
impl ProgressInfo {
|
||||
fn flatten(&self) -> SetupStatus {
|
||||
let mut total_bytes = 0;
|
||||
let mut bytes_transferred = 0;
|
||||
|
||||
for progress in self.package_installs.values() {
|
||||
total_bytes += ((progress.size.unwrap_or(0) as f64) * 2.2) as u64;
|
||||
bytes_transferred += progress.downloaded.load(Ordering::SeqCst);
|
||||
bytes_transferred += ((progress.validated.load(Ordering::SeqCst) as f64) * 0.2) as u64;
|
||||
bytes_transferred += progress.unpacked.load(Ordering::SeqCst);
|
||||
}
|
||||
|
||||
for size in self.src_volume_size.values() {
|
||||
total_bytes += *size;
|
||||
}
|
||||
|
||||
for size in self.target_volume_size.values() {
|
||||
bytes_transferred += *size;
|
||||
}
|
||||
|
||||
if bytes_transferred > total_bytes {
|
||||
bytes_transferred = total_bytes;
|
||||
}
|
||||
|
||||
SetupStatus {
|
||||
total_bytes: Some(total_bytes),
|
||||
bytes_transferred,
|
||||
complete: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx))]
|
||||
pub async fn recover_full_embassy(
|
||||
ctx: SetupContext,
|
||||
@@ -179,7 +81,7 @@ pub async fn recover_full_embassy(
|
||||
)
|
||||
.await?;
|
||||
|
||||
let os_backup_path = backup_guard.as_ref().join("os-backup.cbor");
|
||||
let os_backup_path = backup_guard.path().join("os-backup.cbor");
|
||||
let mut os_backup: OsBackup = IoFormat::Cbor.from_slice(
|
||||
&tokio::fs::read(&os_backup_path)
|
||||
.await
|
||||
@@ -199,11 +101,9 @@ pub async fn recover_full_embassy(
|
||||
|
||||
secret_store.close().await;
|
||||
|
||||
let cfg = RpcContextConfig::load(ctx.config_path.clone()).await?;
|
||||
init(&ctx.config).await?;
|
||||
|
||||
init(&cfg).await?;
|
||||
|
||||
let rpc_ctx = RpcContext::init(ctx.config_path.clone(), disk_guid.clone()).await?;
|
||||
let rpc_ctx = RpcContext::init(&ctx.config, disk_guid.clone()).await?;
|
||||
|
||||
let ids: Vec<_> = backup_guard
|
||||
.metadata
|
||||
@@ -211,37 +111,19 @@ pub async fn recover_full_embassy(
|
||||
.keys()
|
||||
.cloned()
|
||||
.collect();
|
||||
let (backup_guard, tasks, progress_info) =
|
||||
restore_packages(&rpc_ctx, backup_guard, ids).await?;
|
||||
let task_consumer_rpc_ctx = rpc_ctx.clone();
|
||||
tokio::select! {
|
||||
_ = async move {
|
||||
stream::iter(tasks.into_iter().map(|x| (x, task_consumer_rpc_ctx.clone())))
|
||||
.for_each_concurrent(5, |(res, ctx)| async move {
|
||||
match res.await {
|
||||
(Ok(_), _) => (),
|
||||
(Err(err), package_id) => {
|
||||
if let Err(err) = ctx.notification_manager.notify(
|
||||
ctx.db.clone(),
|
||||
Some(package_id.clone()),
|
||||
NotificationLevel::Error,
|
||||
"Restoration Failure".to_string(), format!("Error restoring package {}: {}", package_id,err), (), None).await{
|
||||
tracing::error!("Failed to notify: {}", err);
|
||||
tracing::debug!("{:?}", err);
|
||||
};
|
||||
tracing::error!("Error restoring package {}: {}", package_id, err);
|
||||
tracing::debug!("{:?}", err);
|
||||
},
|
||||
}
|
||||
}).await;
|
||||
let tasks = restore_packages(&rpc_ctx, backup_guard, ids).await?;
|
||||
stream::iter(tasks)
|
||||
.for_each_concurrent(5, |(id, res)| async move {
|
||||
match async { res.await?.await }.await {
|
||||
Ok(_) => (),
|
||||
Err(err) => {
|
||||
tracing::error!("Error restoring package {}: {}", id, err);
|
||||
tracing::debug!("{:?}", err);
|
||||
}
|
||||
}
|
||||
})
|
||||
.await;
|
||||
|
||||
} => {
|
||||
|
||||
},
|
||||
_ = approximate_progress_loop(&ctx, &rpc_ctx, progress_info) => unreachable!(concat!(module_path!(), "::approximate_progress_loop should not terminate")),
|
||||
}
|
||||
|
||||
backup_guard.unmount().await?;
|
||||
rpc_ctx.shutdown().await?;
|
||||
|
||||
Ok((
|
||||
@@ -257,205 +139,25 @@ async fn restore_packages(
|
||||
ctx: &RpcContext,
|
||||
backup_guard: BackupMountGuard<TmpMountGuard>,
|
||||
ids: Vec<PackageId>,
|
||||
) -> Result<
|
||||
(
|
||||
BackupMountGuard<TmpMountGuard>,
|
||||
Vec<BoxFuture<'static, (Result<(), Error>, PackageId)>>,
|
||||
ProgressInfo,
|
||||
),
|
||||
Error,
|
||||
> {
|
||||
let guards = assure_restoring(ctx, ids, &backup_guard).await?;
|
||||
|
||||
let mut progress_info = ProgressInfo::default();
|
||||
|
||||
let mut tasks = Vec::with_capacity(guards.len());
|
||||
for (manifest, guard) in guards {
|
||||
let id = manifest.id.clone();
|
||||
let (progress, task) = restore_package(ctx.clone(), manifest, guard).await?;
|
||||
progress_info
|
||||
.package_installs
|
||||
.insert(id.clone(), progress.clone());
|
||||
progress_info
|
||||
.src_volume_size
|
||||
.insert(id.clone(), dir_size(backup_dir(&id), None).await?);
|
||||
progress_info.target_volume_size.insert(id.clone(), 0);
|
||||
let package_id = id.clone();
|
||||
tasks.push(
|
||||
async move {
|
||||
if let Err(e) = task.await {
|
||||
tracing::error!("Error restoring package {}: {}", id, e);
|
||||
tracing::debug!("{:?}", e);
|
||||
Err(e)
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
.map(|x| (x, package_id))
|
||||
.boxed(),
|
||||
);
|
||||
}
|
||||
|
||||
Ok((backup_guard, tasks, progress_info))
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, backup_guard))]
|
||||
async fn assure_restoring(
|
||||
ctx: &RpcContext,
|
||||
ids: Vec<PackageId>,
|
||||
backup_guard: &BackupMountGuard<TmpMountGuard>,
|
||||
) -> Result<Vec<(Manifest, PackageBackupMountGuard)>, Error> {
|
||||
let mut guards = Vec::with_capacity(ids.len());
|
||||
|
||||
let mut insert_packages = BTreeMap::new();
|
||||
|
||||
) -> Result<BTreeMap<PackageId, DownloadInstallFuture>, Error> {
|
||||
let backup_guard = Arc::new(backup_guard);
|
||||
let mut tasks = BTreeMap::new();
|
||||
for id in ids {
|
||||
let peek = ctx.db.peek().await;
|
||||
|
||||
let model = peek.as_package_data().as_idx(&id);
|
||||
|
||||
if !model.is_none() {
|
||||
return Err(Error::new(
|
||||
eyre!("Can't restore over existing package: {}", id),
|
||||
crate::ErrorKind::InvalidRequest,
|
||||
));
|
||||
}
|
||||
let guard = backup_guard.mount_package_backup(&id).await?;
|
||||
let s9pk_path = Path::new(BACKUP_DIR).join(&id).join(format!("{}.s9pk", id));
|
||||
let mut rdr = S9pkReader::open(&s9pk_path, false).await?;
|
||||
|
||||
let manifest = rdr.manifest().await?;
|
||||
let version = manifest.version.clone();
|
||||
let progress = Arc::new(InstallProgress::new(Some(
|
||||
tokio::fs::metadata(&s9pk_path).await?.len(),
|
||||
)));
|
||||
|
||||
let public_dir_path = ctx
|
||||
.datadir
|
||||
.join(PKG_PUBLIC_DIR)
|
||||
.join(&id)
|
||||
.join(version.as_str());
|
||||
tokio::fs::create_dir_all(&public_dir_path).await?;
|
||||
|
||||
let license_path = public_dir_path.join("LICENSE.md");
|
||||
let mut dst = File::create(&license_path).await?;
|
||||
tokio::io::copy(&mut rdr.license().await?, &mut dst).await?;
|
||||
dst.sync_all().await?;
|
||||
|
||||
let instructions_path = public_dir_path.join("INSTRUCTIONS.md");
|
||||
let mut dst = File::create(&instructions_path).await?;
|
||||
tokio::io::copy(&mut rdr.instructions().await?, &mut dst).await?;
|
||||
dst.sync_all().await?;
|
||||
|
||||
let icon_path = Path::new("icon").with_extension(&manifest.assets.icon_type());
|
||||
let icon_path = public_dir_path.join(&icon_path);
|
||||
let mut dst = File::create(&icon_path).await?;
|
||||
tokio::io::copy(&mut rdr.icon().await?, &mut dst).await?;
|
||||
dst.sync_all().await?;
|
||||
insert_packages.insert(
|
||||
id.clone(),
|
||||
PackageDataEntry::Restoring(PackageDataEntryRestoring {
|
||||
install_progress: progress.clone(),
|
||||
static_files: StaticFiles::local(&id, &version, manifest.assets.icon_type()),
|
||||
manifest: manifest.clone(),
|
||||
}),
|
||||
);
|
||||
|
||||
guards.push((manifest, guard));
|
||||
}
|
||||
ctx.db
|
||||
.mutate(|db| {
|
||||
for (id, package) in insert_packages {
|
||||
db.as_package_data_mut().insert(&id, &package)?;
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.await?;
|
||||
Ok(guards)
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, guard))]
|
||||
async fn restore_package<'a>(
|
||||
ctx: RpcContext,
|
||||
manifest: Manifest,
|
||||
guard: PackageBackupMountGuard,
|
||||
) -> Result<(Arc<InstallProgress>, BoxFuture<'static, Result<(), Error>>), Error> {
|
||||
let id = manifest.id.clone();
|
||||
let s9pk_path = Path::new(BACKUP_DIR)
|
||||
.join(&manifest.id)
|
||||
.join(format!("{}.s9pk", id));
|
||||
|
||||
let metadata_path = Path::new(BACKUP_DIR).join(&id).join("metadata.cbor");
|
||||
let metadata: BackupMetadata = IoFormat::Cbor.from_slice(
|
||||
&tokio::fs::read(&metadata_path)
|
||||
.await
|
||||
.with_ctx(|_| (ErrorKind::Filesystem, metadata_path.display().to_string()))?,
|
||||
)?;
|
||||
|
||||
let mut secrets = ctx.secret_store.acquire().await?;
|
||||
let mut secrets_tx = secrets.begin().await?;
|
||||
for (iface, key) in metadata.network_keys {
|
||||
let k = key.0.as_slice();
|
||||
sqlx::query!(
|
||||
"INSERT INTO network_keys (package, interface, key) VALUES ($1, $2, $3) ON CONFLICT (package, interface) DO NOTHING",
|
||||
id.to_string(),
|
||||
iface.to_string(),
|
||||
k,
|
||||
)
|
||||
.execute(secrets_tx.as_mut()).await?;
|
||||
}
|
||||
// DEPRECATED
|
||||
for (iface, key) in metadata.tor_keys {
|
||||
let k = key.0.as_slice();
|
||||
sqlx::query!(
|
||||
"INSERT INTO tor (package, interface, key) VALUES ($1, $2, $3) ON CONFLICT (package, interface) DO NOTHING",
|
||||
id.to_string(),
|
||||
iface.to_string(),
|
||||
k,
|
||||
)
|
||||
.execute(secrets_tx.as_mut()).await?;
|
||||
}
|
||||
secrets_tx.commit().await?;
|
||||
drop(secrets);
|
||||
|
||||
let len = tokio::fs::metadata(&s9pk_path)
|
||||
.await
|
||||
.with_ctx(|_| (ErrorKind::Filesystem, s9pk_path.display().to_string()))?
|
||||
.len();
|
||||
let file = File::open(&s9pk_path)
|
||||
.await
|
||||
.with_ctx(|_| (ErrorKind::Filesystem, s9pk_path.display().to_string()))?;
|
||||
|
||||
let progress = InstallProgress::new(Some(len));
|
||||
let marketplace_url = metadata.marketplace_url;
|
||||
|
||||
let progress = Arc::new(progress);
|
||||
|
||||
ctx.db
|
||||
.mutate(|db| {
|
||||
db.as_package_data_mut().insert(
|
||||
&id,
|
||||
&PackageDataEntry::Restoring(PackageDataEntryRestoring {
|
||||
install_progress: progress.clone(),
|
||||
static_files: StaticFiles::local(
|
||||
&id,
|
||||
&manifest.version,
|
||||
manifest.assets.icon_type(),
|
||||
),
|
||||
manifest: manifest.clone(),
|
||||
}),
|
||||
let backup_dir = backup_guard.clone().package_backup(&id);
|
||||
let task = ctx
|
||||
.services
|
||||
.install(
|
||||
ctx.clone(),
|
||||
S9pk::open(
|
||||
backup_dir.path().join(&id).with_extension("s9pk"),
|
||||
Some(&id),
|
||||
)
|
||||
.await?,
|
||||
Some(backup_dir),
|
||||
)
|
||||
})
|
||||
.await?;
|
||||
Ok((
|
||||
progress.clone(),
|
||||
async move {
|
||||
download_install_s9pk(ctx, manifest, marketplace_url, progress, file, None).await?;
|
||||
.await?;
|
||||
tasks.insert(id, task);
|
||||
}
|
||||
|
||||
guard.unmount().await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
.boxed(),
|
||||
))
|
||||
Ok(tasks)
|
||||
}
|
||||
|
||||
@@ -1,19 +1,19 @@
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use clap::Parser;
|
||||
use color_eyre::eyre::eyre;
|
||||
use futures::TryStreamExt;
|
||||
use rpc_toolkit::command;
|
||||
use rpc_toolkit::{command, from_fn_async, HandlerExt, ParentHandler};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sqlx::{Executor, Postgres};
|
||||
|
||||
use super::{BackupTarget, BackupTargetId};
|
||||
use crate::context::RpcContext;
|
||||
use crate::context::{CliContext, RpcContext};
|
||||
use crate::disk::mount::filesystem::cifs::Cifs;
|
||||
use crate::disk::mount::filesystem::ReadOnly;
|
||||
use crate::disk::mount::guard::TmpMountGuard;
|
||||
use crate::disk::mount::guard::{GenericMountGuard, TmpMountGuard};
|
||||
use crate::disk::util::{recovery_info, EmbassyOsRecoveryInfo};
|
||||
use crate::prelude::*;
|
||||
use crate::util::display_none;
|
||||
use crate::util::serde::KeyVal;
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
@@ -26,18 +26,46 @@ pub struct CifsBackupTarget {
|
||||
embassy_os: Option<EmbassyOsRecoveryInfo>,
|
||||
}
|
||||
|
||||
#[command(subcommands(add, update, remove))]
|
||||
pub fn cifs() -> Result<(), Error> {
|
||||
Ok(())
|
||||
pub fn cifs() -> ParentHandler {
|
||||
ParentHandler::new()
|
||||
.subcommand(
|
||||
"add",
|
||||
from_fn_async(add)
|
||||
.no_display()
|
||||
.with_remote_cli::<CliContext>(),
|
||||
)
|
||||
.subcommand(
|
||||
"update",
|
||||
from_fn_async(update)
|
||||
.no_display()
|
||||
.with_remote_cli::<CliContext>(),
|
||||
)
|
||||
.subcommand(
|
||||
"remove",
|
||||
from_fn_async(remove)
|
||||
.no_display()
|
||||
.with_remote_cli::<CliContext>(),
|
||||
)
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct AddParams {
|
||||
pub hostname: String,
|
||||
pub path: PathBuf,
|
||||
pub username: String,
|
||||
pub password: Option<String>,
|
||||
}
|
||||
|
||||
#[command(display(display_none))]
|
||||
pub async fn add(
|
||||
#[context] ctx: RpcContext,
|
||||
#[arg] hostname: String,
|
||||
#[arg] path: PathBuf,
|
||||
#[arg] username: String,
|
||||
#[arg] password: Option<String>,
|
||||
ctx: RpcContext,
|
||||
AddParams {
|
||||
hostname,
|
||||
path,
|
||||
username,
|
||||
password,
|
||||
}: AddParams,
|
||||
) -> Result<KeyVal<BackupTargetId, BackupTarget>, Error> {
|
||||
let cifs = Cifs {
|
||||
hostname,
|
||||
@@ -46,7 +74,7 @@ pub async fn add(
|
||||
password,
|
||||
};
|
||||
let guard = TmpMountGuard::mount(&cifs, ReadOnly).await?;
|
||||
let embassy_os = recovery_info(&guard).await?;
|
||||
let embassy_os = recovery_info(guard.path()).await?;
|
||||
guard.unmount().await?;
|
||||
let path_string = Path::new("/").join(&cifs.path).display().to_string();
|
||||
let id: i32 = sqlx::query!(
|
||||
@@ -70,14 +98,26 @@ pub async fn add(
|
||||
})
|
||||
}
|
||||
|
||||
#[command(display(display_none))]
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct UpdateParams {
|
||||
pub id: BackupTargetId,
|
||||
pub hostname: String,
|
||||
pub path: PathBuf,
|
||||
pub username: String,
|
||||
pub password: Option<String>,
|
||||
}
|
||||
|
||||
pub async fn update(
|
||||
#[context] ctx: RpcContext,
|
||||
#[arg] id: BackupTargetId,
|
||||
#[arg] hostname: String,
|
||||
#[arg] path: PathBuf,
|
||||
#[arg] username: String,
|
||||
#[arg] password: Option<String>,
|
||||
ctx: RpcContext,
|
||||
UpdateParams {
|
||||
id,
|
||||
hostname,
|
||||
path,
|
||||
username,
|
||||
password,
|
||||
}: UpdateParams,
|
||||
) -> Result<KeyVal<BackupTargetId, BackupTarget>, Error> {
|
||||
let id = if let BackupTargetId::Cifs { id } = id {
|
||||
id
|
||||
@@ -94,7 +134,7 @@ pub async fn update(
|
||||
password,
|
||||
};
|
||||
let guard = TmpMountGuard::mount(&cifs, ReadOnly).await?;
|
||||
let embassy_os = recovery_info(&guard).await?;
|
||||
let embassy_os = recovery_info(guard.path()).await?;
|
||||
guard.unmount().await?;
|
||||
let path_string = Path::new("/").join(&cifs.path).display().to_string();
|
||||
if sqlx::query!(
|
||||
@@ -127,8 +167,14 @@ pub async fn update(
|
||||
})
|
||||
}
|
||||
|
||||
#[command(display(display_none))]
|
||||
pub async fn remove(#[context] ctx: RpcContext, #[arg] id: BackupTargetId) -> Result<(), Error> {
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct RemoveParams {
|
||||
pub id: BackupTargetId,
|
||||
}
|
||||
|
||||
pub async fn remove(ctx: RpcContext, RemoveParams { id }: RemoveParams) -> Result<(), Error> {
|
||||
let id = if let BackupTargetId::Cifs { id } = id {
|
||||
id
|
||||
} else {
|
||||
@@ -189,7 +235,7 @@ where
|
||||
};
|
||||
let embassy_os = async {
|
||||
let guard = TmpMountGuard::mount(&mount_info, ReadOnly).await?;
|
||||
let embassy_os = recovery_info(&guard).await?;
|
||||
let embassy_os = recovery_info(guard.path()).await?;
|
||||
guard.unmount().await?;
|
||||
Ok::<_, Error>(embassy_os)
|
||||
}
|
||||
|
||||
@@ -1,13 +1,14 @@
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use async_trait::async_trait;
|
||||
use chrono::{DateTime, Utc};
|
||||
use clap::ArgMatches;
|
||||
use clap::builder::ValueParserFactory;
|
||||
use clap::Parser;
|
||||
use color_eyre::eyre::eyre;
|
||||
use digest::generic_array::GenericArray;
|
||||
use digest::OutputSizeUser;
|
||||
use rpc_toolkit::command;
|
||||
use models::PackageId;
|
||||
use rpc_toolkit::{command, from_fn_async, AnyContext, HandlerExt, ParentHandler};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sha2::Sha256;
|
||||
use sqlx::{Executor, Postgres};
|
||||
@@ -15,17 +16,19 @@ use tokio::sync::Mutex;
|
||||
use tracing::instrument;
|
||||
|
||||
use self::cifs::CifsBackupTarget;
|
||||
use crate::context::RpcContext;
|
||||
use crate::context::{CliContext, RpcContext};
|
||||
use crate::disk::mount::backup::BackupMountGuard;
|
||||
use crate::disk::mount::filesystem::block_dev::BlockDev;
|
||||
use crate::disk::mount::filesystem::cifs::Cifs;
|
||||
use crate::disk::mount::filesystem::{FileSystem, MountType, ReadWrite};
|
||||
use crate::disk::mount::guard::TmpMountGuard;
|
||||
use crate::disk::mount::guard::{GenericMountGuard, TmpMountGuard};
|
||||
use crate::disk::util::PartitionInfo;
|
||||
use crate::prelude::*;
|
||||
use crate::s9pk::manifest::PackageId;
|
||||
use crate::util::serde::{deserialize_from_str, display_serializable, serialize_display};
|
||||
use crate::util::{display_none, Version};
|
||||
use crate::util::clap::FromStrParser;
|
||||
use crate::util::serde::{
|
||||
deserialize_from_str, display_serializable, serialize_display, HandlerExtSerde, WithIoFormat,
|
||||
};
|
||||
use crate::util::Version;
|
||||
|
||||
pub mod cifs;
|
||||
|
||||
@@ -84,6 +87,12 @@ impl std::str::FromStr for BackupTargetId {
|
||||
}
|
||||
}
|
||||
}
|
||||
impl ValueParserFactory for BackupTargetId {
|
||||
type Parser = FromStrParser<Self>;
|
||||
fn value_parser() -> Self::Parser {
|
||||
FromStrParser::new()
|
||||
}
|
||||
}
|
||||
impl<'de> Deserialize<'de> for BackupTargetId {
|
||||
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
|
||||
where
|
||||
@@ -108,9 +117,8 @@ pub enum BackupTargetFS {
|
||||
Disk(BlockDev<PathBuf>),
|
||||
Cifs(Cifs),
|
||||
}
|
||||
#[async_trait]
|
||||
impl FileSystem for BackupTargetFS {
|
||||
async fn mount<P: AsRef<Path> + Send + Sync>(
|
||||
async fn mount<P: AsRef<Path> + Send>(
|
||||
&self,
|
||||
mountpoint: P,
|
||||
mount_type: MountType,
|
||||
@@ -130,15 +138,29 @@ impl FileSystem for BackupTargetFS {
|
||||
}
|
||||
}
|
||||
|
||||
#[command(subcommands(cifs::cifs, list, info, mount, umount))]
|
||||
pub fn target() -> Result<(), Error> {
|
||||
Ok(())
|
||||
// #[command(subcommands(cifs::cifs, list, info, mount, umount))]
|
||||
pub fn target() -> ParentHandler {
|
||||
ParentHandler::new()
|
||||
.subcommand("cifs", cifs::cifs())
|
||||
.subcommand(
|
||||
"list",
|
||||
from_fn_async(list)
|
||||
.with_display_serializable()
|
||||
.with_remote_cli::<CliContext>(),
|
||||
)
|
||||
.subcommand(
|
||||
"info",
|
||||
from_fn_async(info)
|
||||
.with_display_serializable()
|
||||
.with_custom_display_fn::<AnyContext, _>(|params, info| {
|
||||
Ok(display_backup_info(params.params, info))
|
||||
})
|
||||
.with_remote_cli::<CliContext>(),
|
||||
)
|
||||
}
|
||||
|
||||
#[command(display(display_serializable))]
|
||||
pub async fn list(
|
||||
#[context] ctx: RpcContext,
|
||||
) -> Result<BTreeMap<BackupTargetId, BackupTarget>, Error> {
|
||||
// #[command(display(display_serializable))]
|
||||
pub async fn list(ctx: RpcContext) -> Result<BTreeMap<BackupTargetId, BackupTarget>, Error> {
|
||||
let mut sql_handle = ctx.secret_store.acquire().await?;
|
||||
let (disks_res, cifs) = tokio::try_join!(
|
||||
crate::disk::util::list(&ctx.os_partitions),
|
||||
@@ -187,11 +209,11 @@ pub struct PackageBackupInfo {
|
||||
pub timestamp: DateTime<Utc>,
|
||||
}
|
||||
|
||||
fn display_backup_info(info: BackupInfo, matches: &ArgMatches) {
|
||||
fn display_backup_info(params: WithIoFormat<InfoParams>, info: BackupInfo) {
|
||||
use prettytable::*;
|
||||
|
||||
if matches.is_present("format") {
|
||||
return display_serializable(info, matches);
|
||||
if let Some(format) = params.format {
|
||||
return display_serializable(format, info);
|
||||
}
|
||||
|
||||
let mut table = Table::new();
|
||||
@@ -223,12 +245,21 @@ fn display_backup_info(info: BackupInfo, matches: &ArgMatches) {
|
||||
table.print_tty(false).unwrap();
|
||||
}
|
||||
|
||||
#[command(display(display_backup_info))]
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct InfoParams {
|
||||
target_id: BackupTargetId,
|
||||
password: String,
|
||||
}
|
||||
|
||||
#[instrument(skip(ctx, password))]
|
||||
pub async fn info(
|
||||
#[context] ctx: RpcContext,
|
||||
#[arg(rename = "target-id")] target_id: BackupTargetId,
|
||||
#[arg] password: String,
|
||||
ctx: RpcContext,
|
||||
InfoParams {
|
||||
target_id,
|
||||
password,
|
||||
}: InfoParams,
|
||||
) -> Result<BackupInfo, Error> {
|
||||
let guard = BackupMountGuard::mount(
|
||||
TmpMountGuard::mount(
|
||||
@@ -254,17 +285,26 @@ lazy_static::lazy_static! {
|
||||
Mutex::new(BTreeMap::new());
|
||||
}
|
||||
|
||||
#[command]
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct MountParams {
|
||||
target_id: BackupTargetId,
|
||||
password: String,
|
||||
}
|
||||
|
||||
#[instrument(skip_all)]
|
||||
pub async fn mount(
|
||||
#[context] ctx: RpcContext,
|
||||
#[arg(rename = "target-id")] target_id: BackupTargetId,
|
||||
#[arg] password: String,
|
||||
ctx: RpcContext,
|
||||
MountParams {
|
||||
target_id,
|
||||
password,
|
||||
}: MountParams,
|
||||
) -> Result<String, Error> {
|
||||
let mut mounts = USER_MOUNTS.lock().await;
|
||||
|
||||
if let Some(existing) = mounts.get(&target_id) {
|
||||
return Ok(existing.as_ref().display().to_string());
|
||||
return Ok(existing.path().display().to_string());
|
||||
}
|
||||
|
||||
let guard = BackupMountGuard::mount(
|
||||
@@ -280,19 +320,23 @@ pub async fn mount(
|
||||
)
|
||||
.await?;
|
||||
|
||||
let res = guard.as_ref().display().to_string();
|
||||
let res = guard.path().display().to_string();
|
||||
|
||||
mounts.insert(target_id, guard);
|
||||
|
||||
Ok(res)
|
||||
}
|
||||
#[command(display(display_none))]
|
||||
|
||||
#[derive(Deserialize, Serialize, Parser)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
#[command(rename_all = "kebab-case")]
|
||||
pub struct UmountParams {
|
||||
target_id: Option<BackupTargetId>,
|
||||
}
|
||||
|
||||
#[instrument(skip_all)]
|
||||
pub async fn umount(
|
||||
#[context] _ctx: RpcContext,
|
||||
#[arg(rename = "target-id")] target_id: Option<BackupTargetId>,
|
||||
) -> Result<(), Error> {
|
||||
let mut mounts = USER_MOUNTS.lock().await;
|
||||
pub async fn umount(_: RpcContext, UmountParams { target_id }: UmountParams) -> Result<(), Error> {
|
||||
let mut mounts = USER_MOUNTS.lock().await; // TODO: move to context
|
||||
if let Some(target_id) = target_id {
|
||||
if let Some(existing) = mounts.remove(&target_id) {
|
||||
existing.unmount().await?;
|
||||
|
||||
Reference in New Issue
Block a user