Feature/lxc container runtime (#2514)

* wip: static-server errors

* wip: fix wifi

* wip: Fix the service_effects

* wip: Fix cors in the middleware

* wip(chore): Auth clean up the lint.

* wip(fix): Vhost

* wip: continue manager refactor

Co-authored-by: J H <Blu-J@users.noreply.github.com>

* wip: service manager refactor

* wip: Some fixes

* wip(fix): Fix the lib.rs

* wip

* wip(fix): Logs

* wip: bins

* wip(innspect): Add in the inspect

* wip: config

* wip(fix): Diagnostic

* wip(fix): Dependencies

* wip: context

* wip(fix) Sorta auth

* wip: warnings

* wip(fix): registry/admin

* wip(fix) marketplace

* wip(fix) Some more converted and fixed with the linter and config

* wip: Working on the static server

* wip(fix)static server

* wip: Remove some asynnc

* wip: Something about the request and regular rpc

* wip: gut install

Co-authored-by: J H <Blu-J@users.noreply.github.com>

* wip: Convert the static server into the new system

* wip delete file

* test

* wip(fix) vhost does not need the with safe defaults

* wip: Adding in the wifi

* wip: Fix the developer and the verify

* wip: new install flow

Co-authored-by: J H <Blu-J@users.noreply.github.com>

* fix middleware

* wip

* wip: Fix the auth

* wip

* continue service refactor

* feature: Service get_config

* feat: Action

* wip: Fighting the great fight against the borrow checker

* wip: Remove an error in a file that I just need to deel with later

* chore: Add in some more lifetime stuff to the services

* wip: Install fix on lifetime

* cleanup

* wip: Deal with the borrow later

* more cleanup

* resolve borrowchecker errors

* wip(feat): add in the handler for the socket, for now

* wip(feat): Update the service_effect_handler::action

* chore: Add in the changes to make sure the from_service goes to context

* chore: Change the

* refactor service map

* fix references to service map

* fill out restore

* wip: Before I work on the store stuff

* fix backup module

* handle some warnings

* feat: add in the ui components on the rust side

* feature: Update the procedures

* chore: Update the js side of the main and a few of the others

* chore: Update the rpc listener to match the persistant container

* wip: Working on updating some things to have a better name

* wip(feat): Try and get the rpc to return the correct shape?

* lxc wip

* wip(feat): Try and get the rpc to return the correct shape?

* build for container runtime wip

* remove container-init

* fix build

* fix error

* chore: Update to work I suppose

* lxc wip

* remove docker module and feature

* download alpine squashfs automatically

* overlays effect

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* chore: Add the overlay effect

* feat: Add the mounter in the main

* chore: Convert to use the mounts, still need to work with the sandbox

* install fixes

* fix ssl

* fixes from testing

* implement tmpfile for upload

* wip

* misc fixes

* cleanup

* cleanup

* better progress reporting

* progress for sideload

* return real guid

* add devmode script

* fix lxc rootfs path

* fix percentage bar

* fix progress bar styling

* fix build for unstable

* tweaks

* label progress

* tweaks

* update progress more often

* make symlink in rpc_client

* make socket dir

* fix parent path

* add start-cli to container

* add echo and gitInfo commands

* wip: Add the init + errors

* chore: Add in the exit effect for the system

* chore: Change the type to null for failure to parse

* move sigterm timeout to stopping status

* update order

* chore: Update the return type

* remove dbg

* change the map error

* chore: Update the thing to capture id

* chore add some life changes

* chore: Update the loging

* chore: Update the package to run module

* us From for RpcError

* chore: Update to use import instead

* chore: update

* chore: Use require for the backup

* fix a default

* update the type that is wrong

* chore: Update the type of the manifest

* chore: Update to make null

* only symlink if not exists

* get rid of double result

* better debug info for ErrorCollection

* chore: Update effects

* chore: fix

* mount assets and volumes

* add exec instead of spawn

* fix mounting in image

* fix overlay mounts

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* misc fixes

* feat: Fix two

* fix: systemForEmbassy main

* chore: Fix small part of main loop

* chore: Modify the bundle

* merge

* fixMain loop"

* move tsc to makefile

* chore: Update the return types of the health check

* fix client

* chore: Convert the todo to use tsmatches

* add in the fixes for the seen and create the hack to allow demo

* chore: Update to include the systemForStartOs

* chore UPdate to the latest types from the expected outout

* fixes

* fix typo

* Don't emit if failure on tsc

* wip

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* add s9pk api

* add inspection

* add inspect manifest

* newline after display serializable

* fix squashfs in image name

* edit manifest

Co-authored-by: Jade <Blu-J@users.noreply.github.com>

* wait for response on repl

* ignore sig for now

* ignore sig for now

* re-enable sig verification

* fix

* wip

* env and chroot

* add profiling logs

* set uid & gid in squashfs to 100000

* set uid of sqfs to 100000

* fix mksquashfs args

* add env to compat

* fix

* re-add docker feature flag

* fix docker output format being stupid

* here be dragons

* chore: Add in the cross compiling for something

* fix npm link

* extract logs from container on exit

* chore: Update for testing

* add log capture to drop trait

* chore: add in the modifications that I make

* chore: Update small things for no updates

* chore: Update the types of something

* chore: Make main not complain

* idmapped mounts

* idmapped volumes

* re-enable kiosk

* chore: Add in some logging for the new system

* bring in start-sdk

* remove avahi

* chore: Update the deps

* switch to musl

* chore: Update the version of prettier

* chore: Organize'

* chore: Update some of the headers back to the standard of fetch

* fix musl build

* fix idmapped mounts

* fix cross build

* use cross compiler for correct arch

* feat: Add in the faked ssl stuff for the effects

* @dr_bonez Did a solution here

* chore: Something that DrBonez

* chore: up

* wip: We have a working server!!!

* wip

* uninstall

* wip

* tes

---------

Co-authored-by: J H <dragondef@gmail.com>
Co-authored-by: J H <Blu-J@users.noreply.github.com>
Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com>
This commit is contained in:
Aiden McClelland
2024-02-17 11:14:14 -07:00
committed by GitHub
parent 65009e2f69
commit fab13db4b4
326 changed files with 31708 additions and 13987 deletions

View File

@@ -7,8 +7,8 @@ use tracing::instrument;
use super::fsck::{RepairStrategy, RequiresReboot};
use super::util::pvscan;
use crate::disk::mount::filesystem::block_dev::mount;
use crate::disk::mount::filesystem::ReadWrite;
use crate::disk::mount::filesystem::block_dev::BlockDev;
use crate::disk::mount::filesystem::{FileSystem, ReadWrite};
use crate::disk::mount::util::unmount;
use crate::util::Invoke;
use crate::{Error, ErrorKind, ResultExt};
@@ -142,7 +142,9 @@ pub async fn create_fs<P: AsRef<Path>>(
.arg(&blockdev_path)
.invoke(crate::ErrorKind::DiskManagement)
.await?;
mount(&blockdev_path, datadir.as_ref().join(name), ReadWrite).await?;
BlockDev::new(&blockdev_path)
.mount(datadir.as_ref().join(name), ReadWrite)
.await?;
Ok(())
}
@@ -318,7 +320,9 @@ pub async fn mount_fs<P: AsRef<Path>>(
tokio::fs::rename(&tmp_luks_bak, &luks_bak).await?;
}
mount(&blockdev_path, datadir.as_ref().join(name), ReadWrite).await?;
BlockDev::new(&blockdev_path)
.mount(datadir.as_ref().join(name), ReadWrite)
.await?;
Ok(reboot)
}

View File

@@ -1,13 +1,11 @@
use std::path::{Path, PathBuf};
use clap::ArgMatches;
use rpc_toolkit::command;
use rpc_toolkit::{from_fn_async, AnyContext, Empty, HandlerExt, ParentHandler};
use serde::{Deserialize, Serialize};
use crate::context::RpcContext;
use crate::context::{CliContext, RpcContext};
use crate::disk::util::DiskInfo;
use crate::util::display_none;
use crate::util::serde::{display_serializable, IoFormat};
use crate::util::serde::{display_serializable, HandlerExtSerde, WithIoFormat};
use crate::Error;
pub mod fsck;
@@ -42,16 +40,30 @@ impl OsPartitionInfo {
}
}
#[command(subcommands(list, repair))]
pub fn disk() -> Result<(), Error> {
Ok(())
pub fn disk() -> ParentHandler {
ParentHandler::new()
.subcommand(
"list",
from_fn_async(list)
.with_display_serializable()
.with_custom_display_fn::<AnyContext, _>(|handle, result| {
Ok(display_disk_info(handle.params, result))
})
.with_remote_cli::<CliContext>(),
)
.subcommand(
"repair",
from_fn_async(repair)
.no_display()
.with_remote_cli::<CliContext>(),
)
}
fn display_disk_info(info: Vec<DiskInfo>, matches: &ArgMatches) {
fn display_disk_info(params: WithIoFormat<Empty>, args: Vec<DiskInfo>) {
use prettytable::*;
if matches.is_present("format") {
return display_serializable(info, matches);
if let Some(format) = params.format {
return display_serializable(format, args);
}
let mut table = Table::new();
@@ -60,9 +72,9 @@ fn display_disk_info(info: Vec<DiskInfo>, matches: &ArgMatches) {
"LABEL",
"CAPACITY",
"USED",
"EMBASSY OS VERSION"
"STARTOS VERSION"
]);
for disk in info {
for disk in args {
let row = row![
disk.logicalname.display(),
"N/A",
@@ -101,17 +113,11 @@ fn display_disk_info(info: Vec<DiskInfo>, matches: &ArgMatches) {
table.print_tty(false).unwrap();
}
#[command(display(display_disk_info))]
pub async fn list(
#[context] ctx: RpcContext,
#[allow(unused_variables)]
#[arg]
format: Option<IoFormat>,
) -> Result<Vec<DiskInfo>, Error> {
// #[command(display(display_disk_info))]
pub async fn list(ctx: RpcContext, _: Empty) -> Result<Vec<DiskInfo>, Error> {
crate::disk::util::list(&ctx.os_partitions).await
}
#[command(display(display_none))]
pub async fn repair() -> Result<(), Error> {
tokio::fs::write(REPAIR_DISK_PATH, b"").await?;
Ok(())

View File

@@ -1,24 +1,24 @@
use std::path::{Path, PathBuf};
use std::sync::Arc;
use color_eyre::eyre::eyre;
use helpers::AtomicFile;
use models::PackageId;
use tokio::io::AsyncWriteExt;
use tracing::instrument;
use super::filesystem::ecryptfs::EcryptFS;
use super::guard::{GenericMountGuard, TmpMountGuard};
use super::util::{bind, unmount};
use crate::auth::check_password;
use crate::backup::target::BackupInfo;
use crate::disk::mount::filesystem::ReadWrite;
use crate::disk::mount::guard::SubPath;
use crate::disk::util::EmbassyOsRecoveryInfo;
use crate::middleware::encrypt::{decrypt_slice, encrypt_slice};
use crate::s9pk::manifest::PackageId;
use crate::util::crypto::{decrypt_slice, encrypt_slice};
use crate::util::serde::IoFormat;
use crate::util::FileLock;
use crate::volume::BACKUP_DIR;
use crate::{Error, ErrorKind, ResultExt};
#[derive(Clone, Debug)]
pub struct BackupMountGuard<G: GenericMountGuard> {
backup_disk_mount_guard: Option<G>,
encrypted_guard: Option<TmpMountGuard>,
@@ -29,7 +29,7 @@ pub struct BackupMountGuard<G: GenericMountGuard> {
impl<G: GenericMountGuard> BackupMountGuard<G> {
fn backup_disk_path(&self) -> &Path {
if let Some(guard) = &self.backup_disk_mount_guard {
guard.as_ref()
guard.path()
} else {
unreachable!()
}
@@ -37,7 +37,7 @@ impl<G: GenericMountGuard> BackupMountGuard<G> {
#[instrument(skip_all)]
pub async fn mount(backup_disk_mount_guard: G, password: &str) -> Result<Self, Error> {
let backup_disk_path = backup_disk_mount_guard.as_ref();
let backup_disk_path = backup_disk_mount_guard.path();
let unencrypted_metadata_path =
backup_disk_path.join("EmbassyBackups/unencrypted-metadata.cbor");
let mut unencrypted_metadata: EmbassyOsRecoveryInfo =
@@ -108,7 +108,7 @@ impl<G: GenericMountGuard> BackupMountGuard<G> {
let encrypted_guard =
TmpMountGuard::mount(&EcryptFS::new(&crypt_path, &enc_key), ReadWrite).await?;
let metadata_path = encrypted_guard.as_ref().join("metadata.cbor");
let metadata_path = encrypted_guard.path().join("metadata.cbor");
let metadata: BackupInfo = if tokio::fs::metadata(&metadata_path).await.is_ok() {
IoFormat::Cbor.from_slice(&tokio::fs::read(&metadata_path).await.with_ctx(|_| {
(
@@ -146,22 +146,13 @@ impl<G: GenericMountGuard> BackupMountGuard<G> {
}
#[instrument(skip_all)]
pub async fn mount_package_backup(
&self,
id: &PackageId,
) -> Result<PackageBackupMountGuard, Error> {
let lock = FileLock::new(Path::new(BACKUP_DIR).join(format!("{}.lock", id)), false).await?;
let mountpoint = Path::new(BACKUP_DIR).join(id);
bind(self.as_ref().join(id), &mountpoint, false).await?;
Ok(PackageBackupMountGuard {
mountpoint: Some(mountpoint),
lock: Some(lock),
})
pub fn package_backup(self: &Arc<Self>, id: &PackageId) -> SubPath<Arc<Self>> {
SubPath::new(self.clone(), id)
}
#[instrument(skip_all)]
pub async fn save(&self) -> Result<(), Error> {
let metadata_path = self.as_ref().join("metadata.cbor");
let metadata_path = self.path().join("metadata.cbor");
let backup_disk_path = self.backup_disk_path();
let mut file = AtomicFile::new(&metadata_path, None::<PathBuf>)
.await
@@ -181,7 +172,22 @@ impl<G: GenericMountGuard> BackupMountGuard<G> {
}
#[instrument(skip_all)]
pub async fn unmount(mut self) -> Result<(), Error> {
pub async fn save_and_unmount(self) -> Result<(), Error> {
self.save().await?;
self.unmount().await?;
Ok(())
}
}
#[async_trait::async_trait]
impl<G: GenericMountGuard> GenericMountGuard for BackupMountGuard<G> {
fn path(&self) -> &Path {
if let Some(guard) = &self.encrypted_guard {
guard.path()
} else {
unreachable!()
}
}
async fn unmount(mut self) -> Result<(), Error> {
if let Some(guard) = self.encrypted_guard.take() {
guard.unmount().await?;
}
@@ -190,22 +196,6 @@ impl<G: GenericMountGuard> BackupMountGuard<G> {
}
Ok(())
}
#[instrument(skip_all)]
pub async fn save_and_unmount(self) -> Result<(), Error> {
self.save().await?;
self.unmount().await?;
Ok(())
}
}
impl<G: GenericMountGuard> AsRef<Path> for BackupMountGuard<G> {
fn as_ref(&self) -> &Path {
if let Some(guard) = &self.encrypted_guard {
guard.as_ref()
} else {
unreachable!()
}
}
}
impl<G: GenericMountGuard> Drop for BackupMountGuard<G> {
fn drop(&mut self) {
@@ -221,42 +211,3 @@ impl<G: GenericMountGuard> Drop for BackupMountGuard<G> {
});
}
}
pub struct PackageBackupMountGuard {
mountpoint: Option<PathBuf>,
lock: Option<FileLock>,
}
impl PackageBackupMountGuard {
pub async fn unmount(mut self) -> Result<(), Error> {
if let Some(mountpoint) = self.mountpoint.take() {
unmount(&mountpoint).await?;
}
if let Some(lock) = self.lock.take() {
lock.unlock().await?;
}
Ok(())
}
}
impl AsRef<Path> for PackageBackupMountGuard {
fn as_ref(&self) -> &Path {
if let Some(mountpoint) = &self.mountpoint {
mountpoint
} else {
unreachable!()
}
}
}
impl Drop for PackageBackupMountGuard {
fn drop(&mut self) {
let mountpoint = self.mountpoint.take();
let lock = self.lock.take();
tokio::spawn(async move {
if let Some(mountpoint) = mountpoint {
unmount(&mountpoint).await.unwrap();
}
if let Some(lock) = lock {
lock.unlock().await.unwrap();
}
});
}
}

View File

@@ -1,14 +1,12 @@
use std::os::unix::ffi::OsStrExt;
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use sha2::Sha256;
use super::{FileSystem, MountType, ReadOnly};
use crate::disk::mount::util::bind;
use crate::{Error, ResultExt};
use super::FileSystem;
use crate::prelude::*;
pub struct Bind<SrcDir: AsRef<Path>> {
src_dir: SrcDir,
@@ -18,19 +16,16 @@ impl<SrcDir: AsRef<Path>> Bind<SrcDir> {
Self { src_dir }
}
}
#[async_trait]
impl<SrcDir: AsRef<Path> + Send + Sync> FileSystem for Bind<SrcDir> {
async fn mount<P: AsRef<Path> + Send + Sync>(
&self,
mountpoint: P,
mount_type: MountType,
) -> Result<(), Error> {
bind(
self.src_dir.as_ref(),
mountpoint,
matches!(mount_type, ReadOnly),
)
.await
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
Ok(Some(&self.src_dir))
}
fn extra_args(&self) -> impl IntoIterator<Item = impl AsRef<std::ffi::OsStr>> {
["--bind"]
}
async fn pre_mount(&self) -> Result<(), Error> {
tokio::fs::create_dir_all(self.src_dir.as_ref()).await?;
Ok(())
}
async fn source_hash(
&self,

View File

@@ -1,30 +1,13 @@
use std::os::unix::ffi::OsStrExt;
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use super::{FileSystem, MountType, ReadOnly};
use crate::util::Invoke;
use crate::{Error, ResultExt};
pub async fn mount(
logicalname: impl AsRef<Path>,
mountpoint: impl AsRef<Path>,
mount_type: MountType,
) -> Result<(), Error> {
tokio::fs::create_dir_all(mountpoint.as_ref()).await?;
let mut cmd = tokio::process::Command::new("mount");
cmd.arg(logicalname.as_ref()).arg(mountpoint.as_ref());
if mount_type == ReadOnly {
cmd.arg("-o").arg("ro");
}
cmd.invoke(crate::ErrorKind::Filesystem).await?;
Ok(())
}
use super::FileSystem;
use crate::prelude::*;
#[derive(Debug, Deserialize, Serialize)]
#[serde(rename_all = "kebab-case")]
@@ -36,14 +19,9 @@ impl<LogicalName: AsRef<Path>> BlockDev<LogicalName> {
BlockDev { logicalname }
}
}
#[async_trait]
impl<LogicalName: AsRef<Path> + Send + Sync> FileSystem for BlockDev<LogicalName> {
async fn mount<P: AsRef<Path> + Send + Sync>(
&self,
mountpoint: P,
mount_type: MountType,
) -> Result<(), Error> {
mount(self.logicalname.as_ref(), mountpoint, mount_type).await
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
Ok(Some(&self.logicalname))
}
async fn source_hash(
&self,

View File

@@ -2,7 +2,6 @@ use std::net::IpAddr;
use std::os::unix::ffi::OsStrExt;
use std::path::{Path, PathBuf};
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use serde::{Deserialize, Serialize};
@@ -11,7 +10,7 @@ use tokio::process::Command;
use tracing::instrument;
use super::{FileSystem, MountType, ReadOnly};
use crate::disk::mount::guard::TmpMountGuard;
use crate::disk::mount::guard::{GenericMountGuard, TmpMountGuard};
use crate::util::Invoke;
use crate::Error;
@@ -78,9 +77,8 @@ impl Cifs {
Ok(())
}
}
#[async_trait]
impl FileSystem for Cifs {
async fn mount<P: AsRef<std::path::Path> + Send + Sync>(
async fn mount<P: AsRef<std::path::Path> + Send>(
&self,
mountpoint: P,
mount_type: MountType,

View File

@@ -1,33 +1,17 @@
use std::fmt::Display;
use std::os::unix::ffi::OsStrExt;
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use lazy_format::lazy_format;
use sha2::Sha256;
use tokio::process::Command;
use super::{FileSystem, MountType};
use super::FileSystem;
use crate::disk::mount::filesystem::default_mount_command;
use crate::prelude::*;
use crate::util::Invoke;
use crate::{Error, ResultExt};
pub async fn mount_ecryptfs<P0: AsRef<Path>, P1: AsRef<Path>>(
src: P0,
dst: P1,
key: &str,
) -> Result<(), Error> {
tokio::fs::create_dir_all(dst.as_ref()).await?;
tokio::process::Command::new("mount")
.arg("-t")
.arg("ecryptfs")
.arg(src.as_ref())
.arg(dst.as_ref())
.arg("-o")
// for more information `man ecryptfs`
.arg(format!("key=passphrase:passphrase_passwd={},ecryptfs_cipher=aes,ecryptfs_key_bytes=32,ecryptfs_passthrough=n,ecryptfs_enable_filename_crypto=y,no_sig_cache", key))
.input(Some(&mut std::io::Cursor::new(b"\n")))
.invoke(crate::ErrorKind::Filesystem).await?;
Ok(())
}
pub struct EcryptFS<EncryptedDir: AsRef<Path>, Key: AsRef<str>> {
encrypted_dir: EncryptedDir,
@@ -38,16 +22,45 @@ impl<EncryptedDir: AsRef<Path>, Key: AsRef<str>> EcryptFS<EncryptedDir, Key> {
EcryptFS { encrypted_dir, key }
}
}
#[async_trait]
impl<EncryptedDir: AsRef<Path> + Send + Sync, Key: AsRef<str> + Send + Sync> FileSystem
for EcryptFS<EncryptedDir, Key>
{
async fn mount<P: AsRef<Path> + Send + Sync>(
fn mount_type(&self) -> Option<impl AsRef<str>> {
Some("ecryptfs")
}
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
Ok(Some(&self.encrypted_dir))
}
fn mount_options(&self) -> impl IntoIterator<Item = impl Display> {
[
Box::new(lazy_format!(
"key=passphrase:passphrase_passwd={}",
self.key.as_ref()
)) as Box<dyn Display>,
Box::new("ecryptfs_cipher=aes"),
Box::new("ecryptfs_key_bytes=32"),
Box::new("ecryptfs_passthrough=n"),
Box::new("ecryptfs_enable_filename_crypto=y"),
Box::new("no_sig_cache"),
]
}
async fn mount<P: AsRef<Path> + Send>(
&self,
mountpoint: P,
_mount_type: MountType, // ignored - inherited from parent fs
mount_type: super::MountType,
) -> Result<(), Error> {
mount_ecryptfs(self.encrypted_dir.as_ref(), mountpoint, self.key.as_ref()).await
self.pre_mount().await?;
tokio::fs::create_dir_all(mountpoint.as_ref()).await?;
Command::new("mount")
.args(
default_mount_command(self, mountpoint, mount_type)
.await?
.get_args(),
)
.input(Some(&mut std::io::Cursor::new(b"\n")))
.invoke(crate::ErrorKind::Filesystem)
.await?;
Ok(())
}
async fn source_hash(
&self,

View File

@@ -1,33 +1,19 @@
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use sha2::Sha256;
use super::{FileSystem, MountType, ReadOnly};
use crate::util::Invoke;
use crate::Error;
use super::FileSystem;
use crate::prelude::*;
pub struct EfiVarFs;
#[async_trait]
impl FileSystem for EfiVarFs {
async fn mount<P: AsRef<Path> + Send + Sync>(
&self,
mountpoint: P,
mount_type: MountType,
) -> Result<(), Error> {
tokio::fs::create_dir_all(mountpoint.as_ref()).await?;
let mut cmd = tokio::process::Command::new("mount");
cmd.arg("-t")
.arg("efivarfs")
.arg("efivarfs")
.arg(mountpoint.as_ref());
if mount_type == ReadOnly {
cmd.arg("-o").arg("ro");
}
cmd.invoke(crate::ErrorKind::Filesystem).await?;
Ok(())
fn mount_type(&self) -> Option<impl AsRef<str>> {
Some("efivarfs")
}
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
Ok(Some("efivarfs"))
}
async fn source_hash(
&self,

View File

@@ -1,6 +1,5 @@
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use reqwest::Url;
@@ -32,9 +31,8 @@ impl HttpDirFS {
HttpDirFS { url }
}
}
#[async_trait]
impl FileSystem for HttpDirFS {
async fn mount<P: AsRef<Path> + Send + Sync>(
async fn mount<P: AsRef<Path> + Send>(
&self,
mountpoint: P,
_mount_type: MountType,

View File

@@ -0,0 +1,88 @@
use std::ffi::OsStr;
use std::fmt::Display;
use std::path::Path;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use tokio::process::Command;
use super::{FileSystem, MountType};
use crate::disk::mount::filesystem::default_mount_command;
use crate::prelude::*;
use crate::util::Invoke;
#[derive(Debug, Deserialize, Serialize)]
#[serde(rename_all = "kebab-case")]
pub struct IdMapped<Fs: FileSystem> {
filesystem: Fs,
from_id: u32,
to_id: u32,
range: u32,
}
impl<Fs: FileSystem> IdMapped<Fs> {
pub fn new(filesystem: Fs, from_id: u32, to_id: u32, range: u32) -> Self {
Self {
filesystem,
from_id,
to_id,
range,
}
}
}
impl<Fs: FileSystem> FileSystem for IdMapped<Fs> {
fn mount_type(&self) -> Option<impl AsRef<str>> {
self.filesystem.mount_type()
}
fn extra_args(&self) -> impl IntoIterator<Item = impl AsRef<OsStr>> {
self.filesystem.extra_args()
}
fn mount_options(&self) -> impl IntoIterator<Item = impl Display> {
self.filesystem
.mount_options()
.into_iter()
.map(|a| Box::new(a) as Box<dyn Display>)
.chain(std::iter::once(Box::new(lazy_format!(
"X-mount.idmap=b:{}:{}:{}",
self.from_id,
self.to_id,
self.range,
)) as Box<dyn Display>))
}
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
self.filesystem.source().await
}
async fn pre_mount(&self) -> Result<(), Error> {
self.filesystem.pre_mount().await
}
async fn mount<P: AsRef<Path> + Send>(
&self,
mountpoint: P,
mount_type: MountType,
) -> Result<(), Error> {
self.pre_mount().await?;
tokio::fs::create_dir_all(mountpoint.as_ref()).await?;
Command::new("mount.next")
.args(
default_mount_command(self, mountpoint, mount_type)
.await?
.get_args(),
)
.invoke(ErrorKind::Filesystem)
.await?;
Ok(())
}
async fn source_hash(
&self,
) -> Result<GenericArray<u8, <Sha256 as OutputSizeUser>::OutputSize>, Error> {
let mut sha = Sha256::new();
sha.update("IdMapped");
sha.update(self.filesystem.source_hash().await?);
sha.update(u32::to_be_bytes(self.from_id));
sha.update(u32::to_be_bytes(self.to_id));
sha.update(u32::to_be_bytes(self.range));
Ok(sha.finalize())
}
}

View File

@@ -1,28 +1,11 @@
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use sha2::Sha256;
use super::{FileSystem, MountType, ReadOnly};
use crate::util::Invoke;
use crate::Error;
pub async fn mount_label(
label: &str,
mountpoint: impl AsRef<Path>,
mount_type: MountType,
) -> Result<(), Error> {
tokio::fs::create_dir_all(mountpoint.as_ref()).await?;
let mut cmd = tokio::process::Command::new("mount");
cmd.arg("-L").arg(label).arg(mountpoint.as_ref());
if mount_type == ReadOnly {
cmd.arg("-o").arg("ro");
}
cmd.invoke(crate::ErrorKind::Filesystem).await?;
Ok(())
}
use super::FileSystem;
use crate::prelude::*;
pub struct Label<S: AsRef<str>> {
label: S,
@@ -32,14 +15,12 @@ impl<S: AsRef<str>> Label<S> {
Label { label }
}
}
#[async_trait]
impl<S: AsRef<str> + Send + Sync> FileSystem for Label<S> {
async fn mount<P: AsRef<Path> + Send + Sync>(
&self,
mountpoint: P,
mount_type: MountType,
) -> Result<(), Error> {
mount_label(self.label.as_ref(), mountpoint, mount_type).await
fn extra_args(&self) -> impl IntoIterator<Item = impl AsRef<std::ffi::OsStr>> {
["-L", self.label.as_ref()]
}
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
Ok(None::<&Path>)
}
async fn source_hash(
&self,

View File

@@ -1,38 +1,15 @@
use std::fmt::Display;
use std::os::unix::ffi::OsStrExt;
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use lazy_format::lazy_format;
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use super::{FileSystem, MountType, ReadOnly};
use crate::util::Invoke;
use crate::{Error, ResultExt};
pub async fn mount(
logicalname: impl AsRef<Path>,
offset: u64,
size: u64,
mountpoint: impl AsRef<Path>,
mount_type: MountType,
) -> Result<(), Error> {
tokio::fs::create_dir_all(mountpoint.as_ref()).await?;
let mut opts = format!("loop,offset={offset},sizelimit={size}");
if mount_type == ReadOnly {
opts += ",ro";
}
tokio::process::Command::new("mount")
.arg(logicalname.as_ref())
.arg(mountpoint.as_ref())
.arg("-o")
.arg(opts)
.invoke(crate::ErrorKind::Filesystem)
.await?;
Ok(())
}
use super::FileSystem;
use crate::prelude::*;
#[derive(Debug, Deserialize, Serialize)]
#[serde(rename_all = "kebab-case")]
@@ -50,21 +27,18 @@ impl<LogicalName: AsRef<Path>> LoopDev<LogicalName> {
}
}
}
#[async_trait]
impl<LogicalName: AsRef<Path> + Send + Sync> FileSystem for LoopDev<LogicalName> {
async fn mount<P: AsRef<Path> + Send + Sync>(
&self,
mountpoint: P,
mount_type: MountType,
) -> Result<(), Error> {
mount(
self.logicalname.as_ref(),
self.offset,
self.size,
mountpoint,
mount_type,
)
.await
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
Ok(Some(
tokio::fs::canonicalize(self.logicalname.as_ref()).await?,
))
}
fn mount_options(&self) -> impl IntoIterator<Item = impl Display> {
[
Box::new("loop") as Box<dyn Display>,
Box::new(lazy_format!("offset={}", self.offset)),
Box::new(lazy_format!("sizelimit={}", self.size)),
]
}
async fn source_hash(
&self,

View File

@@ -1,11 +1,15 @@
use std::ffi::OsStr;
use std::fmt::{Display, Write};
use std::path::Path;
use async_trait::async_trait;
use digest::generic_array::GenericArray;
use digest::OutputSizeUser;
use futures::Future;
use sha2::Sha256;
use tokio::process::Command;
use crate::Error;
use crate::prelude::*;
use crate::util::Invoke;
pub mod bind;
pub mod block_dev;
@@ -13,8 +17,10 @@ pub mod cifs;
pub mod ecryptfs;
pub mod efivarfs;
pub mod httpdirfs;
pub mod idmapped;
pub mod label;
pub mod loop_dev;
pub mod overlayfs;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum MountType {
@@ -24,14 +30,78 @@ pub enum MountType {
pub use MountType::*;
#[async_trait]
pub trait FileSystem {
async fn mount<P: AsRef<Path> + Send + Sync>(
pub(self) async fn default_mount_command(
fs: &(impl FileSystem + ?Sized),
mountpoint: impl AsRef<Path> + Send,
mount_type: MountType,
) -> Result<std::process::Command, Error> {
let mut cmd = std::process::Command::new("mount");
if mount_type == ReadOnly {
cmd.arg("-r");
}
cmd.args(fs.extra_args());
if let Some(ty) = fs.mount_type() {
cmd.arg("-t").arg(ty.as_ref());
}
if let Some(options) = fs
.mount_options()
.into_iter()
.fold(None, |acc: Option<String>, x| match acc {
Some(mut s) => {
write!(s, ",{}", x).unwrap();
Some(s)
}
None => Some(x.to_string()),
})
{
cmd.arg("-o").arg(options);
}
if let Some(source) = fs.source().await? {
cmd.arg(source.as_ref());
}
cmd.arg(mountpoint.as_ref());
Ok(dbg!(cmd))
}
pub(self) async fn default_mount_impl(
fs: &(impl FileSystem + ?Sized),
mountpoint: impl AsRef<Path> + Send,
mount_type: MountType,
) -> Result<(), Error> {
fs.pre_mount().await?;
tokio::fs::create_dir_all(mountpoint.as_ref()).await?;
Command::from(default_mount_command(fs, mountpoint, mount_type).await?)
.invoke(ErrorKind::Filesystem)
.await?;
Ok(())
}
pub trait FileSystem: Send + Sync {
fn mount_type(&self) -> Option<impl AsRef<str>> {
None::<&str>
}
fn extra_args(&self) -> impl IntoIterator<Item = impl AsRef<OsStr>> {
[] as [&str; 0]
}
fn mount_options(&self) -> impl IntoIterator<Item = impl Display> {
[] as [&str; 0]
}
fn source(&self) -> impl Future<Output = Result<Option<impl AsRef<Path>>, Error>> + Send {
async { Ok(None::<&Path>) }
}
fn pre_mount(&self) -> impl Future<Output = Result<(), Error>> + Send {
async { Ok(()) }
}
fn mount<P: AsRef<Path> + Send>(
&self,
mountpoint: P,
mount_type: MountType,
) -> Result<(), Error>;
async fn source_hash(
) -> impl Future<Output = Result<(), Error>> + Send {
default_mount_impl(self, mountpoint, mount_type)
}
fn source_hash(
&self,
) -> Result<GenericArray<u8, <Sha256 as OutputSizeUser>::OutputSize>, Error>;
) -> impl Future<Output = Result<GenericArray<u8, <Sha256 as OutputSizeUser>::OutputSize>, Error>>
+ Send;
}

View File

@@ -0,0 +1,153 @@
use std::fmt::Display;
use std::os::unix::ffi::OsStrExt;
use std::path::Path;
use digest::generic_array::GenericArray;
use digest::{Digest, OutputSizeUser};
use sha2::Sha256;
use crate::disk::mount::filesystem::{FileSystem, ReadOnly, ReadWrite};
use crate::disk::mount::guard::{GenericMountGuard, MountGuard, TmpMountGuard};
use crate::prelude::*;
use crate::util::io::TmpDir;
struct OverlayFs<P0: AsRef<Path>, P1: AsRef<Path>> {
lower: P0,
upper: P1,
}
impl<P0: AsRef<Path>, P1: AsRef<Path>> OverlayFs<P0, P1> {
pub fn new(lower: P0, upper: P1) -> Self {
Self { lower, upper }
}
}
impl<P0: AsRef<Path> + Send + Sync, P1: AsRef<Path> + Send + Sync> FileSystem
for OverlayFs<P0, P1>
{
fn mount_type(&self) -> Option<impl AsRef<str>> {
Some("overlay")
}
async fn source(&self) -> Result<Option<impl AsRef<Path>>, Error> {
Ok(Some("overlay"))
}
fn mount_options(&self) -> impl IntoIterator<Item = impl Display> {
[
Box::new(lazy_format!("lowerdir={}", self.lower.as_ref().display()))
as Box<dyn Display>,
Box::new(lazy_format!(
"upperdir={}/upper",
self.upper.as_ref().display()
)),
Box::new(lazy_format!(
"workdir={}/work",
self.upper.as_ref().display()
)),
]
}
async fn pre_mount(&self) -> Result<(), Error> {
tokio::fs::create_dir_all(self.upper.as_ref().join("upper")).await?;
tokio::fs::create_dir_all(self.upper.as_ref().join("work")).await?;
Ok(())
}
async fn source_hash(
&self,
) -> Result<GenericArray<u8, <Sha256 as OutputSizeUser>::OutputSize>, Error> {
let mut sha = Sha256::new();
sha.update("OverlayFs");
sha.update(
tokio::fs::canonicalize(self.lower.as_ref())
.await
.with_ctx(|_| {
(
crate::ErrorKind::Filesystem,
self.lower.as_ref().display().to_string(),
)
})?
.as_os_str()
.as_bytes(),
);
sha.update(
tokio::fs::canonicalize(self.upper.as_ref())
.await
.with_ctx(|_| {
(
crate::ErrorKind::Filesystem,
self.upper.as_ref().display().to_string(),
)
})?
.as_os_str()
.as_bytes(),
);
Ok(sha.finalize())
}
}
#[derive(Debug)]
pub struct OverlayGuard {
lower: Option<TmpMountGuard>,
upper: Option<TmpDir>,
inner_guard: MountGuard,
}
impl OverlayGuard {
pub async fn mount(
base: &impl FileSystem,
mountpoint: impl AsRef<Path>,
) -> Result<Self, Error> {
let lower = TmpMountGuard::mount(base, ReadOnly).await?;
let upper = TmpDir::new().await?;
let inner_guard = MountGuard::mount(
&OverlayFs::new(lower.path(), upper.as_ref()),
mountpoint,
ReadWrite,
)
.await?;
Ok(Self {
lower: Some(lower),
upper: Some(upper),
inner_guard,
})
}
pub async fn unmount(mut self, delete_mountpoint: bool) -> Result<(), Error> {
self.inner_guard.take().unmount(delete_mountpoint).await?;
if let Some(lower) = self.lower.take() {
lower.unmount().await?;
}
if let Some(upper) = self.upper.take() {
upper.delete().await?;
}
Ok(())
}
pub fn take(&mut self) -> Self {
Self {
lower: self.lower.take(),
upper: self.upper.take(),
inner_guard: self.inner_guard.take(),
}
}
}
#[async_trait::async_trait]
impl GenericMountGuard for OverlayGuard {
fn path(&self) -> &Path {
self.inner_guard.path()
}
async fn unmount(mut self) -> Result<(), Error> {
self.unmount(false).await
}
}
impl Drop for OverlayGuard {
fn drop(&mut self) {
let lower = self.lower.take();
let upper = self.upper.take();
let guard = self.inner_guard.take();
if lower.is_some() || upper.is_some() || guard.mounted {
tokio::spawn(async move {
guard.unmount(false).await.unwrap();
if let Some(lower) = lower {
lower.unmount().await.unwrap();
}
if let Some(upper) = upper {
upper.delete().await.unwrap();
}
});
}
}
}

View File

@@ -9,20 +9,47 @@ use tracing::instrument;
use super::filesystem::{FileSystem, MountType, ReadOnly, ReadWrite};
use super::util::unmount;
use crate::util::Invoke;
use crate::util::{Invoke, Never};
use crate::Error;
pub const TMP_MOUNTPOINT: &'static str = "/media/embassy/tmp";
#[async_trait::async_trait]
pub trait GenericMountGuard: AsRef<Path> + std::fmt::Debug + Send + Sync + 'static {
pub trait GenericMountGuard: std::fmt::Debug + Send + Sync + 'static {
fn path(&self) -> &Path;
async fn unmount(mut self) -> Result<(), Error>;
}
#[async_trait::async_trait]
impl GenericMountGuard for Never {
fn path(&self) -> &Path {
match *self {}
}
async fn unmount(mut self) -> Result<(), Error> {
match self {}
}
}
#[async_trait::async_trait]
impl<T> GenericMountGuard for Arc<T>
where
T: GenericMountGuard,
{
fn path(&self) -> &Path {
(&**self).path()
}
async fn unmount(mut self) -> Result<(), Error> {
if let Ok(guard) = Arc::try_unwrap(self) {
guard.unmount().await?;
}
Ok(())
}
}
#[derive(Debug)]
pub struct MountGuard {
mountpoint: PathBuf,
mounted: bool,
pub(super) mounted: bool,
}
impl MountGuard {
pub async fn mount(
@@ -37,6 +64,16 @@ impl MountGuard {
mounted: true,
})
}
fn as_unmounted(&self) -> Self {
Self {
mountpoint: self.mountpoint.clone(),
mounted: false,
}
}
pub fn take(&mut self) -> Self {
let unmounted = self.as_unmounted();
std::mem::replace(self, unmounted)
}
pub async fn unmount(mut self, delete_mountpoint: bool) -> Result<(), Error> {
if self.mounted {
unmount(&self.mountpoint).await?;
@@ -57,11 +94,6 @@ impl MountGuard {
Ok(())
}
}
impl AsRef<Path> for MountGuard {
fn as_ref(&self) -> &Path {
&self.mountpoint
}
}
impl Drop for MountGuard {
fn drop(&mut self) {
if self.mounted {
@@ -72,6 +104,9 @@ impl Drop for MountGuard {
}
#[async_trait::async_trait]
impl GenericMountGuard for MountGuard {
fn path(&self) -> &Path {
&self.mountpoint
}
async fn unmount(mut self) -> Result<(), Error> {
MountGuard::unmount(self, false).await
}
@@ -89,7 +124,7 @@ lazy_static! {
Mutex::new(BTreeMap::new());
}
#[derive(Debug)]
#[derive(Debug, Clone)]
pub struct TmpMountGuard {
guard: Arc<MountGuard>,
}
@@ -122,21 +157,42 @@ impl TmpMountGuard {
Ok(TmpMountGuard { guard })
}
}
pub async fn unmount(self) -> Result<(), Error> {
if let Ok(guard) = Arc::try_unwrap(self.guard) {
guard.unmount(true).await?;
}
Ok(())
}
}
impl AsRef<Path> for TmpMountGuard {
fn as_ref(&self) -> &Path {
(&*self.guard).as_ref()
pub fn take(&mut self) -> Self {
let unmounted = Self {
guard: Arc::new(self.guard.as_unmounted()),
};
std::mem::replace(self, unmounted)
}
}
#[async_trait::async_trait]
impl GenericMountGuard for TmpMountGuard {
fn path(&self) -> &Path {
self.guard.path()
}
async fn unmount(mut self) -> Result<(), Error> {
TmpMountGuard::unmount(self).await
self.guard.unmount().await
}
}
#[derive(Debug)]
pub struct SubPath<G: GenericMountGuard> {
guard: G,
path: PathBuf,
}
impl<G: GenericMountGuard> SubPath<G> {
pub fn new(guard: G, path: impl AsRef<Path>) -> Self {
let path = path.as_ref();
let path = guard.path().join(path.strip_prefix("/").unwrap_or(path));
Self { guard, path }
}
}
#[async_trait::async_trait]
impl<G: GenericMountGuard> GenericMountGuard for SubPath<G> {
fn path(&self) -> &Path {
self.path.as_path()
}
async fn unmount(mut self) -> Result<(), Error> {
self.guard.unmount().await
}
}

View File

@@ -44,7 +44,7 @@ pub async fn bind<P0: AsRef<Path>, P1: AsRef<Path>>(
pub async fn unmount<P: AsRef<Path>>(mountpoint: P) -> Result<(), Error> {
tracing::debug!("Unmounting {}.", mountpoint.as_ref().display());
tokio::process::Command::new("umount")
.arg("-l")
.arg("-Rl")
.arg(mountpoint.as_ref())
.invoke(crate::ErrorKind::Filesystem)
.await?;

View File

@@ -17,6 +17,7 @@ use tracing::instrument;
use super::mount::filesystem::block_dev::BlockDev;
use super::mount::filesystem::ReadOnly;
use super::mount::guard::TmpMountGuard;
use crate::disk::mount::guard::GenericMountGuard;
use crate::disk::OsPartitionInfo;
use crate::util::serde::IoFormat;
use crate::util::{Invoke, Version};
@@ -403,13 +404,13 @@ async fn part_info(part: PathBuf) -> PartitionInfo {
match TmpMountGuard::mount(&BlockDev::new(&part), ReadOnly).await {
Err(e) => tracing::warn!("Could not collect usage information: {}", e.source),
Ok(mount_guard) => {
used = get_used(&mount_guard)
used = get_used(mount_guard.path())
.await
.map_err(|e| {
tracing::warn!("Could not get usage of {}: {}", part.display(), e.source)
})
.ok();
if let Some(recovery_info) = match recovery_info(&mount_guard).await {
if let Some(recovery_info) = match recovery_info(mount_guard.path()).await {
Ok(a) => a,
Err(e) => {
tracing::error!("Error fetching unencrypted backup metadata: {}", e);