Http proxy (#1772)

* adding skeleton for new http_proxy.

* more stuff yay

* more stuff yay

* more stuff

* more stuff

* "working" poc

* more stuff

* more stuff

* fix mored stuff

* working proxy

* moved to handler style for requests

* clean up

* cleaning stuff up

* more stuff

* refactoring code

* more changes

* refactoring handle

* refactored code

* more stuff

* Co-authored-by: J M <Blu-J@users.noreply.github.com>

* Co-authored-by: J M <Blu-J@users.noreply.github.com>

* more stuff

* more stuff

* working main ui handle

* Implement old code to handler in static server

* Feat/long running (#1676)

* feat: Start the long running container

* feat: Long running docker, running, stoping, and uninstalling

* feat: Just make the folders that we would like to mount.

* fix: Uninstall not working

* chore: remove some logging

* feat: Smarter cleanup

* feat: Wait for start

* wip: Need to kill

* chore: Remove the bad tracing

* feat: Stopping the long running processes without killing the long
running

* Mino Feat: Change the Manifest To have a new type (#1736)

* Add build-essential to README.md (#1716)

Update README.md

* write image to sparse-aware archive format (#1709)

* fix: Add modification to the max_user_watches (#1695)

* fix: Add modification to the max_user_watches

* chore: Move to initialization

* [Feat] follow logs (#1714)

* tail logs

* add cli

* add FE

* abstract http to shared

* batch new logs

* file download for logs

* fix modal error when no config

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: BluJ <mogulslayer@gmail.com>

* Update README.md (#1728)

* fix build for patch-db client for consistency (#1722)

* fix cli install (#1720)

* highlight instructions if not viewed (#1731)

* wip:

* [ ] Fix the build (dependencies:634 map for option)

* fix: Cargo build

* fix: Long running wasn't starting

* fix: uninstall works

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* chore: Fix a dbg!

* chore: Make the commands of the docker-inject do inject instead of exec

* chore: Fix compile mistake

* chore: Change to use simpler

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* remove recovered services and drop reordering feature (#1829)

* chore: Convert the migration to use receipt. (#1842)

* feat: remove ionic storage (#1839)

* feat: remove ionic storage

* grayscal when disconncted, rename local storage service for clarity

* remove storage from package lock

* update patchDB

Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>

* update patch DB

* workring http server

* Feat/community marketplace (#1790)

* add community marketplace

* Update embassy-mock-api.service.ts

* expect ui/marketplace to be undefined

* possible undefined from getpackage

* fix marketplace pages

* rework marketplace infrastructure

* fix bugs

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>

* remove unwrap

* cleanup code

* d

* more stuff

* fix: make `shared` module independent of `config.js` (#1870)

* cert stuff WIP

* MORE CERT STUFF

* more stuff

* more stuff

* more stuff

* abstract service fn

* almost ssl

* fix ssl export

* Feat/long running (#1676)

* feat: Start the long running container

* feat: Long running docker, running, stoping, and uninstalling

* feat: Just make the folders that we would like to mount.

* fix: Uninstall not working

* chore: remove some logging

* feat: Smarter cleanup

* feat: Wait for start

* wip: Need to kill

* chore: Remove the bad tracing

* feat: Stopping the long running processes without killing the long
running

* Mino Feat: Change the Manifest To have a new type (#1736)

* Add build-essential to README.md (#1716)

Update README.md

* write image to sparse-aware archive format (#1709)

* fix: Add modification to the max_user_watches (#1695)

* fix: Add modification to the max_user_watches

* chore: Move to initialization

* [Feat] follow logs (#1714)

* tail logs

* add cli

* add FE

* abstract http to shared

* batch new logs

* file download for logs

* fix modal error when no config

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: BluJ <mogulslayer@gmail.com>

* Update README.md (#1728)

* fix build for patch-db client for consistency (#1722)

* fix cli install (#1720)

* highlight instructions if not viewed (#1731)

* wip:

* [ ] Fix the build (dependencies:634 map for option)

* fix: Cargo build

* fix: Long running wasn't starting

* fix: uninstall works

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* chore: Fix a dbg!

* chore: Make the commands of the docker-inject do inject instead of exec

* chore: Fix compile mistake

* chore: Change to use simpler

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* remove recovered services and drop reordering feature (#1829)

* chore: Convert the migration to use receipt. (#1842)

* feat: remove ionic storage (#1839)

* feat: remove ionic storage

* grayscal when disconncted, rename local storage service for clarity

* remove storage from package lock

* update patchDB

Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>

* update patch DB

* Feat/community marketplace (#1790)

* add community marketplace

* Update embassy-mock-api.service.ts

* expect ui/marketplace to be undefined

* possible undefined from getpackage

* fix marketplace pages

* rework marketplace infrastructure

* fix bugs

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>

* fix: make `shared` module independent of `config.js` (#1870)

* fix: small fix for marketplace header styles (#1873)

* feat: setup publishing of share and marketplace packages (#1874)

* inlcude marketplace in linter

* fix npm publish scrips and bump versions of libs

* feat: add assets to published packages and fix peerDeps versions (#1875)

* bump peer dep

* fix: add assets to published paths (#1876)

* allow ca download over lan (#1877)

* only desaturate when logged in and not fully

* Feature/multi platform (#1866)

* wip

* wip

* wip

* wip

* wip

* wip

* remove debian dir

* lazy env and git hash

* remove env and git hash on clean

* don't leave project dir

* use docker for native builds

* start9 rust

* correctly mount registry

* remove systemd config

* switch to /usr/bin

* disable sound for now

* wip

* change disk list

* multi-arch images

* multi-arch system images

* default aarch64

* edition 2021

* dynamic wifi interface name

* use wifi interface from config

* bugfixes

* add beep based sound

* wip

* wip

* wip

* separate out raspberry pi specific files

* fixes

* use new initramfs always

* switch journald conf to sed script

* fixes

* fix permissions

* talking about kernel modules not scripts

* fix

* fix

* switch to MBR

* install to /usr/lib

* fixes

* fixes

* fixes

* fixes

* add media config to cfg path

* fixes

* fixes

* fixes

* raspi image fixes

* fix test

* fix workflow

* sync boot partition

* gahhhhh

* more stuff

* remove restore warning and better messaging for backup/restore (#1881)

* Update READMEs (#1885)

* docs

* fix host key generation

* debugging eos with tokio console

* fix recursive default

* build improvements (#1886)

* build improvements

* no workdir

* kiosk fully working

* setup profile prefs

* Feat/js long running (#1879)

* wip: combining the streams

* chore: Testing locally

* chore: Fix some lint

* Feat/long running (#1676)

* feat: Start the long running container

* feat: Long running docker, running, stoping, and uninstalling

* feat: Just make the folders that we would like to mount.

* fix: Uninstall not working

* chore: remove some logging

* feat: Smarter cleanup

* feat: Wait for start

* wip: Need to kill

* chore: Remove the bad tracing

* feat: Stopping the long running processes without killing the long
running

* Mino Feat: Change the Manifest To have a new type (#1736)

* Add build-essential to README.md (#1716)

Update README.md

* write image to sparse-aware archive format (#1709)

* fix: Add modification to the max_user_watches (#1695)

* fix: Add modification to the max_user_watches

* chore: Move to initialization

* [Feat] follow logs (#1714)

* tail logs

* add cli

* add FE

* abstract http to shared

* batch new logs

* file download for logs

* fix modal error when no config

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: BluJ <mogulslayer@gmail.com>

* Update README.md (#1728)

* fix build for patch-db client for consistency (#1722)

* fix cli install (#1720)

* highlight instructions if not viewed (#1731)

* wip:

* [ ] Fix the build (dependencies:634 map for option)

* fix: Cargo build

* fix: Long running wasn't starting

* fix: uninstall works

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* chore: Fix a dbg!

* chore: Make the commands of the docker-inject do inject instead of exec

* chore: Fix compile mistake

* chore: Change to use simpler

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* wip: making the mananger create

* wip: Working on trying to make the long running docker container command

* Feat/long running (#1676)

* feat: Start the long running container

* feat: Long running docker, running, stoping, and uninstalling

* feat: Just make the folders that we would like to mount.

* fix: Uninstall not working

* chore: remove some logging

* feat: Smarter cleanup

* feat: Wait for start

* wip: Need to kill

* chore: Remove the bad tracing

* feat: Stopping the long running processes without killing the long
running

* Mino Feat: Change the Manifest To have a new type (#1736)

* Add build-essential to README.md (#1716)

Update README.md

* write image to sparse-aware archive format (#1709)

* fix: Add modification to the max_user_watches (#1695)

* fix: Add modification to the max_user_watches

* chore: Move to initialization

* [Feat] follow logs (#1714)

* tail logs

* add cli

* add FE

* abstract http to shared

* batch new logs

* file download for logs

* fix modal error when no config

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: BluJ <mogulslayer@gmail.com>

* Update README.md (#1728)

* fix build for patch-db client for consistency (#1722)

* fix cli install (#1720)

* highlight instructions if not viewed (#1731)

* wip:

* [ ] Fix the build (dependencies:634 map for option)

* fix: Cargo build

* fix: Long running wasn't starting

* fix: uninstall works

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* chore: Fix a dbg!

* chore: Make the commands of the docker-inject do inject instead of exec

* chore: Fix compile mistake

* chore: Change to use simpler

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* feat: Use the long running feature in the manager

* remove recovered services and drop reordering feature (#1829)

* wip: Need to get the initial docker command running?

* chore: Add in the new procedure for the docker.

* feat: Get the system to finally run long

* wip: Added the command inserter to the docker persistance

* wip: Added the command inserter to the docker persistance

* Feat/long running (#1676)

* feat: Start the long running container

* feat: Long running docker, running, stoping, and uninstalling

* feat: Just make the folders that we would like to mount.

* fix: Uninstall not working

* chore: remove some logging

* feat: Smarter cleanup

* feat: Wait for start

* wip: Need to kill

* chore: Remove the bad tracing

* feat: Stopping the long running processes without killing the long
running

* Mino Feat: Change the Manifest To have a new type (#1736)

* Add build-essential to README.md (#1716)

Update README.md

* write image to sparse-aware archive format (#1709)

* fix: Add modification to the max_user_watches (#1695)

* fix: Add modification to the max_user_watches

* chore: Move to initialization

* [Feat] follow logs (#1714)

* tail logs

* add cli

* add FE

* abstract http to shared

* batch new logs

* file download for logs

* fix modal error when no config

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: BluJ <mogulslayer@gmail.com>

* Update README.md (#1728)

* fix build for patch-db client for consistency (#1722)

* fix cli install (#1720)

* highlight instructions if not viewed (#1731)

* wip:

* [ ] Fix the build (dependencies:634 map for option)

* fix: Cargo build

* fix: Long running wasn't starting

* fix: uninstall works

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* chore: Fix a dbg!

* chore: Make the commands of the docker-inject do inject instead of exec

* chore: Fix compile mistake

* chore: Change to use simpler

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* remove recovered services and drop reordering feature (#1829)

* chore: Convert the migration to use receipt. (#1842)

* feat: remove ionic storage (#1839)

* feat: remove ionic storage

* grayscal when disconncted, rename local storage service for clarity

* remove storage from package lock

* update patchDB

Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>

* update patchDB

* feat: Move the run_command into the js

* Feat/long running (#1676)

* feat: Start the long running container

* feat: Long running docker, running, stoping, and uninstalling

* feat: Just make the folders that we would like to mount.

* fix: Uninstall not working

* chore: remove some logging

* feat: Smarter cleanup

* feat: Wait for start

* wip: Need to kill

* chore: Remove the bad tracing

* feat: Stopping the long running processes without killing the long
running

* Mino Feat: Change the Manifest To have a new type (#1736)

* Add build-essential to README.md (#1716)

Update README.md

* write image to sparse-aware archive format (#1709)

* fix: Add modification to the max_user_watches (#1695)

* fix: Add modification to the max_user_watches

* chore: Move to initialization

* [Feat] follow logs (#1714)

* tail logs

* add cli

* add FE

* abstract http to shared

* batch new logs

* file download for logs

* fix modal error when no config

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: BluJ <mogulslayer@gmail.com>

* Update README.md (#1728)

* fix build for patch-db client for consistency (#1722)

* fix cli install (#1720)

* highlight instructions if not viewed (#1731)

* wip:

* [ ] Fix the build (dependencies:634 map for option)

* fix: Cargo build

* fix: Long running wasn't starting

* fix: uninstall works

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* chore: Fix a dbg!

* chore: Make the commands of the docker-inject do inject instead of exec

* chore: Fix compile mistake

* chore: Change to use simpler

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>

* remove recovered services and drop reordering feature (#1829)

* chore: Convert the migration to use receipt. (#1842)

* feat: remove ionic storage (#1839)

* feat: remove ionic storage

* grayscal when disconncted, rename local storage service for clarity

* remove storage from package lock

* update patchDB

Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>

* update patch DB

* chore: Change the error catching for the long running to try all

* Feat/community marketplace (#1790)

* add community marketplace

* Update embassy-mock-api.service.ts

* expect ui/marketplace to be undefined

* possible undefined from getpackage

* fix marketplace pages

* rework marketplace infrastructure

* fix bugs

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>

* WIP: Fix the build, needed to move around creation of exec

* wip: Working on solving why there is a missing end.

* fix: make `shared` module independent of `config.js` (#1870)

* feat: Add in the kill and timeout

* feat: Get the run to actually work.

* chore: Add when/ why/ where comments

* feat: Convert inject main to use exec main.

* Fix: Ability to stop services

* wip: long running js main

* feat: Kill for the main

* Fix

* fix: Fix the build for x86

* wip: Working on changes

* wip: Working on trying to kill js

* fix: Testing for slow

* feat: Test that the new manifest works

* chore: Try and fix build?

* chore: Fix? the build

* chore: Fix the long input dies and never restarts

* build improvements

* no workdir

* fix: Architecture for long running

* chore: Fix and remove the docker inject

* chore: Undo the changes to the kiosk mode

* fix: Remove the it from the prod build

* fix: Start issue

* fix: The compat build

* chore: Add in the conditional compilation again for the missing impl

* chore: Change to aux

* chore: Remove the aux for now

* chore: Add some documentation to docker container

Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>
Co-authored-by: Alex Inkin <alexander@inkin.ru>

* use old resolv.conf until systemd is on

* update patchdb

* update patch db submodule

* no x11 wrapper config

* working poc

* fixing misc stuff

* switch patchdb to next

* Feat/update tab (#1865)

* implement updates tab for viewing all updates from all marketplaces in one place

* remove auto-check-updates

* feat: implement updates page (#1888)

* feat: implement updates page

* chore: comments

* better styling in update tab

* rework marketplace service (#1891)

* rework marketplace service

* remove unneeded ?

* fix: refactor marketplace to cache requests

Co-authored-by: waterplea <alexander@inkin.ru>

Co-authored-by: Alex Inkin <alexander@inkin.ru>

* misc fixes (#1894)

* changing hostname stuff

* changes

* move marketplace settings into marketplace tab (#1895)

* move marketplace settings into marketplace tab

* Update frontend/projects/ui/src/app/modals/marketplace-settings/marketplace-settings.page.ts

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>

* bump marketplace version

* removing oldd code

* working service proxy

* fqdn struct wwip

* new types for ssl proxy

* restructure restore.rs and embassyd.rs

* adding dbg

* debugging proxy handlers

* add lots of debugging for the svc handler removal bug

* debugging

* remove extra code

* fixing proxy and removing old debug code

* finalizing proxy code to serve the setup ui and diag ui

* final new eos http proxy

* remove uneeded trace error

* remove extra file

* not needed flags

* clean up

* Fix/debug (#1909)

chore: Use debug by default"

* chore: Fix on the rsync not having stdout. (#1911)

* install wizard project (#1893)

* install wizard project

* reboot endpoint

* Update frontend/projects/install-wizard/src/app/pages/home/home.page.ts

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>

* Update frontend/projects/install-wizard/src/app/pages/home/home.page.ts

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>

* Update frontend/projects/install-wizard/src/app/pages/home/home.page.ts

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>

* update build

* fix build

* backend portion

* increase image size

* loaded

* dont auto resize

* fix install wizard

* use localhost if still in setup mode

* fix compat

Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>

* fix kiosk

* integrate install wizard

* fix build typo

* no nginx

* fix build

* remove nginx stuff from build

* fixes

Co-authored-by: Stephen Chavez <stephen@start9labs.com>
Co-authored-by: J M <2364004+Blu-J@users.noreply.github.com>
Co-authored-by: Chris Guida <chrisguida@users.noreply.github.com>
Co-authored-by: Aiden McClelland <3732071+dr-bonez@users.noreply.github.com>
Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <matthewonthemoon@gmail.com>
Co-authored-by: Lucy C <12953208+elvece@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>
Co-authored-by: Alex Inkin <alexander@inkin.ru>
This commit is contained in:
Stephen Chavez
2022-11-09 05:30:26 +00:00
committed by Aiden McClelland
parent d215d96b9b
commit 0fc546962e
38 changed files with 2130 additions and 1410 deletions

View File

@@ -0,0 +1,384 @@
use std::collections::BTreeMap;
use std::net::{Ipv4Addr, SocketAddr};
use std::str::FromStr;
use std::sync::Arc;
use color_eyre::eyre::eyre;
use futures::FutureExt;
use http::{Method, Request, Response};
use hyper::upgrade::Upgraded;
use hyper::{Body, Error as HyperError};
use models::{InterfaceId, PackageId};
use openssl::pkey::{PKey, Private};
use openssl::x509::X509;
use tokio::net::TcpStream;
use tokio::sync::Mutex;
use tracing::{error, info, instrument};
use crate::net::net_utils::{host_addr_fqdn, ResourceFqdn};
use crate::net::ssl::SslManager;
use crate::net::vhost_controller::VHOSTController;
use crate::net::{HttpClient, HttpHandler, InterfaceMetadata, PackageNetInfo};
use crate::{Error, ResultExt};
pub struct ProxyController {
inner: Mutex<ProxyControllerInner>,
}
impl ProxyController {
pub async fn init(
embassyd_socket_addr: SocketAddr,
embassy_fqdn: ResourceFqdn,
ssl_manager: SslManager,
) -> Result<Self, Error> {
Ok(ProxyController {
inner: Mutex::new(
ProxyControllerInner::init(embassyd_socket_addr, embassy_fqdn, ssl_manager).await?,
),
})
}
pub async fn add_docker_service<I: IntoIterator<Item = (InterfaceId, InterfaceMetadata)>>(
&self,
package: PackageId,
ipv4: Ipv4Addr,
interfaces: I,
) -> Result<(), Error> {
self.inner
.lock()
.await
.add_docker_service(package, ipv4, interfaces)
.await
}
pub async fn remove_docker_service(&self, package: &PackageId) -> Result<(), Error> {
self.inner.lock().await.remove_docker_service(package).await
}
pub async fn add_certificate_to_resolver(
&self,
fqdn: ResourceFqdn,
cert_data: (PKey<Private>, Vec<X509>),
) -> Result<(), Error> {
self.inner
.lock()
.await
.add_certificate_to_resolver(fqdn, cert_data)
.await
}
pub async fn add_handle(
&self,
ext_port: u16,
fqdn: ResourceFqdn,
handler: HttpHandler,
is_ssl: bool,
) -> Result<(), Error> {
self.inner
.lock()
.await
.add_handle(ext_port, fqdn, handler, is_ssl)
.await
}
pub async fn get_hostname(&self) -> String {
self.inner.lock().await.get_embassy_hostname()
}
pub async fn proxy(
client: HttpClient,
req: Request<Body>,
) -> Result<Response<Body>, HyperError> {
if Method::CONNECT == req.method() {
// Received an HTTP request like:
// ```
// CONNECT www.domain.com:443 HTTP/1.1s
// Host: www.domain.com:443
// Proxy-Connection: Keep-Alive
// ```
//
// When HTTP method is CONNECT we should return an empty body
// then we can eventually upgrade the connection and talk a new protocol.
//
// Note: only after client received an empty body with STATUS_OK can the
// connection be upgraded, so we can't return a response inside
// `on_upgrade` future.
match host_addr_fqdn(&req) {
Ok(host) => {
tokio::task::spawn(async move {
match hyper::upgrade::on(req).await {
Ok(upgraded) => match host {
ResourceFqdn::IpAddr(ip) => {
if let Err(e) = Self::tunnel(upgraded, ip.to_string()).await {
error!("server io error: {}", e);
};
}
ResourceFqdn::Uri {
full_uri,
root: _,
tld: _,
} => {
if let Err(e) =
Self::tunnel(upgraded, full_uri.to_string()).await
{
error!("server io error: {}", e);
};
}
},
Err(e) => error!("upgrade error: {}", e),
}
});
Ok(Response::new(Body::empty()))
}
Err(e) => {
let err_txt = format!("CONNECT host is not socket addr: {:?}", &req.uri());
let mut resp = Response::new(Body::from(format!(
"CONNECT must be to a socket address: {}: {}",
err_txt, e
)));
*resp.status_mut() = http::StatusCode::BAD_REQUEST;
Ok(resp)
}
}
} else {
client.request(req).await
}
}
// Create a TCP connection to host:port, build a tunnel between the connection and
// the upgraded connection
async fn tunnel(mut upgraded: Upgraded, addr: String) -> std::io::Result<()> {
let mut server = TcpStream::connect(addr).await?;
let (from_client, from_server) =
tokio::io::copy_bidirectional(&mut upgraded, &mut server).await?;
info!(
"client wrote {} bytes and received {} bytes",
from_client, from_server
);
Ok(())
}
}
struct ProxyControllerInner {
ssl_manager: SslManager,
vhosts: VHOSTController,
embassyd_fqdn: ResourceFqdn,
docker_interfaces: BTreeMap<PackageId, PackageNetInfo>,
docker_iface_lookups: BTreeMap<(PackageId, InterfaceId), ResourceFqdn>,
}
impl ProxyControllerInner {
#[instrument]
async fn init(
embassyd_socket_addr: SocketAddr,
embassyd_fqdn: ResourceFqdn,
ssl_manager: SslManager,
) -> Result<Self, Error> {
let inner = ProxyControllerInner {
vhosts: VHOSTController::init(embassyd_socket_addr),
ssl_manager,
embassyd_fqdn,
docker_interfaces: BTreeMap::new(),
docker_iface_lookups: BTreeMap::new(),
};
Ok(inner)
}
async fn add_certificate_to_resolver(
&mut self,
hostname: ResourceFqdn,
cert_data: (PKey<Private>, Vec<X509>),
) -> Result<(), Error> {
self.vhosts
.cert_resolver
.add_certificate_to_resolver(hostname, cert_data)
.await
.map_err(|err| {
Error::new(
eyre!("Unable to add ssl cert to the resolver: {}", err),
crate::ErrorKind::Network,
)
})?;
Ok(())
}
async fn add_package_certificate_to_resolver(
&mut self,
resource_fqdn: ResourceFqdn,
pkg_id: PackageId,
) -> Result<(), Error> {
let package_cert = match resource_fqdn.clone() {
ResourceFqdn::IpAddr(ip) => {
self.ssl_manager
.certificate_for(&ip.to_string(), &pkg_id)
.await?
}
ResourceFqdn::Uri {
full_uri: _,
root,
tld: _,
} => self.ssl_manager.certificate_for(&root, &pkg_id).await?,
};
self.vhosts
.cert_resolver
.add_certificate_to_resolver(resource_fqdn, package_cert)
.await
.map_err(|err| {
Error::new(
eyre!("Unable to add ssl cert to the resolver: {}", err),
crate::ErrorKind::Network,
)
})?;
Ok(())
}
pub async fn add_handle(
&mut self,
external_svc_port: u16,
fqdn: ResourceFqdn,
svc_handler: HttpHandler,
is_ssl: bool,
) -> Result<(), Error> {
self.vhosts
.add_server_or_handle(external_svc_port, fqdn, svc_handler, is_ssl)
.await
}
#[instrument(skip(self, interfaces))]
pub async fn add_docker_service<I: IntoIterator<Item = (InterfaceId, InterfaceMetadata)>>(
&mut self,
package: PackageId,
docker_ipv4: Ipv4Addr,
interfaces: I,
) -> Result<(), Error> {
let mut interface_map = interfaces
.into_iter()
.filter(|(_, meta)| {
// don't add stuff for anything we can't connect to over some flavor of http
(meta.protocols.contains("http") || meta.protocols.contains("https"))
// also don't add anything unless it has at least one exposed port
&& !meta.lan_config.is_empty()
})
.collect::<BTreeMap<InterfaceId, InterfaceMetadata>>();
for (id, meta) in interface_map.iter() {
for (external_svc_port, lan_port_config) in meta.lan_config.iter() {
let full_fqdn = ResourceFqdn::from_str(&meta.fqdn).unwrap();
self.docker_iface_lookups
.insert((package.clone(), id.clone()), full_fqdn.clone());
self.add_package_certificate_to_resolver(full_fqdn.clone(), package.clone())
.await?;
let svc_handler =
Self::create_docker_handle(docker_ipv4.to_string(), lan_port_config.internal)
.await;
self.add_handle(
external_svc_port.0,
full_fqdn.clone(),
svc_handler,
lan_port_config.ssl,
)
.await?;
}
}
let docker_interface = self.docker_interfaces.entry(package.clone()).or_default();
docker_interface.interfaces.append(&mut interface_map);
Ok(())
}
async fn create_docker_handle(internal_ip: String, port: u16) -> HttpHandler {
let svc_handler: HttpHandler = Arc::new(move |mut req| {
let proxy_addr = internal_ip.clone();
async move {
let client = HttpClient::new();
let uri_string = format!(
"http://{}:{}{}",
proxy_addr,
port,
req.uri()
.path_and_query()
.map(|x| x.as_str())
.unwrap_or("/")
);
let uri = uri_string.parse().unwrap();
*req.uri_mut() = uri;
ProxyController::proxy(client, req).await
}
.boxed()
});
svc_handler
}
#[instrument(skip(self))]
pub async fn remove_docker_service(&mut self, package: &PackageId) -> Result<(), Error> {
let mut server_removals: Vec<(u16, InterfaceId)> = Default::default();
let net_info = match self.docker_interfaces.get(package) {
Some(a) => a,
None => return Ok(()),
};
for (id, meta) in &net_info.interfaces {
for (service_ext_port, _lan_port_config) in meta.lan_config.iter() {
if let Some(server) = self.vhosts.service_servers.get_mut(&service_ext_port.0) {
if let Some(fqdn) = self
.docker_iface_lookups
.get(&(package.clone(), id.clone()))
{
server.remove_svc_handler_mapping(fqdn.to_owned()).await?;
self.vhosts
.cert_resolver
.remove_cert(fqdn.to_owned())
.await?;
let mapping = server.svc_mapping.read().await;
if mapping.is_empty() {
server_removals.push((service_ext_port.0, id.to_owned()));
}
}
}
}
}
for (port, interface_id) in server_removals {
if let Some(removed_server) = self.vhosts.service_servers.remove(&port) {
removed_server.shutdown.send(()).map_err(|_| {
Error::new(
eyre!("Hyper server did not quit properly"),
crate::ErrorKind::JoinError,
)
})?;
removed_server
.handle
.await
.with_kind(crate::ErrorKind::JoinError)?;
self.docker_interfaces.remove(&package.clone());
self.docker_iface_lookups
.remove(&(package.clone(), interface_id));
}
}
Ok(())
}
pub fn get_embassy_hostname(&self) -> String {
self.embassyd_fqdn.to_string()
}
}