Refactor/patch db (#2415)

* the only way to begin is by beginning

* chore: Convert over 3444 migration

* fix imports

* wip

* feat: convert volume

* convert: system.rs

* wip(convert): Setup

* wip properties

* wip notifications

* wip

* wip migration

* wip init

* wip auth/control

* wip action

* wip control

* wiip 034

* wip 344

* wip some more versions converted

* feat: Reserialize the version of the db

* wip rest of the versions

* wip s9pk/manifest

* wip wifi

* chore: net/keys

* chore: net/dns

* wip net/dhcp

* wip manager manager-map

* gut dependency errors

* wip update/mod

* detect breakages locally for updates

* wip: manager/mod

* wip: manager/health

* wip: backup/target/mod

* fix: Typo addresses

* clean control.rs

* fix system package id

* switch to btreemap for now

* config wip

* wip manager/mod

* install wip

Co-authored-by: J H <Blu-J@users.noreply.github.com>

* chore: Update the last of the errors

* feat: Change the prelude de to borrow

* feat: Adding in some more things

* chore: add to the prelude

* chore: Small fixes

* chore: Fixing the small errors

* wip: Cleaning up check errors

* wip: Fix some of the issues

* chore: Fix setup

* chore:fix version

* chore: prelude, mod, http_reader

* wip backup_bulk

* chore: Last of the errors

* upadte package.json

* chore: changes needed for a build

* chore: Removing some of the linting errors in the manager

* chore: Some linting 101

* fix: Wrong order of who owns what

* chore: Remove the unstable

* chore: Remove the test in the todo

* @dr-bonez did a refactoring on the backup

* chore: Make sure that there can only be one override guard at a time

* resolve most todos

* wip: Add some more tracing to debug an error

* wip: Use a mv instead of rename

* wip: Revert some of the missing code segments found earlier

* chore: Make the build

* chore: Something about the lib looks like it iis broken

* wip: More instrument and dev working

* kill netdummy before creating it

* better db analysis tools

* fixes from testing

* fix: Make add start the service

* fix status after install

* make wormhole

* fix missing icon file

* fix data url for icons

* fix: Bad deser

* bugfixes

* fix: Backup

* fix: Some of the restor

* fix: Restoring works

* update frontend patch-db types

* hack it in (#2424)

* hack it in

* optimize

* slightly cleaner

* handle config pointers

* dependency config errs

* fix compat

* cache docker

* fix dependency expectation

* fix dependency auto-config

---------

Co-authored-by: Aiden McClelland <me@drbonez.dev>
Co-authored-by: Matt Hill <mattnine@protonmail.com>
Co-authored-by: J H <Blu-J@users.noreply.github.com>
Co-authored-by: Matt Hill <MattDHill@users.noreply.github.com>
This commit is contained in:
J H
2023-09-27 15:46:48 -06:00
committed by GitHub
parent c305deab52
commit 9a202cc124
132 changed files with 7641 additions and 20541 deletions

325
backend/Cargo.lock generated
View File

@@ -58,6 +58,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2c99f64d1e06488f620f932677e24bc6e2897582980441ae90a671415bd7ec2f"
dependencies = [
"cfg-if 1.0.0",
"getrandom 0.2.10",
"once_cell",
"version_check",
]
@@ -374,15 +375,6 @@ version = "2.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "630be753d4e58660abd17930c71b647fe46c27ea6b63cc59e1e3851406972e42"
[[package]]
name = "bitmaps"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "031043d04099746d8db04daf1fa424b2bc8bd69d92b25962dcde24da39ab64a2"
dependencies = [
"typenum",
]
[[package]]
name = "bitmaps"
version = "3.2.0"
@@ -437,44 +429,6 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d696c370c750c948ada61c69a0ee2cbbb9c50b1019ddb86d9317157a99c2cae"
[[package]]
name = "bollard"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d82e7850583ead5f8bbef247e2a3c37a19bd576e8420cd262a6711921827e1e5"
dependencies = [
"base64 0.13.1",
"bollard-stubs",
"bytes",
"futures-core",
"futures-util",
"hex",
"http",
"hyper",
"hyperlocal",
"log",
"pin-project-lite",
"serde",
"serde_derive",
"serde_json",
"serde_urlencoded",
"thiserror",
"tokio",
"tokio-util",
"url",
"winapi",
]
[[package]]
name = "bollard-stubs"
version = "1.42.0-rc.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed59b5c00048f48d7af971b71f800fdf23e858844a6f9e4d32ca72e9399e7864"
dependencies = [
"serde",
"serde_with 1.14.0",
]
[[package]]
name = "brotli"
version = "3.3.4"
@@ -664,7 +618,7 @@ dependencies = [
"indenter",
"once_cell",
"owo-colors",
"tracing-error 0.2.0",
"tracing-error",
]
[[package]]
@@ -676,7 +630,7 @@ dependencies = [
"once_cell",
"owo-colors",
"tracing-core",
"tracing-error 0.2.0",
"tracing-error",
]
[[package]]
@@ -912,38 +866,14 @@ dependencies = [
"zeroize",
]
[[package]]
name = "darling"
version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a01d95850c592940db9b8194bc39f4bc0e89dee5c4265e4b1807c34a9aba453c"
dependencies = [
"darling_core 0.13.4",
"darling_macro 0.13.4",
]
[[package]]
name = "darling"
version = "0.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0209d94da627ab5605dcccf08bb18afa5009cfbef48d8a8b7d7bdbc79be25c5e"
dependencies = [
"darling_core 0.20.3",
"darling_macro 0.20.3",
]
[[package]]
name = "darling_core"
version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "859d65a907b6852c9361e3185c862aae7fafd2887876799fa55f5f99dc40d610"
dependencies = [
"fnv",
"ident_case",
"proc-macro2 1.0.66",
"quote 1.0.31",
"strsim 0.10.0",
"syn 1.0.109",
"darling_core",
"darling_macro",
]
[[package]]
@@ -960,24 +890,13 @@ dependencies = [
"syn 2.0.18",
]
[[package]]
name = "darling_macro"
version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c972679f83bdf9c42bd905396b6c3588a843a17f0f16dfcfa3e2c5d57441835"
dependencies = [
"darling_core 0.13.4",
"quote 1.0.31",
"syn 1.0.109",
]
[[package]]
name = "darling_macro"
version = "0.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "836a9bbc7ad63342d6d6e7b815ccab164bc77a2d95d84bc3117a8c0d5c98e2d5"
dependencies = [
"darling_core 0.20.3",
"darling_core",
"quote 1.0.31",
"syn 2.0.18",
]
@@ -1339,7 +1258,7 @@ dependencies = [
"color-eyre",
"futures",
"helpers",
"imbl 2.0.0",
"imbl",
"nix 0.25.1",
"procfs",
"serde",
@@ -1347,9 +1266,9 @@ dependencies = [
"tokio",
"tokio-stream",
"tracing",
"tracing-error 0.2.0",
"tracing-error",
"tracing-futures",
"tracing-subscriber 0.3.17",
"tracing-subscriber",
"yajrc 0.1.0 (git+https://github.com/dr-bonez/yajrc.git?branch=develop)",
]
@@ -1400,7 +1319,7 @@ version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c9720bba047d567ffc8a3cba48bf19126600e249ab7f128e9233e6376976a116"
dependencies = [
"heck 0.4.1",
"heck",
"proc-macro2 1.0.66",
"quote 1.0.31",
"syn 1.0.109",
@@ -1821,6 +1740,15 @@ dependencies = [
"ahash 0.7.6",
]
[[package]]
name = "hashbrown"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43a3c133739dddd0d2990f9a4bdf8eb4b21ef50e4851ca85ab661199821d510e"
dependencies = [
"ahash 0.8.3",
]
[[package]]
name = "hashbrown"
version = "0.14.0"
@@ -1840,15 +1768,6 @@ dependencies = [
"hashbrown 0.14.0",
]
[[package]]
name = "heck"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d621efb26863f0e9924c6ac577e8275e5e6b77455db64ffa6c65c904e9e132c"
dependencies = [
"unicode-segmentation",
]
[[package]]
name = "heck"
version = "0.4.1"
@@ -2033,19 +1952,6 @@ dependencies = [
"tokio-tungstenite",
]
[[package]]
name = "hyperlocal"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fafdf7b2b2de7c9784f76e02c0935e65a8117ec3b768644379983ab333ac98c"
dependencies = [
"futures-util",
"hex",
"hyper",
"pin-project",
"tokio",
]
[[package]]
name = "iana-time-zone"
version = "0.1.57"
@@ -2112,30 +2018,17 @@ version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb56e1aa765b4b4f3aadfab769793b7087bb03a4ea4920644a6d238e2df5b9ed"
[[package]]
name = "imbl"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "543682c9082b25e63d03b5acbd65ad111fd49dd93e70843e5175db4ff81d606b"
dependencies = [
"bitmaps 2.1.0",
"rand_core 0.6.4",
"rand_xoshiro",
"sized-chunks",
"typenum",
"version_check",
]
[[package]]
name = "imbl"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2806b69cd9f4664844027b64465eacb444c67c1db9c778e341adff0c25cdb0d"
dependencies = [
"bitmaps 3.2.0",
"bitmaps",
"imbl-sized-chunks",
"rand_core 0.6.4",
"rand_xoshiro",
"serde",
"version_check",
]
@@ -2145,7 +2038,19 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6957ea0b2541c5ca561d3ef4538044af79f8a05a1eb3a3b148936aaceaa1076"
dependencies = [
"bitmaps 3.2.0",
"bitmaps",
]
[[package]]
name = "imbl-value"
version = "0.1.0"
source = "git+https://github.com/Start9Labs/imbl-value.git#929395141c3a882ac366c12ac9402d0ebaa2201b"
dependencies = [
"imbl",
"serde",
"serde_json",
"treediff",
"yasi",
]
[[package]]
@@ -2203,20 +2108,6 @@ dependencies = [
"cfg-if 1.0.0",
]
[[package]]
name = "internment"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "161079c3ad892faa215fcfcf3fd7a6a3c9288df2b06a2c2bad7fbfad4f01d69d"
dependencies = [
"ahash 0.7.6",
"dashmap",
"hashbrown 0.12.3",
"once_cell",
"parking_lot 0.12.1",
"serde",
]
[[package]]
name = "io-lifetimes"
version = "1.0.11"
@@ -2395,9 +2286,9 @@ dependencies = [
name = "json-patch"
version = "0.2.7-alpha.0"
dependencies = [
"imbl-value",
"json-ptr",
"serde",
"serde_json",
"treediff",
]
@@ -2405,17 +2296,18 @@ dependencies = [
name = "json-ptr"
version = "0.1.0"
dependencies = [
"imbl",
"imbl-value",
"serde",
"serde_json",
"thiserror",
]
[[package]]
name = "jsonpath_lib"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eaa63191d68230cccb81c5aa23abd53ed64d83337cacbb25a7b8c7979523774f"
source = "git+https://github.com/Start9Labs/jsonpath.git#1cacbd64afa2e1941a21fef06bad14317ba92f30"
dependencies = [
"imbl-value",
"log",
"serde",
"serde_json",
@@ -2701,6 +2593,15 @@ dependencies = [
"autocfg",
]
[[package]]
name = "memoffset"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5de893c32cde5f383baa4c04c5d6dbdd735cfd4a794b0debdb2bb1b421da5ff4"
dependencies = [
"autocfg",
]
[[package]]
name = "mime"
version = "0.3.17"
@@ -2737,11 +2638,10 @@ dependencies = [
name = "models"
version = "0.1.0"
dependencies = [
"bollard",
"base64 0.21.2",
"color-eyre",
"ed25519-dalek",
"emver",
"internment",
"ipnet",
"lazy_static",
"mbrman",
@@ -2749,6 +2649,7 @@ dependencies = [
"patch-db",
"rand 0.8.5",
"regex",
"reqwest",
"rpc-toolkit",
"serde",
"serde_json",
@@ -2758,6 +2659,7 @@ dependencies = [
"tokio",
"torut",
"tracing",
"yasi",
]
[[package]]
@@ -2803,19 +2705,6 @@ dependencies = [
"smallvec",
]
[[package]]
name = "nix"
version = "0.23.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f3790c00a0150112de0f4cd161e3d7fc4b2d8a5542ffc35f099a2562aecb35c"
dependencies = [
"bitflags 1.3.2",
"cc",
"cfg-if 1.0.0",
"libc",
"memoffset",
]
[[package]]
name = "nix"
version = "0.24.3"
@@ -2825,7 +2714,7 @@ dependencies = [
"bitflags 1.3.2",
"cfg-if 1.0.0",
"libc",
"memoffset",
"memoffset 0.6.5",
]
[[package]]
@@ -2838,10 +2727,24 @@ dependencies = [
"bitflags 1.3.2",
"cfg-if 1.0.0",
"libc",
"memoffset",
"memoffset 0.6.5",
"pin-utils",
]
[[package]]
name = "nix"
version = "0.26.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bfdda3d196821d6af13126e40375cdf7da646a96114af134d5f417a9a1dc8e1a"
dependencies = [
"bitflags 1.3.2",
"cfg-if 1.0.0",
"libc",
"memoffset 0.7.1",
"pin-utils",
"static_assertions",
]
[[package]]
name = "nom"
version = "5.1.3"
@@ -3211,19 +3114,19 @@ dependencies = [
"async-trait",
"fd-lock-rs",
"futures",
"imbl 1.0.1",
"imbl",
"imbl-value",
"json-patch",
"json-ptr",
"lazy_static",
"nix 0.23.2",
"nix 0.26.2",
"patch-db-macro",
"serde",
"serde_cbor 0.11.1",
"serde_json",
"thiserror",
"tokio",
"tracing",
"tracing-error 0.1.2",
"tracing-error",
]
[[package]]
@@ -3239,7 +3142,7 @@ dependencies = [
name = "patch-db-macro-internals"
version = "0.1.0"
dependencies = [
"heck 0.3.3",
"heck",
"proc-macro2 1.0.66",
"quote 1.0.31",
"syn 1.0.109",
@@ -4237,16 +4140,6 @@ dependencies = [
"v8",
]
[[package]]
name = "serde_with"
version = "1.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "678b5a069e50bf00ecd22d0cd8ddf7c236f68581b03db652061ed5eb13a312ff"
dependencies = [
"serde",
"serde_with_macros 1.5.2",
]
[[package]]
name = "serde_with"
version = "2.3.3"
@@ -4259,29 +4152,17 @@ dependencies = [
"indexmap 1.9.3",
"serde",
"serde_json",
"serde_with_macros 2.3.3",
"serde_with_macros",
"time 0.3.23",
]
[[package]]
name = "serde_with_macros"
version = "1.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e182d6ec6f05393cc0e5ed1bf81ad6db3a8feedf8ee515ecdd369809bcce8082"
dependencies = [
"darling 0.13.4",
"proc-macro2 1.0.66",
"quote 1.0.31",
"syn 1.0.109",
]
[[package]]
name = "serde_with_macros"
version = "2.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "881b6f881b17d13214e5d494c939ebab463d01264ce1811e9d4ac3a882e7695f"
dependencies = [
"darling 0.20.3",
"darling",
"proc-macro2 1.0.66",
"quote 1.0.31",
"syn 2.0.18",
@@ -4419,16 +4300,6 @@ version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7bd3e3206899af3f8b12af284fafc038cc1dc2b41d1b89dd17297221c5d225de"
[[package]]
name = "sized-chunks"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "16d69225bde7a69b235da73377861095455d298f2b970996eec25ddbb42b3d1e"
dependencies = [
"bitmaps 2.1.0",
"typenum",
]
[[package]]
name = "slab"
version = "0.4.8"
@@ -4579,7 +4450,7 @@ checksum = "9966e64ae989e7e575b19d7265cb79d7fc3cbbdf179835cb0d716f294c2049c9"
dependencies = [
"dotenvy",
"either",
"heck 0.4.1",
"heck",
"hex",
"once_cell",
"proc-macro2 1.0.66",
@@ -4690,7 +4561,8 @@ dependencies = [
"http",
"hyper",
"hyper-ws-listener",
"imbl 2.0.0",
"imbl",
"imbl-value",
"include_dir",
"indexmap 1.9.3",
"ipnet",
@@ -4733,7 +4605,7 @@ dependencies = [
"scopeguard",
"serde",
"serde_json",
"serde_with 2.3.3",
"serde_with",
"serde_yaml",
"sha2 0.10.7",
"sha2 0.9.9",
@@ -4753,9 +4625,9 @@ dependencies = [
"toml",
"torut",
"tracing",
"tracing-error 0.2.0",
"tracing-error",
"tracing-futures",
"tracing-subscriber 0.3.17",
"tracing-subscriber",
"trust-dns-server",
"typed-builder",
"url",
@@ -4859,7 +4731,7 @@ version = "0.24.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e385be0d24f186b4ce2f9982191e7101bb737312ad61c1f2f984f34bcf85d59"
dependencies = [
"heck 0.4.1",
"heck",
"proc-macro2 1.0.66",
"quote 1.0.31",
"rustversion",
@@ -5627,16 +5499,6 @@ dependencies = [
"valuable",
]
[[package]]
name = "tracing-error"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4d7c0b83d4a500748fa5879461652b361edf5c9d51ede2a2ac03875ca185e24"
dependencies = [
"tracing",
"tracing-subscriber 0.2.25",
]
[[package]]
name = "tracing-error"
version = "0.2.0"
@@ -5644,7 +5506,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d686ec1c0f384b1277f097b2f279a2ecc11afe8c133c1aabf036a27cb4cd206e"
dependencies = [
"tracing",
"tracing-subscriber 0.3.17",
"tracing-subscriber",
]
[[package]]
@@ -5668,17 +5530,6 @@ dependencies = [
"tracing-core",
]
[[package]]
name = "tracing-subscriber"
version = "0.2.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e0d2eaa99c3c2e41547cfa109e910a68ea03823cccad4a0525dcbc9b01e8c71"
dependencies = [
"sharded-slab",
"thread_local",
"tracing-core",
]
[[package]]
name = "tracing-subscriber"
version = "0.3.17"
@@ -6390,6 +6241,18 @@ dependencies = [
"thiserror",
]
[[package]]
name = "yasi"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f355ab62ebe30b758c1f4ab096a306722c4b7dbfb9d8c07d18c70d71a945588"
dependencies = [
"ahash 0.8.3",
"hashbrown 0.13.2",
"lazy_static",
"serde",
]
[[package]]
name = "zeroize"
version = "1.6.0"