mirror of
https://github.com/Start9Labs/start-os.git
synced 2026-03-31 04:23:40 +00:00
Feature/lxc container runtime (#2514)
* wip: static-server errors * wip: fix wifi * wip: Fix the service_effects * wip: Fix cors in the middleware * wip(chore): Auth clean up the lint. * wip(fix): Vhost * wip: continue manager refactor Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: service manager refactor * wip: Some fixes * wip(fix): Fix the lib.rs * wip * wip(fix): Logs * wip: bins * wip(innspect): Add in the inspect * wip: config * wip(fix): Diagnostic * wip(fix): Dependencies * wip: context * wip(fix) Sorta auth * wip: warnings * wip(fix): registry/admin * wip(fix) marketplace * wip(fix) Some more converted and fixed with the linter and config * wip: Working on the static server * wip(fix)static server * wip: Remove some asynnc * wip: Something about the request and regular rpc * wip: gut install Co-authored-by: J H <Blu-J@users.noreply.github.com> * wip: Convert the static server into the new system * wip delete file * test * wip(fix) vhost does not need the with safe defaults * wip: Adding in the wifi * wip: Fix the developer and the verify * wip: new install flow Co-authored-by: J H <Blu-J@users.noreply.github.com> * fix middleware * wip * wip: Fix the auth * wip * continue service refactor * feature: Service get_config * feat: Action * wip: Fighting the great fight against the borrow checker * wip: Remove an error in a file that I just need to deel with later * chore: Add in some more lifetime stuff to the services * wip: Install fix on lifetime * cleanup * wip: Deal with the borrow later * more cleanup * resolve borrowchecker errors * wip(feat): add in the handler for the socket, for now * wip(feat): Update the service_effect_handler::action * chore: Add in the changes to make sure the from_service goes to context * chore: Change the * refactor service map * fix references to service map * fill out restore * wip: Before I work on the store stuff * fix backup module * handle some warnings * feat: add in the ui components on the rust side * feature: Update the procedures * chore: Update the js side of the main and a few of the others * chore: Update the rpc listener to match the persistant container * wip: Working on updating some things to have a better name * wip(feat): Try and get the rpc to return the correct shape? * lxc wip * wip(feat): Try and get the rpc to return the correct shape? * build for container runtime wip * remove container-init * fix build * fix error * chore: Update to work I suppose * lxc wip * remove docker module and feature * download alpine squashfs automatically * overlays effect Co-authored-by: Jade <Blu-J@users.noreply.github.com> * chore: Add the overlay effect * feat: Add the mounter in the main * chore: Convert to use the mounts, still need to work with the sandbox * install fixes * fix ssl * fixes from testing * implement tmpfile for upload * wip * misc fixes * cleanup * cleanup * better progress reporting * progress for sideload * return real guid * add devmode script * fix lxc rootfs path * fix percentage bar * fix progress bar styling * fix build for unstable * tweaks * label progress * tweaks * update progress more often * make symlink in rpc_client * make socket dir * fix parent path * add start-cli to container * add echo and gitInfo commands * wip: Add the init + errors * chore: Add in the exit effect for the system * chore: Change the type to null for failure to parse * move sigterm timeout to stopping status * update order * chore: Update the return type * remove dbg * change the map error * chore: Update the thing to capture id * chore add some life changes * chore: Update the loging * chore: Update the package to run module * us From for RpcError * chore: Update to use import instead * chore: update * chore: Use require for the backup * fix a default * update the type that is wrong * chore: Update the type of the manifest * chore: Update to make null * only symlink if not exists * get rid of double result * better debug info for ErrorCollection * chore: Update effects * chore: fix * mount assets and volumes * add exec instead of spawn * fix mounting in image * fix overlay mounts Co-authored-by: Jade <Blu-J@users.noreply.github.com> * misc fixes * feat: Fix two * fix: systemForEmbassy main * chore: Fix small part of main loop * chore: Modify the bundle * merge * fixMain loop" * move tsc to makefile * chore: Update the return types of the health check * fix client * chore: Convert the todo to use tsmatches * add in the fixes for the seen and create the hack to allow demo * chore: Update to include the systemForStartOs * chore UPdate to the latest types from the expected outout * fixes * fix typo * Don't emit if failure on tsc * wip Co-authored-by: Jade <Blu-J@users.noreply.github.com> * add s9pk api * add inspection * add inspect manifest * newline after display serializable * fix squashfs in image name * edit manifest Co-authored-by: Jade <Blu-J@users.noreply.github.com> * wait for response on repl * ignore sig for now * ignore sig for now * re-enable sig verification * fix * wip * env and chroot * add profiling logs * set uid & gid in squashfs to 100000 * set uid of sqfs to 100000 * fix mksquashfs args * add env to compat * fix * re-add docker feature flag * fix docker output format being stupid * here be dragons * chore: Add in the cross compiling for something * fix npm link * extract logs from container on exit * chore: Update for testing * add log capture to drop trait * chore: add in the modifications that I make * chore: Update small things for no updates * chore: Update the types of something * chore: Make main not complain * idmapped mounts * idmapped volumes * re-enable kiosk * chore: Add in some logging for the new system * bring in start-sdk * remove avahi * chore: Update the deps * switch to musl * chore: Update the version of prettier * chore: Organize' * chore: Update some of the headers back to the standard of fetch * fix musl build * fix idmapped mounts * fix cross build * use cross compiler for correct arch * feat: Add in the faked ssl stuff for the effects * @dr_bonez Did a solution here * chore: Something that DrBonez * chore: up * wip: We have a working server!!! * wip * uninstall * wip * tes --------- Co-authored-by: J H <dragondef@gmail.com> Co-authored-by: J H <Blu-J@users.noreply.github.com> Co-authored-by: J H <2364004+Blu-J@users.noreply.github.com>
This commit is contained in:
Aiden McClelland
61
Makefile
61
Makefile
@@ -6,10 +6,10 @@ BASENAME := $(shell ./basename.sh)
|
||||
PLATFORM := $(shell if [ -f ./PLATFORM.txt ]; then cat ./PLATFORM.txt; else echo unknown; fi)
|
||||
ARCH := $(shell if [ "$(PLATFORM)" = "raspberrypi" ]; then echo aarch64; else echo $(PLATFORM) | sed 's/-nonfree$$//g'; fi)
|
||||
IMAGE_TYPE=$(shell if [ "$(PLATFORM)" = raspberrypi ]; then echo img; else echo iso; fi)
|
||||
BINS := core/target/$(ARCH)-unknown-linux-gnu/release/startbox core/target/aarch64-unknown-linux-musl/release/container-init core/target/x86_64-unknown-linux-musl/release/container-init
|
||||
BINS := core/target/$(ARCH)-unknown-linux-musl/release/startbox core/target/$(ARCH)-unknown-linux-musl/release/containerbox
|
||||
WEB_UIS := web/dist/raw/ui web/dist/raw/setup-wizard web/dist/raw/diagnostic-ui web/dist/raw/install-wizard
|
||||
FIRMWARE_ROMS := ./firmware/$(PLATFORM) $(shell jq --raw-output '.[] | select(.platform[] | contains("$(PLATFORM)")) | "./firmware/$(PLATFORM)/" + .id + ".rom.gz"' build/lib/firmware.json)
|
||||
BUILD_SRC := $(shell git ls-files build) build/lib/depends build/lib/conflicts $(FIRMWARE_ROMS)
|
||||
BUILD_SRC := $(shell git ls-files build) build/lib/depends build/lib/conflicts build/lib/container-runtime/rootfs.squashfs $(FIRMWARE_ROMS)
|
||||
DEBIAN_SRC := $(shell git ls-files debian/)
|
||||
IMAGE_RECIPE_SRC := $(shell git ls-files image-recipe/)
|
||||
STARTD_SRC := core/startos/startd.service $(BUILD_SRC)
|
||||
@@ -26,7 +26,7 @@ PATCH_DB_CLIENT_SRC := $(shell git ls-files --recurse-submodules patch-db/client
|
||||
GZIP_BIN := $(shell which pigz || which gzip)
|
||||
TAR_BIN := $(shell which gtar || which tar)
|
||||
COMPILED_TARGETS := $(BINS) system-images/compat/docker-images/$(ARCH).tar system-images/utils/docker-images/$(ARCH).tar system-images/binfmt/docker-images/$(ARCH).tar
|
||||
ALL_TARGETS := $(STARTD_SRC) $(ENVIRONMENT_FILE) $(GIT_HASH_FILE) $(VERSION_FILE) $(COMPILED_TARGETS) $(shell if [ "$(PLATFORM)" = "raspberrypi" ]; then echo cargo-deps/aarch64-unknown-linux-gnu/release/pi-beep; fi) $(shell /bin/bash -c 'if [[ "${ENVIRONMENT}" =~ (^|-)unstable($$|-) ]]; then echo cargo-deps/$(ARCH)-unknown-linux-gnu/release/tokio-console; fi') $(PLATFORM_FILE)
|
||||
ALL_TARGETS := $(STARTD_SRC) $(ENVIRONMENT_FILE) $(GIT_HASH_FILE) $(VERSION_FILE) $(COMPILED_TARGETS) $(shell if [ "$(PLATFORM)" = "raspberrypi" ]; then echo cargo-deps/aarch64-unknown-linux-musl/release/pi-beep; fi) $(shell /bin/bash -c 'if [[ "${ENVIRONMENT}" =~ (^|-)unstable($$|-) ]]; then echo cargo-deps/$(ARCH)-unknown-linux-musl/release/tokio-console; fi') $(PLATFORM_FILE)
|
||||
|
||||
ifeq ($(REMOTE),)
|
||||
mkdir = mkdir -p $1
|
||||
@@ -49,7 +49,7 @@ endif
|
||||
|
||||
.DELETE_ON_ERROR:
|
||||
|
||||
.PHONY: all metadata install clean format sdk snapshots uis ui reflash deb $(IMAGE_TYPE) squashfs sudo wormhole test
|
||||
.PHONY: all metadata install clean format cli uis ui reflash deb $(IMAGE_TYPE) squashfs sudo wormhole test
|
||||
|
||||
all: $(ALL_TARGETS)
|
||||
|
||||
@@ -74,6 +74,11 @@ clean:
|
||||
rm -rf image-recipe/deb
|
||||
rm -rf results
|
||||
rm -rf build/lib/firmware
|
||||
rm -rf container-runtime/dist
|
||||
rm -rf container-runtime/node_modules
|
||||
rm -f build/lib/container-runtime/rootfs.squashfs
|
||||
rm -rf sdk/dist
|
||||
rm -rf sdk/node_modules
|
||||
rm -f ENVIRONMENT.txt
|
||||
rm -f PLATFORM.txt
|
||||
rm -f GIT_HASH.txt
|
||||
@@ -85,8 +90,8 @@ format:
|
||||
test: $(CORE_SRC) $(ENVIRONMENT_FILE)
|
||||
cd core && cargo build && cargo test
|
||||
|
||||
sdk:
|
||||
cd core && ./install-sdk.sh
|
||||
cli:
|
||||
cd core && ./install-cli.sh
|
||||
|
||||
deb: results/$(BASENAME).deb
|
||||
|
||||
@@ -106,15 +111,13 @@ results/$(BASENAME).$(IMAGE_TYPE) results/$(BASENAME).squashfs: $(IMAGE_RECIPE_S
|
||||
# For creating os images. DO NOT USE
|
||||
install: $(ALL_TARGETS)
|
||||
$(call mkdir,$(DESTDIR)/usr/bin)
|
||||
$(call cp,core/target/$(ARCH)-unknown-linux-gnu/release/startbox,$(DESTDIR)/usr/bin/startbox)
|
||||
$(call cp,core/target/$(ARCH)-unknown-linux-musl/release/startbox,$(DESTDIR)/usr/bin/startbox)
|
||||
$(call ln,/usr/bin/startbox,$(DESTDIR)/usr/bin/startd)
|
||||
$(call ln,/usr/bin/startbox,$(DESTDIR)/usr/bin/start-cli)
|
||||
$(call ln,/usr/bin/startbox,$(DESTDIR)/usr/bin/start-sdk)
|
||||
$(call ln,/usr/bin/startbox,$(DESTDIR)/usr/bin/start-deno)
|
||||
$(call ln,/usr/bin/startbox,$(DESTDIR)/usr/bin/avahi-alias)
|
||||
$(call ln,/usr/bin/startbox,$(DESTDIR)/usr/bin/embassy-cli)
|
||||
if [ "$(PLATFORM)" = "raspberrypi" ]; then $(call cp,cargo-deps/aarch64-unknown-linux-gnu/release/pi-beep,$(DESTDIR)/usr/bin/pi-beep); fi
|
||||
if /bin/bash -c '[[ "${ENVIRONMENT}" =~ (^|-)unstable($$|-) ]]'; then $(call cp,cargo-deps/$(ARCH)-unknown-linux-gnu/release/tokio-console,$(DESTDIR)/usr/bin/tokio-console); fi
|
||||
if [ "$(PLATFORM)" = "raspberrypi" ]; then $(call cp,cargo-deps/aarch64-unknown-linux-musl/release/pi-beep,$(DESTDIR)/usr/bin/pi-beep); fi
|
||||
if /bin/bash -c '[[ "${ENVIRONMENT}" =~ (^|-)unstable($$|-) ]]'; then $(call cp,cargo-deps/$(ARCH)-unknown-linux-musl/release/tokio-console,$(DESTDIR)/usr/bin/tokio-console); fi
|
||||
|
||||
$(call mkdir,$(DESTDIR)/lib/systemd/system)
|
||||
$(call cp,core/startos/startd.service,$(DESTDIR)/lib/systemd/system/startd.service)
|
||||
@@ -128,10 +131,6 @@ install: $(ALL_TARGETS)
|
||||
$(call cp,GIT_HASH.txt,$(DESTDIR)/usr/lib/startos/GIT_HASH.txt)
|
||||
$(call cp,VERSION.txt,$(DESTDIR)/usr/lib/startos/VERSION.txt)
|
||||
|
||||
$(call mkdir,$(DESTDIR)/usr/lib/startos/container)
|
||||
$(call cp,core/target/aarch64-unknown-linux-musl/release/container-init,$(DESTDIR)/usr/lib/startos/container/container-init.arm64)
|
||||
$(call cp,core/target/x86_64-unknown-linux-musl/release/container-init,$(DESTDIR)/usr/lib/startos/container/container-init.amd64)
|
||||
|
||||
$(call mkdir,$(DESTDIR)/usr/lib/startos/system-images)
|
||||
$(call cp,system-images/compat/docker-images/$(ARCH).tar,$(DESTDIR)/usr/lib/startos/system-images/compat.tar)
|
||||
$(call cp,system-images/utils/docker-images/$(ARCH).tar,$(DESTDIR)/usr/lib/startos/system-images/utils.tar)
|
||||
@@ -148,8 +147,8 @@ update-overlay: $(ALL_TARGETS)
|
||||
$(MAKE) install REMOTE=$(REMOTE) SSHPASS=$(SSHPASS) PLATFORM=$(PLATFORM)
|
||||
$(call ssh,"sudo systemctl start startd")
|
||||
|
||||
wormhole: core/target/$(ARCH)-unknown-linux-gnu/release/startbox
|
||||
@wormhole send core/target/$(ARCH)-unknown-linux-gnu/release/startbox 2>&1 | awk -Winteractive '/wormhole receive/ { printf "sudo /usr/lib/startos/scripts/chroot-and-upgrade \"cd /usr/bin && rm startbox && wormhole receive --accept-file %s && chmod +x startbox\"\n", $$3 }'
|
||||
wormhole: core/target/$(ARCH)-unknown-linux-musl/release/startbox
|
||||
@wormhole send core/target/$(ARCH)-unknown-linux-musl/release/startbox 2>&1 | awk -Winteractive '/wormhole receive/ { printf "sudo /usr/lib/startos/scripts/chroot-and-upgrade \"cd /usr/bin && rm startbox && wormhole receive --accept-file %s && chmod +x startbox\"\n", $$3 }'
|
||||
|
||||
update: $(ALL_TARGETS)
|
||||
@if [ -z "$(REMOTE)" ]; then >&2 echo "Must specify REMOTE" && false; fi
|
||||
@@ -166,6 +165,26 @@ emulate-reflash: $(ALL_TARGETS)
|
||||
upload-ota: results/$(BASENAME).squashfs
|
||||
TARGET=$(TARGET) KEY=$(KEY) ./upload-ota.sh
|
||||
|
||||
container-runtime/alpine.squashfs: $(PLATFORM_FILE)
|
||||
ARCH=$(ARCH) ./container-runtime/download-base-image.sh
|
||||
|
||||
container-runtime/node_modules: container-runtime/package.json container-runtime/package-lock.json sdk/dist
|
||||
npm --prefix container-runtime ci
|
||||
touch container-runtime/node_modules
|
||||
|
||||
sdk/dist: $(shell git ls-files sdk)
|
||||
(cd sdk && make bundle)
|
||||
|
||||
container-runtime/dist: container-runtime/node_modules $(shell git ls-files container-runtime/src) container-runtime/package.json container-runtime/tsconfig.json
|
||||
npm --prefix container-runtime run build
|
||||
|
||||
container-runtime/dist/node_modules container-runtime/dist/package.json container-runtime/dist/package-lock.json: container-runtime/package.json container-runtime/package-lock.json sdk/dist container-runtime/install-dist-deps.sh
|
||||
./container-runtime/install-dist-deps.sh
|
||||
touch container-runtime/dist/node_modules
|
||||
|
||||
build/lib/container-runtime/rootfs.squashfs: container-runtime/alpine.squashfs container-runtime/containerRuntime.rc container-runtime/update-image.sh container-runtime/dist container-runtime/dist/node_modules core/target/$(ARCH)-unknown-linux-musl/release/containerbox $(PLATFORM_FILE) | sudo
|
||||
ARCH=$(ARCH) ./container-runtime/update-image.sh
|
||||
|
||||
build/lib/depends build/lib/conflicts: build/dpkg-deps/*
|
||||
build/dpkg-deps/generate.sh
|
||||
|
||||
@@ -181,10 +200,6 @@ system-images/utils/docker-images/$(ARCH).tar: $(UTILS_SRC)
|
||||
system-images/binfmt/docker-images/$(ARCH).tar: $(BINFMT_SRC)
|
||||
cd system-images/binfmt && make docker-images/$(ARCH).tar && touch docker-images/$(ARCH).tar
|
||||
|
||||
snapshots: core/snapshot-creator/Cargo.toml
|
||||
cd core/ && ARCH=aarch64 ./build-v8-snapshot.sh
|
||||
cd core/ && ARCH=x86_64 ./build-v8-snapshot.sh
|
||||
|
||||
$(BINS): $(CORE_SRC) $(ENVIRONMENT_FILE)
|
||||
cd core && ARCH=$(ARCH) ./build-prod.sh
|
||||
touch $(BINS)
|
||||
@@ -231,8 +246,8 @@ uis: $(WEB_UIS)
|
||||
# this is a convenience step to build the UI
|
||||
ui: web/dist/raw/ui
|
||||
|
||||
cargo-deps/aarch64-unknown-linux-gnu/release/pi-beep:
|
||||
cargo-deps/aarch64-unknown-linux-musl/release/pi-beep:
|
||||
ARCH=aarch64 ./build-cargo-dep.sh pi-beep
|
||||
|
||||
cargo-deps/$(ARCH)-unknown-linux-gnu/release/tokio-console:
|
||||
cargo-deps/$(ARCH)-unknown-linux-musl/release/tokio-console:
|
||||
ARCH=$(ARCH) ./build-cargo-dep.sh tokio-console
|
||||
Reference in New Issue
Block a user