diff --git a/source/contributing/services/config.rst b/source/contributing/services/config.rst index 210692b..b82949d 100644 --- a/source/contributing/services/config.rst +++ b/source/contributing/services/config.rst @@ -4,7 +4,12 @@ Service Config Specification **************************** -Configuration options are a crucial component of any application. EmbassyOS requires a particular format to ensure your service integrates smoothly with the user interface. This enables clean handling of improper values and dependency management. +Overview +======== + +Most self-hosted applications require the user to tell the app how to behave using a config file in a specific format, environment variables, command-line arguments, or some combination of these inputs. One of the coolest features of EmbassyOS is that, when packaged correctly, the app's configuration will be available to the user as a slick GUI that always produces a valid configuration no matter how little experience or skill the user has. + +With EmbassyOS, this means a service wrappers' configuration requires a particular format and rule structure to ensure it integrates smoothly with the user interface. This format enables clean handling of improper values and dependency management. The outcome of this step is two files: diff --git a/source/contributing/services/docker.rst b/source/contributing/services/docker.rst index 9cfc571..1af80ca 100644 --- a/source/contributing/services/docker.rst +++ b/source/contributing/services/docker.rst @@ -7,7 +7,7 @@ Service Dockerfile Dockerfile ========== -This file defines how to build the image for the service. It declares the environment and building stages. +This purpose of the Dockerfile is to define how to build the image for the service. It declares the environment and building stages. The Dockerfile is responsible for mounting the service application to whatever volume the manifest specifies, typically ``/root``. Since the build requires specific arm runtime environments, these base images can be imported into the Dockerfile as a starting point so the base system does not need to be completely redefined. This enables importing a specific build environment version that encapsulates any necessary libraries for running the service build, eg. golang, rust. @@ -18,7 +18,7 @@ For instance: ``FROM arm32v7/alpine`` -``arm32v7/golang:alpine`` +``FROM arm32v7/golang:alpine`` Entry-point @@ -30,6 +30,12 @@ This file defines what to do when the service application starts. It consists of a bash script that completes any environment setup (eg. creating folder substructure), sets any environment variables, and executes the run command. The only required feature of this file is to execute the run commands on EmbassyOS. +Example +======= + +See an example `here `_. + + .. role:: raw-html(raw) :format: html diff --git a/source/contributing/services/index.rst b/source/contributing/services/index.rst index a2cfd05..79c0c9d 100644 --- a/source/contributing/services/index.rst +++ b/source/contributing/services/index.rst @@ -4,15 +4,21 @@ Service Packaging Guide *********************** -This technical guide covers the necessary steps to package up a service for deployment to the EmbassyOS platform. To start, you will need to acquire EmbassyOS for testing the packaged service. This can be done by: +Welcome! If you are here, you are interested in becoming part of the mission to change the future of personal computing. This guide will take you through the process of packaging a service for EmbassyOS, a novel, self-hosted, sovereign computing platform. + +A service in this context is any open source project that has been bundled into the appropriate format to run on EmbassyOS. By configuring and packaging a project according to this guide, it can be installed on EmbassyOS with no command line or technical expertise required. This opens up an entire platform for self-hosted software to run independent of third parties in a completely private and sovereign way for all individuals. + +This guide is technical, but breaks down the steps for deployment. If you have any feedback or questions concerning this guide, please don't hesitate to `reach out `_. + +To start, you will need to acquire EmbassyOS for testing the packaged service. This can be done by: - building from `source `_ -- :ref:`DIY ` -- :ref:`purchasing ` +- following the :ref:`DIY ` guide +- :ref:`purchasing ` the ready to run personal server -While you are waiting to assemble a device, you can begin the process of building your project in the format necessary to run on the Embassy. The sections below outline these steps in detail. +While you are waiting to receive or assemble a device, you can begin the process of packaging a project. The sections below outline these steps in detail. -For reference, the `Hello world `_ repository should be used as an example. A project template can be cloned using the "Use this template" button in GitHub. +Happy building! .. toctree:: :maxdepth: 2 @@ -29,7 +35,7 @@ For reference, the `Hello world `_. +When you have built and tested your project for EmbassyOS, please send Start9 a submission with the project repository to ``dev@start9labs.com``. After being reviewed for security and compatibility, the service will be deployed to the marketplace and available for all EmbassyOS users to download. .. role:: raw-html(raw) :format: html diff --git a/source/contributing/services/makefile.rst b/source/contributing/services/makefile.rst index dd93ec4..0f60858 100644 --- a/source/contributing/services/makefile.rst +++ b/source/contributing/services/makefile.rst @@ -17,6 +17,11 @@ This purpose of this file is to: - Build all dependencies - Package ``config.yaml``, ``manifest.yaml``, and ``image.tar`` into an ``.s9pk`` extension by invoke ``appmgr``. +Example +======= + +See an example `here `_. + .. role:: raw-html(raw) :format: html diff --git a/source/contributing/services/manifest.rst b/source/contributing/services/manifest.rst index 29fb169..fbeccec 100644 --- a/source/contributing/services/manifest.rst +++ b/source/contributing/services/manifest.rst @@ -7,9 +7,19 @@ Service Manifest Overview ======== -This file describes the service and it's requirements. It is used by the marketplace to create a service listing. +This file describes the service and it's requirements. It is used to: -Important of note is the :ref:`dependencies ` key, which contains rules for optional or required other services that are depended on for the service to successfully run. +- create a listing in the marketplace +- denote any installation considerations +- define dependency requirements + +Each time a service is updated, the manifest should be updated to include the new version, release notes, and any pertinent updates to the install, uninstall, or restoration flows. All this information is displayed in the marketplace listing, and the optionally denoted alerts will be displayed when appropriate to provide the user with more information about the particularities of the service. + +For instance, `LND displays alerts `_ around restoration since data will be overwritten. + +There is nothing you need to do as a developer to enable the service to run over Tor. This is *completely* handled by EmbassyOS - a Tor address will be automatically generated when the service is installed. Just define which version of Tor your service needs in this manifest file! + +The manifest is also responsible for outlining service :ref:`dependencies `. By defining rules using the :ref:`EmbassyOS DSL specification `, users can easily and selectively install, uninstall, and update any service without getting stuck in dependency hell. EmbassyOS presents this information in a polished install/uninstall/update wizard, so there's no need for editing configuration files or jumping into the command line. For you as a developer, this simply means populating this key in the manifest! Formatting ========== diff --git a/source/contributing/services/overview.rst b/source/contributing/services/overview.rst index cd476cd..c0c937f 100644 --- a/source/contributing/services/overview.rst +++ b/source/contributing/services/overview.rst @@ -4,26 +4,14 @@ Service Package Overview ************************ -Let's start at a high level. What actually is a service, in a technical sense, on the Embassy? +There are several main components that comprise a service package for EmbassyOS. This overview will introduce them and following sections will dive into technical specifics. -Each service is an image. An image is a snapshot of a linux environment in which an application runs. All service application data is stored in a volume directory. This volume is persisted across updates. The image is immutable: when the service is updated, the image is replaced with the complete new image containing the updated features. - -The volume directory (for seeding data into the volume) is typically: ``/root/volumes/``. - -.. warning:: - Any files that are in the image at the volume path will be overwritten when a backup restore occurs. - -A service is mounted to the EmbassyOS image. Each service application gets a volume allocated to it by EmbassyOS. Within the Docker container that encompasses the service image, a path is specified as to where it should mount on EmbassyOS. This path is specified in the :ref:`manifest `. - -The config spec, config rules, and manifest get bundled into the ``.s9pk`` package, `built using appmgr `_. Each new version release should include the updated version of these files re-bundled into a new binary. - -.. note:: - Due to tech debt, code in appmgr may reference ``app-id`` instead of ``service-id``. +First, let's get your system setup with the necessary software dependencies. Environment Setup ================= -At minimum, docker, docker-builx, and appmgr are required to package a service for EmbassyOS. +At minimum, ``docker``, ``docker-buildx``, and ``appmgr`` are required. The rest of the recommendations serve to optimize the build process. Recommended Dependencies ------------------------ @@ -42,6 +30,73 @@ Recommended Dependencies `musl `_ +Package Components +================== + +Image +----- + +Each service boils down to a Docker image. We're not going to dive into Docker specifics in this guide, since there exists tons of `resources `_ for developing with this containerization tool. + +Because the service ultimately runs on a Raspberry Pi, the created Docker image should be a snapshot of a arm based linux environment. The service image is then mounted to the EmbassyOS image during installation. This path is defined in the :ref:`manifest ` configuration file. + +The image is immutable, meaning that when the service is updated, the image is replaced with a completely new image containing the updated features. + +Volume +------ +Each service application gets a volume allocated to it by EmbassyOS. All service application data is stored in this directory and is persisted across updates. + +The volume directory (for seeding data into the volume) is typically: ``/root/volumes/``. + +.. warning:: + Any files that are in the image at the volume path will be overwritten when a backup restore occurs. + +Dependencies +------------ + +When it comes to running your own server, managing dependencies is perhaps the most difficult part. The term "dependency hell" emerged from the this sentiment. Even the best developers have lost significant amounts of time trying to find, correct, or clarify documentation for dependencies or dependency configuration. Individuals who manage their own servers have specific technical skills and are willing to devote the time and effort necessary to maintain them. Companies have entire departments dedicated to this feat. + +Some other personal server products aimed at non-technical individuals tackle this problem by simply hard coding dependencies. Since EmbassyOS is a platform for running all open-source, self-hosted software, it is not possible to hard code every possible service, service dependency, and service dependency configuration forever. Instead, Start9 built a new, unprecedented operating system that touts a generalized, intuitive approach to dependency management and service configuration. EmbassyOS enables users to easily and selectively install, uninstall, and update any service they want without getting stuck in dependency hell. + +This may sound cool or neat, but it is more than that: *it’s novel*. This has never been done before. + +The key to making the system work is a new, domain-specific-language (DSL) and set of standards that are used by developers to define the rules and requirements of their services. Run in the context of EmbassyOS, these rules and requirements appear as simple UI elements, such as inputs, toggles, and drop downs, and they are enforced by validations and clear user instructions. Using this system, what previously required involved time and expertise, can now be done by anyone in a matter of seconds. + +This DSL is utilized in the :ref:`config rules ` and :ref:`dependencies ` key in the service manifest. + +Manifest +-------- + +This file describes the service and it's requirements. It is used to: + +- create a listing in the marketplace +- denote any installation considerations +- define dependency requirements + +Each time a service is updated, the manifest should be updated to include the new version, release notes, and any pertinent updates to the install, uninstall, or restoration flows. All this information is displayed in the marketplace listing, and the optionally denoted alerts will be displayed when appropriate to provide the user with more information about the particularities of the service. + +For instance, `LND displays alerts `_ around restoration since data will be overwritten. + +There is nothing you need to do as a developer to enable the service to run over Tor. This is *completely* handled by EmbassyOS - a Tor address will be automatically generated when the service is installed. Just define which version of Tor your service needs in this manifest file! + +Config +------ + +Most self-hosted applications require the user to tell the app how to behave using a config file in a specific format, environment variables, command-line arguments, or some combination of these inputs. One of the coolest features of EmbassyOS is that, when packaged correctly, the app's :ref:`configuration ` will be available to the user as a slick GUI that always produces a valid configuration no matter how little experience or skill the user has. + +With EmbassyOS, this means a service wrappers' configuration requires a particular format and rule structure to ensure it integrates smoothly with the user interface. This format enables clean handling of improper values and dependency management. + +.s9pk Bundle +------------ + +The configuration and manifest files get bundled into the ``.s9pk`` package, which is `built using appmgr `_. Each new version release should include the updated version of these files re-bundled into a new binary. + +Hello World Example +------------------- + +For reference, the `Hello world `_ repository should be used as an example. A project template can be cloned using the "Use this template" button in GitHub. + + .. role:: raw-html(raw) :format: html