Engine Installer

Managing on-premise installations is hard. We found that our users have a need for simple tooling that lets them install, update and manage their Engine workspace installations.

This setup script enables Engine on-premise users to install and update Engine on their own, create backups and clean their database.

important

The setup script only backs up files that are modified by the update: configuration, Cloudomation binaries, and database content. It doesn't include the OS or git repositories.

In cases where a full machine backup is needed, you need to create it manually.

Human requirements:

• Basic Linux knowledge
• Service account & license file (you get those when you buy a license)
• Having read the Engine docs https://docs.cloudomation.com/

System requirements:

note

For the on-premise system requirements please click here.

Download

This version of Cloudomation Engine is not yet released.

Usage

Introduction

The setup.sh file is your single entry point to either a fresh Engine workspace installation, an update of an existing workspace installation or to perform database cleanups and licensing tasks.

Flag	Description
-h | --help	Display help in the terminal
-d | --directory DIR	Where Engine should be installed. Defaults to ./cloudomation
-v | --verbose	Enable verbose logging, provides additional output. Useful for debugging
--cert-mode	Which certificate to use. one of selfsigned (default), existing, or letsencrypt
--cert <crt> <key>	Specify path to existing certificate and key. required for cert-mode existing
--interactive	Run interactive (default)
--no-interactive	Run without any interactive prompts
--setup-database	Create databases & database users (default)
--no-setup-database	Do not attempt to create databases & database users
--provide-database	Provide a postgresql service (default).
--no-provide-database	Do not provide a postgresql service. if set, connection parameters for an external database must be provided in .env.
--provide-vault	Provide a HashiCorp Vault instance during setup
--no-provide-vault	Do not provide a HashiCorp Vault instance during setup (default)
--vacuum	Vacuum the postgresql database Engine uses
--save-logs	Collect logs, create a zip containing the logs and exit
--workspaces N	The number of workspace (and paired sandbox) worker containers (default 4). It overrides and is persisted to the WORKER_COUNT key in setup.yml, so it survives future upgrades without re-passing the flag. See Configuration keys
--no-write-setup	Keep the existing setup.yml as-is; do not overwrite it with template values. Useful for a re-run that must preserve the current configuration exactly
--database-remove-config	Remove one or more workspace configuration keys from the database
--database-no-upgrade	Prevent the automatic database-schema upgrade during a deploy
--offline	Air-gapped install: load images from ./images/*.tar.gz, skip the docker-registry login, and set pull_policy: never on every service. Use with a non-letsencrypt --cert-mode. See Offline / Air-gapped Installation
--no-offline	Registry-connected install (default): pull images from the container registry
--smtp-oauth-authorize	Run the SMTP OAUTH2 authorization flow and exit. Used to obtain the OAUTH2 tokens for SMTP_AUTHENTICATION_MODE: XOAUTH2 (see Configuration keys)
--restore-backup <file>	Restore the workspace to the state in backup <file>
--backup-now	Create a backup now and exit, without deploying or upgrading. Briefly stops services to take a consistent snapshot, then restarts them
--backup-target <path>	With --backup-now: write the backup to <path> — a directory receives a timestamped backup-cloudomation-<timestamp>.tar; a .tar path is used verbatim. Default: the installation's backups/ folder
--no-include-data	Omit the installation's data/ directory (file-record content and local git repositories) from the backup. By default data/ is included
--no-backup	Skip the automatic pre-upgrade backup. See Skipping the automatic pre-upgrade backup
--health-timeout <sec>	Seconds to wait for the workspace to become healthy after an upgrade (default 300). See Waiting for the upgraded workspace to become healthy
--no-health-wait	Return as soon as the containers are (re)started, without waiting for the workspace to become healthy
--wait-healthy	Poll the already-running workspace until it becomes healthy and exit, without deploying or upgrading
downgrade <target>	Downgrade the workspace database schema to alembic revision <target> and exit — an expert-only escape hatch with no safety checks or backup. See downgrade the database schema

New Installation

To install a fresh Engine workspace installation first extract the zip file. Place your registry-secret.txt and license.yml file in the same directory as setup.sh. Open a terminal in this directory and run ./setup.sh. Set the desired installation directory with -d flag. If no flag is passed, the default value ./cloudomation is used.

Most configuration values are set to reasonable defaults. Still, you will be prompted for the following settings:

Setting	Description
ORGANIZATION_NAME	Name of your organization
WORKSPACE_NAME	Name of your workspace. Useful if you want to create multiple workspaces.
BOOTSTRAP_USER	An email address. An invite will be sent to this address. The user will be able to set a password and login to Engine.
SMTP_HOST	Hostname of your SMTP server
SMTP_PORT	Port of your SMTP server
SMTP_USE_TLS	Whether or not to use TLS for SMTP

Install self-signed certificate in browser

After the installation is done, you will need to install the self-signed HTTPS certificate in your browser as an authority. The certificate cert.crt is located in the Engine root directory.

Install cert in Chrome

Install cert in Firefox

Install cert in Brave

Your Engine workspace is now ready for action! 🎉🎉

Per default it is reachable via https://localhost

Change Configuration

If you want to change any settings (wether pre-defined or user-defined), edit setup.yml and run setup.sh again. A diff editor will open, enabling you to control which values should be used. The documentation for imediff is available here. Press h for help.

Upgrade Installation

warning

Before upgrading to a new major version please review the Release Notes as sometimes manual steps are required.

warning

It is generally a good idea to make sure that your processes don't use deprecated features. This becomes especially important when upgrading to a new major version as deprecated features might get removed in newer versions. See here how you can easily find deprecation notes in your workspace.

note

The installer can only upgrade Engine workspace installations which were previously installed using the installer. Upgrading from manual installations (pre Version 6) requires additional steps as described in Migrate from a manual installation.

To upgrade an existing workspace installation first download the new version of the installer from above. Extract the zip file. Open a terminal in this directory and run ./setup.sh.

The installer will prompt you for new or modified configurations and then upgrade and restart the Engine workspace.

Waiting for the upgraded workspace to become healthy

After the containers are (re)started the installer waits for the workspace to become healthy — it triggers a fresh engine check and reports success only once that check passes — so a completed ./setup.sh run means the upgraded workspace is serving requests. The wait counts only non-migrating time, so a legitimately long database migration does not cause a spurious failure.

Three flags control this behaviour:

• --health-timeout <sec> — how long to wait for the workspace to become healthy, in seconds (default 300). It also applies when running --wait-healthy.
• --no-health-wait — return as soon as the containers are (re)started, without waiting for the workspace to become healthy. Use this when you manage health checking yourself. With this flag the installer does not detect a failed upgrade, so it triggers no recovery on its own.
• --wait-healthy — poll the already-running workspace until it becomes healthy and exit, without deploying or upgrading anything. It honours --health-timeout and is useful as a scripted "wait until ready" step after starting the workspace separately.

If the workspace does not become healthy within the timeout, the installer exits with a non-zero status so the failure is visible to whatever drives the upgrade. Recover by restoring the automatic pre-upgrade backup with --restore-backup.

Skipping the automatic pre-upgrade backup

A backup is created automatically as the first step of an upgrade. Pass --no-backup to skip it — for example when the automation that drives setup manages backups itself, or when you have just taken one with --backup-now.

Migrate from a manual installation

If you are currently operating an Engine workspace which was not installed using the Engine installer the automatic upgrade does not work. There are two options:

1. Keep the old workspace installation running and create a second workspace installation with the Engine installer. Migrate content and remove the old workspace installation. This is the recommended approach when you are currently using the Git Integration to synchronize Engine content from a git repository.
2. Create a new workspace installation with the Engine installer. Shut down the new installation and copy the database folder from the old installation to the new installation. Start the new installation. The workspace will perform any required database migration steps.

Migrate using Git Integration

This method creates a fresh empty new Engine installation. The Git Integration is used to synchronize your resources into the new Engine installation.

warning

Using this method any content which is not in your git repository will not be migrated to the new Engine installation. This includes all activities, like executions, and resources which were manually created in the Engine workspace.

Please follow the steps outlined below:

1. Make a backup of your old Engine installation.

2. Change the ports of the old Engine installation

   The old Engine installation must be modified to be accessible via a different port. Stop the old installation by running docker-compose down in the old installation folder. Modify docker-compose.yml and change the ports of the "ingress" service:

   # before
   ingress:
     ports:
       - "80:80"
       - "443:443"
   # after
   ingress:
     ports:
       - "8080:80"
       - "8443:443"

3. Start the old Engine installation

   Run docker-compose up -d in the old installation folder to start the old installation. Verify that the processes successfully start and make sure you can log in to your old installation.

4. Create your new Engine installation by following the steps described in New Installation.

5. Log into your new Engine installation and setup the Git Integration.

6. Verify that all resources which are required for your use case are successfully synced to your new Engine installation.

   tip

   If your old Engine installation contained resources which were manually created and not in your git repository they will be missing from the new installation. This is a good chance to identify such resources and the perfect time to add them to your git repository.

7. Test your use cases in the new Engine installation. If needed, you can keep the old Engine installation running in parallel for several days. Once you are confident that the new installation meets your requirements the old installation can be deleted.

   warning

   When activating a use case in the new installation it should at the same time be disabled in the old installation.

note

Using this method it is easily possible to switch back to the old Engine installation at any time should any troubles arise.

Migrate by Copying the Database Folder

1. Make a backup of your old Engine installation.

2. Stop the old Engine installation by running docker-compose down in the old installation folder.

3. Create your new Engine installation by following the steps described in New Installation.

4. Stop the new Engine installation by running docker-compose down in the new installation folder.

5. Delete the pgdata directory in your new installation folder.

   warning

   Make sure you delete the pgdata directory of your new Engine installation.

6. Copy the pgdata directory of your old Engine installation to the new installation folder.

   note

   If you completed a full backup of your old installation (including the pgdata folder) it may be faster to move the pgdata folder to the new installation.

7. Start the new Engine installation.

   Check the logs of the Engine workspace container for successful database migration steps.

8. Test your use cases in the new Engine installation.

Additional Tools

The setup script comes with one additional utility for managing your Engine installation.

vacuum postgresql

This executes postgresql's internal garbage-collection feature, which reclaims space that is not in use by the database anymore. For performance reasons, this dead-space is not reclaimed while the database is running. Further information: https://www.postgresql.org/docs/current/sql-vacuum.html

The --vacuum flag safely shuts down Engine, executes the vacuum command and restarts all services.

create a backup now

Backups are normally created automatically as the first step of an upgrade. The --backup-now flag lets you create a backup on demand — for example before a risky manual change — without deploying or upgrading anything.

It runs the same, trusted backup routine the upgrade path uses: it briefly stops the Engine services to take a consistent on-disk snapshot, writes it to <installation directory>/backups/backup-cloudomation-<timestamp>.tar, and then restarts the existing services without pulling new images or changing configuration. Use --backup-target <path> to write the backup somewhere other than the default backups/ folder (a directory receives a timestamped file; a .tar path is used verbatim).

Because a consistent filesystem snapshot of the database requires the database to be stopped, --backup-now incurs a brief downtime for the duration of the snapshot; it is not a zero-downtime backup. The services are always brought back up afterwards, even if the backup fails. The resulting archive can be restored with --restore-backup <file>.

note

The backup contains the database (pgdata), the installation configuration, and — by default — the data/ directory (the on-disk content of file records under data/files and local git-integration repositories under data/repositories) so that a restore is fully restorable. Pass --no-include-data to omit data/ — useful for a git-integration-heavy installation whose data/repositories repositories are large. The Vault secrets under vault-secrets/ are never included.

Escrow the secret-store passphrase off-host

The installation configuration holds SECRET_STORE_PASSPHRASE, the single value that unwraps the built-in secret store's master key. Every named secret and every inline secret is encrypted under that master key, so this passphrase is the linchpin for all stored secret material.

Keep a copy of SECRET_STORE_PASSPHRASE off-host, escrowed separately from the installation — a password manager or a secrets vault your operations team controls. A database backup restored into an environment where the original passphrase is unknown leaves every stored secret unrecoverable: the wrapped master key cannot be unwrapped, and reads fail closed with a SecretStoreSealedError. Restoring a --backup-now archive in place is self-contained because it carries the matching configuration; a database-only restore, a migration to a fresh installation, or a rebuild after losing the host is not, unless the passphrase is restored alongside it.

The passphrase lives in setup.yml (injected into the workspace containers as SECRET_STORE_PASSPHRASE). It is auto-generated once on the first install and stays stable across deploys. As a safeguard, setup refuses to regenerate it: if the database already holds a wrapped master key but the passphrase value is missing, the deploy fails loudly rather than minting a competing key that would orphan the existing secrets — restore the original passphrase into setup.yml (or the deploy environment) and re-run.

caution

setup never deletes backups. Retention of old backups is the responsibility of the automation that runs setup: if you run setup on a schedule or from an upgrade pipeline, that automation must prune old backups/*.tar files (and reclaim other disk, e.g. docker system prune) to keep the installation's disk bounded. A backup written with --backup-target to a directory outside backups/ is not touched by any setup run.

restore a backup

The --restore-backup <file> flag restores the workspace to the state captured in a backup. <file> is the name of a backup archive in <installation directory>/backups/ — the folder where --backup-now and the automatic pre-upgrade backup write their archives.

Restoring stops the Engine services, replaces the database (pgdata), the installation configuration, and — unless the backup was taken with --no-include-data — the data/ directory with the contents of the archive, then starts the services again. Pair a restore with the code version the backup was taken under.

downgrade the database schema

The downgrade <target> command hands <target> straight to alembic downgrade against the installation's workspace database and exits. It runs no safety checks, takes no backup, and does not stop the services — it is an expert-only escape hatch for reverting a single schema migration. Pair it with the matching code version and a running database. To recover a whole installation, prefer --restore-backup.

HashiCorp Vault

The setup can optionally deploy a vault instance alongside Cloudomation as a secret storage. This feature is activated by the --provide-vault feature flag. If a vault is deployed, then by default a unseal script is also installed, which configures the vault using the following variables:

• VAULT_AUTO_UNSEAL (bool, default true) provide the secript to automatically unseal the vault during startup. This script also runs the initial setup of the vault using the other variables.
• VAULT_AUTO_UNSEAL_KEYS (int, default 4) how many unseal keys to generate during initial setup.
• VAULT_AUTO_UNSEAL_SHARES (int, default 2) how many keys are required to unseal the vault.

warning

You should not use this feature if your threat model includes compromises of the system running Cloudomation. Since the unseal script needs access to the keys they are stored alongside the script and thus on the same system as the vault itself.

Configuration keys (setup.yml)

setup.yml is the single declarative configuration file for an installation. During a fresh install the installer prompts for the required identity and SMTP settings (see New Installation); every other key has a sensible default. To change any value later, edit setup.yml and re-run setup.sh (see Change Configuration).

The keys below are the ones an operator adjusts. Values are read on every setup.sh run, so a change survives upgrades — the file is the durable source of truth.

note

Some keys are managed automatically and are not meant to be edited by hand: identifiers and generated secrets (UID, GID, ORGANIZATION_ID, WORKSPACE_ID, WORKSPACE_API_KEY, SECRET_STORE_PASSPHRASE), the container image pins (*_IMAGE, *_IMAGE_NAME, *_IMAGE_TAG), and the values derived from SELF_URL (SUBDOMAIN, DOMAIN, SERVER_NAME, REACT_APP_REST_API, REACT_APP_WEBSOCKET_API). In particular SECRET_STORE_PASSPHRASE unwraps the built-in secret store's master key and must stay stable across deploys — if it changes, secrets stored under the previous passphrase cannot be decrypted.

Workers

Key	Description
WORKER_COUNT	The number of workspace worker containers, each paired 1:1 with a sandbox container (default 4). Right-size it for the host's available RAM on small or single-tenant installs. The --workspaces flag overrides this key and is persisted back into it.

SMTP

The workspace sends email (user invites, notifications) through these settings. SMTP_HOST, SMTP_PORT, and the TLS/authentication settings are prompted during a fresh install.

Key	Description
SMTP_HOST	Hostname of the SMTP server
SMTP_PORT	Port of the SMTP service (default 587)
SMTP_USE_STARTTLS	Use opportunistic STARTTLS on a plain connection (default true). Mutually exclusive with SMTP_USE_TLS
SMTP_USE_TLS	Connect over implicit TLS (default false). Mutually exclusive with SMTP_USE_STARTTLS
SMTP_FROM	The From address on outgoing mail (default info@cloudomation.com)
SMTP_AUTHENTICATION_MODE	One of NONE, LOGIN (default), or XOAUTH2
SMTP_USER	Username for LOGIN authentication
SMTP_PASSWORD	Password for LOGIN authentication
SMTP_OAUTH2_CLIENT_ID	OAUTH2 client id, for XOAUTH2 authentication
SMTP_OAUTH2_CLIENT_SECRET	OAUTH2 client secret, for XOAUTH2 authentication

note

For SMTP_AUTHENTICATION_MODE: XOAUTH2 (e.g. Gmail or Microsoft 365), set the client id and secret, then run ./setup.sh --smtp-oauth-authorize to complete the authorization flow. The resulting SMTP_OAUTH2_ACCESS_TOKEN and SMTP_OAUTH2_REFRESH_TOKEN are written to setup.yml for you.

External database

By default the installer provides a PostgreSQL container. To use an external database instead, run with --no-provide-database and supply the connection parameters. The workspace, auth, and vault services each have their own connection block; point them at your database and set the credentials.

Key	Description
POSTGRES_SHM_SIZE	Shared-memory size for the bundled database container (default 1gb)
WORKSPACE_POSTGRES_HOST / _PORT / _DATABASE / _USER / _PASSWORD	Connection parameters for the workspace database
AUTH_POSTGRES_HOST / _PORT / _DATABASE / _USER / _PASSWORD	Connection parameters for the auth database
VAULT_POSTGRES_HOST / _PORT / _DATABASE / _USER / _PASSWORD	Connection parameters for the Vault database

Resource limits and container hardening

These bound per-container resource use and reduce the container attack surface. The defaults are safe for most installs; raise the limits on a busy host, or tighten them further on a regulated one.

Key	Description
WORKER_COUNT	See Workers — the primary lever for total resource use, since each worker is a workspace+sandbox pair
SANDBOX_PIDS_LIMIT	Maximum process count per sandbox container (default 1024); a defense against fork bombs in flow code
SANDBOX_CPUS	Optional CPU ceiling per sandbox container (e.g. 1.5). Empty means no limit
SANDBOX_MEM_LIMIT	Optional hard memory cap per sandbox container (e.g. 2g). Empty means no limit. A flow exceeding the cap is OOM-killed
SANDBOX_READ_ONLY	Mount the sandbox root filesystem read-only, with a writable /tmp (default true)
WORKSPACE_PIDS_LIMIT	Maximum process count for a workspace container (default 8192)
CONTAINER_PIDS_LIMIT	Maximum process count for the supporting service containers (default 2048)
CONTAINER_CAP_DROP_ALL	Drop all Linux capabilities from the non-root application containers (default true)
NGINX_READ_ONLY	Mount the frontend nginx root filesystem read-only (default true)

Git integration

These apply when the Git Integration exposes an external git server.

Key	Description
GIT_REPOSITORY_SAVE_PATH	In-container path where synced repositories are stored (default /data/repositories)
GIT_REPOSITORY_EXTERNAL_SAVE_PATH	Path under which repositories are exposed to the external git server (default /git)
GIT_EXTERNAL_SERVER_NAME	Hostname of the external git server
GIT_EXTERNAL_SSH_PORT	SSH port of the external git server (default 2022)

HashiCorp Vault

The Vault deployment and its auto-unseal keys are covered under HashiCorp Vault above. The related setup.yml keys are VAULT_AUTO_UNSEAL, VAULT_AUTO_UNSEAL_KEYS, VAULT_AUTO_UNSEAL_SHARES, VAULT_AUTO_UNSEAL_DELAY_SECONDS_SUCCESS (default 600), VAULT_AUTO_UNSEAL_DELAY_SECONDS_ERROR (default 10), VAULT_PORT (default 443), and VAULT_SUBDOMAIN (default vault).