DM-43555: Add phalanx environment install command

.github/workflows/ci.yaml

@@ -71,10 +71,10 @@ jobs:
           cache-key-prefix: helm
 
   # The minikube job always runs, but it quickly does nothing if no files that
-  # would affect minikube were changed.  This unfortunately requires a lot of
+  # would affect minikube were changed. This unfortunately requires a lot of
   # if conditionals on all the steps of the job, but we need the job to run so
   # that we can make it mandatory before merging, which in turn allows us to
-  # use automerge.
+  # use merge queues.
   minikube:
     name: Test deploy
     runs-on: ubuntu-latest
@@ -99,7 +99,18 @@ jobs:
               - "environments/Chart.yaml"
               - "environments/templates/applications/infrastructure/*"
               - "environments/values-minikube.yaml"
-              - "installer/**"
+              - "src/phalanx/**"
+
+      - name: Download installer dependencies
+        if: steps.filter.outputs.minikube == 'true'
+        run: |
+          curl -sSL -o /tmp/vault.zip https://releases.hashicorp.com/vault/1.15.4/vault_1.15.4_linux_amd64.zip
+          unzip /tmp/vault.zip
+          sudo mv vault /usr/local/bin/vault
+          sudo chmod +x /usr/local/bin/vault
+          sudo curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/download/v2.8.6/argocd-linux-amd64
+          sudo chmod +x /usr/local/bin/argocd
+          sudo apt-get install socat
 
       - name: Setup Minikube
         if: steps.filter.outputs.minikube == 'true'
@@ -113,23 +124,18 @@ jobs:
         if: steps.filter.outputs.minikube == 'true'
         run: kubectl get nodes
 
-      - name: Download installer dependencies
+      - uses: lsst-sqre/run-tox@v1
         if: steps.filter.outputs.minikube == 'true'
-        run: |
-          curl -sSL -o /tmp/vault.zip https://releases.hashicorp.com/vault/1.15.4/vault_1.15.4_linux_amd64.zip
-          unzip /tmp/vault.zip
-          sudo mv vault /usr/local/bin/vault
-          sudo chmod +x /usr/local/bin/vault
-          sudo curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/download/v2.8.6/argocd-linux-amd64
-          sudo chmod +x /usr/local/bin/argocd
-          sudo apt-get install socat
-
-      - name: Run installer
         timeout-minutes: 15
-        if: steps.filter.outputs.minikube == 'true'
-        run: |
-          cd installer
-          ./install.sh minikube "${{ secrets.MINIKUBE_VAULT_ROLE_ID }}" "${{ secrets.MINIKUBE_VAULT_SECRET_ID }}"
+        with:
+          python-version: "3.12"
+          tox-envs: install
+          tox-posargs: >-
+            --force-noninteractive
+            --vault-role-id=${{ secrets.MINIKUBE_VAULT_ROLE_ID }}
+            --vault-secret-id=${{ secrets.MINIKUBE_VAULT_SECRET_ID }}
+            minikube
+          cache-key-prefix: test
 
       - name: Get final list of resources
         if: steps.filter.outputs.minikube == 'true'

.gitignore

@@ -1,5 +1,3 @@
-/installer/secrets/
-/installer/docker-creds
 /applications/*/charts/*.tgz
 /applications-expanded/
 **/Chart.lock

README.md

@@ -1,7 +1,7 @@
 # Phalanx
 
 This is the Argo CD repository for the Rubin Science Platform.
-It stores the root Argo CD application, deployment configuration for the other applications, the installer, and other helper scripts.
+It stores the root Argo CD application, deployment configuration for the other applications, and a command-line tool to manage Phalanx environments.
 
 See [phalanx.lsst.io](https://phalanx.lsst.io/) for full documentation.
 

applications/argocd/secrets.yaml

@@ -2,9 +2,9 @@
   description: >-
     Admin password for Argo CD. This password is normally not used because
     Argo CD is configured to use Keycloak, Google, or GitHub authentication,
-    but it is used by the installer (which cannot use external authentication)
-    and is useful as a fallback if external authentication is not working for
-    some reason. This secret can be changed at any time.
+    but it is used during installation and is useful as a fallback if external
+    authentication is not working for some reason. This secret can be changed
+    at any time.
   generate:
     type: password
 "admin.password":

docs/about/repository.rst

@@ -55,17 +55,6 @@ Each environment then has a file named :file:`values-{environment}.yaml` that de
 
 The templates directory also contains the Argo CD ``AppProject`` resources, which are used to classify the applications into groups for access control.
 
-installer directory
--------------------
-
-:bdg-link-primary-line:`Browse installer/ on GitHub <https://github.com/lsst-sqre/phalanx/tree/main/installer>`
-
-This directory contains a script named `install.sh <https://github.com/lsst-sqre/phalanx/blob/main/installer/install.sh>`__.
-The arguments to this are the name of the environment, the Vault RoleID, and the Vault SecretID (see :ref:`secrets` for more details on Vault).
-This installer script is the entry point for setting up a new environment.
-It can also be run on an existing environment to update it.
-See the :ref:`environment bootstrapping documentation <bootstrapping-toc>` for details.
-
 charts directory
 ----------------
 
@@ -136,7 +125,7 @@ In one check, Pre-commit regenerates Helm chart documentation for applications w
 See the `.pre-commit-config.yaml <https://github.com/lsst-sqre/phalanx/blob/main/.pre-commit-config.yaml>`__ file for configuration details.
 Learn how to set up Pre-commit in your local editing environment in :doc:`local-environment-setup`.
 
-Second, GitHub Actions runs a CI workflow (`.github/workflows/ci.yaml <https://github.com/lsst-sqre/phalanx/blob/main/installer/install.sh>`__).
+Second, GitHub Actions runs a CI workflow (`.github/workflows/ci.yaml <https://github.com/lsst-sqre/phalanx/blob/main/.github/workflows/ci.yaml>`__).
 This workflow has four key jobs:
 
 - Linting with Pre-commit_, mirroring the local editing environment.

docs/admin/create-environment.rst

@@ -0,0 +1,55 @@
+################################
+Create a new Phalanx environment
+################################
+
+Each separate installation of Phalanx is called an environment.
+An environment has a hostname, Vault server and path to its secrets, and a set of Phalanx applications that should be installed in that environment.
+
+Each Phalanx environment must be installed in a separate Kubernetes cluster.
+Two Phalanx environments cannot coexist in the same cluster.
+
+Before starting this process, ensure that you have met the :doc:`requirements to run Phalanx <requirements>` and that you have decided on your :doc:`handling of hostnames and TLS <hostnames>`.
+
+Creating an environment
+=======================
+
+To create a new Phalanx environment, take the following steps:
+
+.. rst-class:: open
+
+#. Fork the `Phalanx repository`_ if this work is separate from the SQuaRE-managed environments.
+
+#. Create a new :file:`values-{environment}.yaml` file in `environments <https://github.com/lsst-sqre/phalanx/tree/main/environments/>`__.
+   Start with a template copied from an existing environment that's similar to the new environment.
+   Edit it so that ``name``, ``fqdn``, ``vaultUrl``, and ``vaultPathPrefix`` at the top match your new environment.
+   You may omit ``vaultUrl`` for SQuaRE-managed environments.
+   See :doc:`secrets-setup` for more information about the latter two settings and additional settings you may need.
+   If the environment will be hosted on Google Kubernetes Engine, also fill out ``gcp.projectId``, ``gcp.region``, and ``gcp.clusterName`` with metadata about where the environment will be hosted.
+   Enable the applications this environment should include.
+
+#. Do what DNS setup you can.
+   If you already know the IP address where your instance will reside, create the DNS records (A or possibly CNAME) for that instance.
+   If you are using a cloud provider or something like minikube where the IP address is not yet known, then you will need to create that record once the top-level ingress is created and has an external IP address.
+
+#. Decide on your approach to user home directory storage.
+   The Notebook Aspect (the ``nublado`` application) requires a POSIX file system.
+   The most frequently used method of providing that file system is NFS mounts, but you may instead want to use persistent volume claims or a different file system that's mounted on the Kubernetes cluster nodes and exposed to pods via ``hostPath``.
+   Whatever storage you choose, you will need to configure appropriate mount points in :px-app:`nublado` when you configure each application in the next step.
+
+#. For each enabled application, create a corresponding :file:`values-{environment}.yaml` file in the relevant directory under `applications <https://github.com/lsst-sqre/phalanx/tree/main/applications/>`__.
+   Customization will vary from application to application.
+   The following applications have special bootstrapping considerations:
+
+   - :px-app-bootstrap:`argocd`
+   - :px-app-bootstrap:`gafaelfawr`
+   - :px-app-bootstrap:`nublado`
+   - :px-app-bootstrap:`portal`
+   - :px-app-bootstrap:`squareone`
+
+#. Add the URL of your new environment to :file:`docs/documenteer.toml` under ``phinx.linkcheck.ignore``.
+   The Argo CD URL of your environment will be unreachable, so you need to tell Sphinx valid link checking to ignore it.
+
+Next steps
+==========
+
+- Define the secrets for your new environment and store them in Vault: :doc:`secrets-setup`

docs/admin/hostnames.rst

@@ -24,7 +24,13 @@ To use the first approach, you must have the following:
 
 * An :abbr:`AWS (Amazon Web Services)` account in which you can create two Route 53 hosted domains.
   You must use this domain for the hostname of the Science Platform installation.
+
 * The ability to delegate to that Route 53 hosted domain from some public DNS domain.
   This means either registering a domain via Amazon, registering a domain elsewhere and pointing it to Amazon's Route 53 DNS servers, or creating a subdomain of an existing public domain by adding ``NS`` records to that domain for a subdomain hosted on Route 53.
 
 If neither of those requirements sound familiar, you almost certainly want to use the second option and purchase a commercial certificate.
+
+Next steps
+==========
+
+- Define the configuration for your new Phalanx environment: :doc:`create-environment`

docs/admin/index.rst

@@ -17,6 +17,7 @@ Administrators operate infrastructure, manage secrets, and are involved in the d
 
    requirements
    hostnames
+   create-environment
    secrets-setup
    installation