Skip to content

Commit

Permalink
For #1
Browse files Browse the repository at this point in the history
  • Loading branch information
mugitty committed Sep 11, 2024
1 parent 5cd4b73 commit 70ef2d3
Show file tree
Hide file tree
Showing 3 changed files with 51 additions and 3 deletions.
50 changes: 47 additions & 3 deletions README.graphstore.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@

Some of this is a repeat of the info found in https://github.com/geneontology/devops-documentation/blob/main/README.setup.md, but with an eye to working from `go-graphstore`. Following this doc from the start is recommended if you are unfamiliar with the process.

Everything is going to be run from a docker image. This process uses prebuilt image 'geneontology/go-devops-base'. This can be inspected via

```docker inspect geneontology/go-devops-base:tools-jammy-0.4.2
```

```
docker rm go-dev
docker run --name go-dev -it geneontology/go-devops-base:tools-jammy-0.4.2 /bin/bash
Expand All @@ -19,28 +24,62 @@ git clone https://github.com/geneontology/go-graphstore.git
cd go-graphstore/provision
```

## Setup AWS credentials and test access
```
emacs /tmp/go-aws-credentials
```
Template of file is
```
[default]
aws_access_key_id = REPLACE_ME_1
aws_secret_access_key = REPLACE_ME_2
```
Replace REPLACE_ME_1 and REPLACE_ME_2 with your access and keys

Set as environment variables
```
export AWS_SHARED_CREDENTIALS_FILE=/tmp/go-aws-credentials
```

Test by listing s3 buckets
```
aws s3 ls
```

Copy key files and set privileges
```
docker cp \your\key\go-ssh.pub go-dev:/tmp
docker cp \your\key\go-ssh go-dev:/tmp
cd /tmp/
chmod 600 go-ssh*
cd go-graphstore/provision
```

### Terraform backend

```
cp ./production/backend.tf.sample ./aws/backend.tf
emacs ./aws/backend.tf
```

- `bucket = "REPLACE_ME_TERRAFORM_S3_STATE_STORE"` should be "go-workspace-graphstore"
- `bucket = "REPLACE_ME"` should be "go-workspace-graphstore"

Setup AWS backend with:

```
go-deploy -init --working-directory aws -verbose
```

Scripts are in https://github.com/geneontology/devops-deployment-scripts

Test with:

See https://github.com/geneontology/devops-documentation/blob/main/README.graphstore.md#test .

_At this point, we are now setup to perform basic listing and destructive operations._ See: https://github.com/geneontology/devops-documentation/blob/main/README.graphstore.md#destroy-previous-instances .

## Setup instance
This sets up a server instance

### config-instance.yaml

Expand Down Expand Up @@ -70,13 +109,14 @@ Commands to deploy instance.
Replace `YYYY-MM-DD` appropriately.

#### For production
This creates Terraform instructions with name "graphstore-production-YYYY-MM-DD". Note, the label for the instructions and the instance are the same

```
go-deploy --workspace production-YYYY-MM-DD --working-directory aws -verbose --conf config-instance.yaml
```

#### For "internal"

This creates Terraform instructions with name "graphstore-internal-YYYY-MM-DD". Note, the label for the instructions and the instance are the same
```
go-deploy --workspace internal-YYYY-MM-DD --working-directory aws -verbose --conf config-instance.yaml
```
Expand All @@ -91,6 +131,7 @@ For example:

```
ssh -i /home/sjcarbon/local/share/secrets/go/ssh-keys/go-ssh [email protected]
logout
```

Workspace placement:
Expand Down Expand Up @@ -155,10 +196,13 @@ emacs ./vars.yaml
- `S3_PREFIX: REPLACE_ME` should be "production-2024-04-24".
- `S3_CRED_FILE: REPLACE_ME` should be "/tmp/go-aws-credentials".
- `S3_BUCKET: REPLACE_ME` should be "go-service-logs-graphstore-production".
- `remote_journal_gzip: http://current.geneontology.org/products/blazegraph/blazegraph-internal.jnl.gz` should be "http://current.geneontology.org/products/blazegraph/blazegraph-production.jnl.gz"

- `GRAPHSTORE_SERVER_NAME: graphstore.example.com` should be "rdf.geneontology.org".
- `GRAPHSTORE_SERVER_ALIAS: REPLACE_ME` should be "graphstore-production-2024-04-24.geneontology.org".

- `remote_journal_gzip: http://current.geneontology.org/products/blazegraph/blazegraph-internal.jnl.gz` should be "http://current.geneontology.org/products/blazegraph/blazegraph-production.jnl.gz"


### Internal variables

```
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# devops-documentation

General and specific documentation for geneontology DevOps.
Intent is to use Software as Infastructure via Terraform and Ansible. This requires crediantials and permission to access and provision resources.

## README.setup

Expand Down
3 changes: 3 additions & 0 deletions README.setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ docker run --name go-dev -it geneontology/go-devops-base:tools-jammy-0.4.1 /bin
- go-deploy (multple install methods: poetry `poetry install go-deploy==0.4.2` (requires python >=3.8), can also be installed incidentally from go-fastapi repo with `poetry install`)

### Credentials
Two sets of keys are required:
1. ssh keys to deploy and or view aws instances
2. aws keys for Terraform to provision aws resources

#### SSH Keys

Expand Down

0 comments on commit 70ef2d3

Please sign in to comment.