embedbase-paul-graham

This is an example of how to use Embedbase to build a search engine in a few lines of code for Paul Graham's essays through Apple's Siri shortcut.

This repository goes in pair with this blog post.

Demo

If you just want to try the end result, you can use the sandboxed version of the index with this Apple Siri Shortcut:

How.to.implement.Natural.language.search.with.Embedbase.mp4

Or in the terminal:

git clone https://github.com/another-ai/embedbase-paul-graham
cd embedbase-paul-graham
npm i
npm run playground https://embedbase-paul-graham-c6txy76x2q-uc.a.run.app

Quickstart

git clone https://github.com/another-ai/embedbase
cd embedbase

# embedbase/config.yaml
# https://app.pinecone.io/
pinecone_index: "my index name"
# replace this with your environment
pinecone_environment: "us-east1-gcp"
pinecone_api_key: ""

# https://platform.openai.com/account/api-keys
openai_api_key: "sk-xxxxxxx"
# https://platform.openai.com/account/org-settings
openai_organization: "org-xxxxx"

docker-compose up

In another terminal:

git clone https://github.com/another-ai/embedbase-paul-graham
cd embedbase-paul-graham
npm i
npm start

Stack

• Embedbase
• Typescript
• Crawlee + Playwright crawler
• Google Cloud Run for deployment
• Apple Siri Shortcuts for querying the index

Deploy

Cloud Run deployment

Don’t want to handle infra? We’re launching a hosted version soon. Just click here to be first to know when it comes out.

Setup

# login to gcloud
gcloud auth login

PROJECT_ID=$(gcloud config get-value project)

# Enable container registry
gcloud services enable containerregistry.googleapis.com

# Enable Cloud Run
gcloud services enable run.googleapis.com

# Enable Secret Manager
gcloud services enable secretmanager.googleapis.com

# create a secret for the config
gcloud secrets create EMBEDBASE_PAUL_GRAHAM --replication-policy=automatic

# add a secret version based on your yaml config
gcloud secrets versions add EMBEDBASE_PAUL_GRAHAM --data-file=config.yaml

IMAGE_URL="gcr.io/${PROJECT_ID}/embedbase-paul-graham:0.0.1"

docker buildx build . --platform linux/amd64 -t ${IMAGE_URL} -f ./search/Dockerfile

docker push ${IMAGE_URL}

gcloud run deploy embedbase-paul-graham \
  --image ${IMAGE_URL} \
  --region us-central1 \
  --allow-unauthenticated \
  --set-secrets /secrets/config.yaml=EMBEDBASE_PAUL_GRAHAM:1

# getting cloud run url
gcloud run services list --platform managed --region us-central1 --format="value(status.url)" --filter="metadata.name=embedbase-paul-graham"