0.4.4 - More reliable qapairs

What's Changed

• disable one qapair prompt that was unreliable
• fix race in subscriber creation by @rusenask in #141

Full Changelog: 0.4.3...0.4.4

0.4.3 - Disable some of the harder qapair prompts

These were causing timeouts

0.4.2 - Give the finetune bot a name

Tell Helix that its name is Helix.

Full Changelog: 0.4.1...0.4.2

0.4.1 - System prompt, retries & bugfixes

What's Changed

• Fix "ignore errors" button by @binocarlos in #132
• Automatic progression to finetuning after qapairs step by @binocarlos in #133
• Fix cloning fetaure: bug where the top level session lora_dir was not being copied by @binocarlos in #134
• The unstructured downloader will now report back with any upstream errors downloading files with better error messages for the user by @binocarlos in #136
• If there's an error in dataprep, don't automatically progress to finetuning by @binocarlos in #138
• Add system prompt for text finetuning improving fine tuning performance and self-awareness, and add retries by @lukemarsden in #139

Full Changelog: 0.4.0...0.4.1

0.4.0 - Mixtral for qapairs, qapair prompt suite

What's Changed

• Fix finetuning uploads by @rusenask in #125
• Finetuning improvements by @lukemarsden in #129

Finetuning improvements

Helix now uses Mixtral on Together.ai instead of GPT-4 on OpenAI to generate the question-answer pairs in the dataprep stage of text finetuning. It also now uses a suite of qapair generation prompts, defined in qapairs_config.yaml. This is both a step towards fully on-prem qapair generation (we can run Mixtral locally) and results in better quality fine text tuning.

Upgrade notes

Be sure to add TOGETHER_API_KEY to an API key from together.ai instead of the OpenAI key in .env.

Full Changelog: 0.3.8...0.4.0

0.3.8 - GPU selection support on multi-GPU systems

GPU selection

You can now specify which GPU you want the runner to target by specifying CUDA_VISIBLE_DEVICES in the environment for your runner.

Example, to use the second GPU in the system, set CUDA_VISIBLE_DEVICES=1 :

sudo docker run --privileged --gpus all --shm-size=10g \
    --restart=always -d \
    --name helix-runner --ipc=host --ulimit memlock=-1 \
    --ulimit stack=67108864 \
    -e CUDA_VISIBLE_DEVICES=1 \
    -v ${HOME}/.cache/huggingface:/root/.cache/huggingface \
    europe-docker.pkg.dev/helixml/helix/runner:0.3.8 \
    --api-host <http(s)://YOUR_CONTROLPLANE_HOSTNAME> --api-token <RUNNER_TOKEN_FROM_ENV> \
    --runner-id $(hostname) \
    --memory 40GB \
    --allow-multiple-copies --timeout-seconds 120

You can still only run the runner against a single GPU at a time but now you can select which one it is using. If you want to use multiple GPUs in a system, you can run multiple runner containers (just give them different names).

Automatic exit on coredump

We've observed GPU runners occasionally segfault with an "Illegal instruction" message. When this happens, it seems to be sufficient simply to restart the container. We've made the runner automatically exit when it detects this condition. This should make the system more reliable. Make sure to run your container with --restart always or equivalent, depending on your container runtime.

0.3.1

What's Changed

• build unstructured image by @rusenask in #114

Full Changelog: 0.3.0...0.3.1

0.3.0 - OpenAI compatible API

What's Changed

• OpenAI API compatibility with /v1/chat/completions endpoint. Both streaming and blocking API modes supported. Use model mistralai/Mistral-7B-Instruct-v0.1,get your API key from https://app.tryhelix.ai/account, example:

{
	"model": "mistralai/Mistral-7B-Instruct-v0.1",
	"stream": true,
	"messages": [
		{
			"role": "system",
			"content": "You are a helpful assistant."
		},
		{
			"role": "user",
			"content": "tell me about oceans!"
		}
	]
}

• Simplified self-hosted Docker Compose setup, now API server routes to both keycloak and frontend containers.