Skip to content

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

License

Notifications You must be signed in to change notification settings

vbedida79/GenAIExamples

 
 

Repository files navigation

Generative AI Examples

version license


Introduction

GenAIComps-based Generative AI examples offer streamlined deployment, testing, and scalability. All examples are fully compatible with Docker and Kubernetes, supporting a wide range of hardware platforms such as Gaudi, Xeon, and other hardwares.

Architecture

GenAIComps is a service-based tool that includes microservice components such as llm, embedding, reranking, and so on. Using these components, various examples in GenAIExample can be constructed, including ChatQnA, DocSum, etc.

GenAIInfra, part of the OPEA containerization and cloud-native suite, enables quick and efficient deployment of GenAIExamples in the cloud.

GenAIEval measures service performance metrics such as throughput, latency, and accuracy for GenAIExamples. This feature helps users compare performance across various hardware configurations easily.

Getting Started

GenAIExamples offers flexible deployment options that cater to different user needs, enabling efficient use and deployment in various environments. Here’s a brief overview of the three primary methods: Python startup, Docker Compose, and Kubernetes.

  1. Docker Compose: Check the released docker images in docker image list for detailed information.
  2. Kubernetes: Follow the steps at K8s Install and GMC Install to setup k8s and GenAI environment .

Users can choose the most suitable approach based on ease of setup, scalability needs, and the environment in which they are operating.

Deployment

Use Cases Deployment
Docker Compose Kubernetes
Xeon Gaudi
ChatQnA Xeon Link Gaudi Link K8s Link
CodeGen Xeon Link Gaudi Link K8s Link
CodeTrans Xeon Link Gaudi Link K8s Link
DocSum Xeon Link Gaudi Link K8s Link
SearchQnA Xeon Link Gaudi Link K8s Link
FaqGen Xeon Link Gaudi Link K8s Link
Translation Xeon Link Gaudi Link K8s Link
AudioQnA Xeon Link Gaudi Link Not supported yet
VisualQnA Xeon Link Gaudi Link Not supported yet

Support Examples

Check here for detailed information of supported examples, models, hardwares, etc.

Additional Content

About

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Svelte 25.9%
  • Shell 25.5%
  • Python 20.8%
  • TypeScript 14.5%
  • JavaScript 3.1%
  • SCSS 3.0%
  • Other 7.2%