feature: To be able to define custom metrics endpoint

[bobanj]

Feature request

Would like to be able to provide something like a prefix_path in the configuration so the metrics, healthz endpoints are
available with my/prefix/path

Motivation

Scenario where multiple models are available under one hostname, ex:

• my_host/model_1/endpoint
• my_host/model_2/endpoint
  Would like to be able to access metrics and healthz endpoints without the need to define custom middleware or define additional healthz action (currently in code as far as can be seen, the comparisons are hardcoded ex: == "/metrics". Would like to be able to setup the accessability for these endpoints by defining somehting like prefix_path`:
• my_host/model_1/metrics
• my_host/model_1/healthz
• my_host/model_2/metrics
• my_host/model_2/healthz

Other

No response

[ssheng]

Are the different models served by one or multiple BentoML servers? If multiple servers, shouldn't they be running on different ports? If the same server, the models should be sharing the same health check path, too. Is my understanding correct?

[bobanj]

We are using a diff infra setup (no EKS), hence the reason for asking for this.

[ssheng]

@frostming Do you think the health check and metrics prefix is best exposed as a configuration field?

[frostming]

Sorry I didn't get the point. The OP seems to ask for multiple metrics endpoints instead of custom metrics endpoint, but the former doesn't make sense to me.

[bobanj]

Not to have multiple metrics endpoints but for the current /metrics path configurable, basically to have something like

BentoML/src/bentoml/_internal/server/http/instruments.py

Line 87 in 0666d62

if scope["path"] == "/metrics":

if scope["path"] == metrics_path_from_config || "/metrics"