document the usage of PREFECT_PROFILE when developing locally and deploying in production

[david-gang]

Describe the current behavior

Currently the documented values here are PREFECT_API_KEY and PREFECT_API_URL.PREFECT_PROFILE is an important variable which is not documented there.

Describe the proposed behavior

PREFECT_PROFILE should be documented as explained at this github issue
it should be explained that whe working with cloud the PREFECT_PROFILE should be cloud and all other variants

Example Use

No response

Additional context

The trigger of this issue was that the call of prefect.get_client caused in my k8s pod that a prefect server will be created because i did not set the environment variable of PREFECT_API_URL Finding the root cause was very hard. It would have been better to get an exception that the url was not set. I know that my request doesn't solve the issue because i can forget to set both PREFECT_API_URL and PREFECT_PROFILE but anyway it is important to document this.

But maybe you have an additional idea on how to make this more fail safe

[zzstoatzz]

hi @david-gang - thank you for the issue! are you using prefect 3.x or 2.x?

it would be helpful to know how you're leveraging PREFECT_PROFILE as an env var, as we are currently working on some settings upgrades

[adityakalburgi]

hi @david-gang i would like to work on this issue and fix it would you please assign this to me?..

[david-gang]

@zzstoatzz I use prefect 3.x

My trigger for this issue was a bug we had in a k8s pod. When calling the prefect.get_client, a prefect server was started. This caused a higher memory usage and ultimately caused an OOM in my pod.

This issue was due to the fact that i didn't set the PREFECT_API_URL. Analyzing this issue and understanding the whole settings and profile mechanism , the settingscontext and the root_settings_context took me time.
I am already an experienced developer but for less experienced developers it would be a frustrating experience

I want to make it easy to add prefect as the workflow orchestration in multiple products in the company i work. In our current code bases we have a mechanism to know if we run in a local or production/(staging) environment, so i want to make a python wrapper which identifies the environment and sets the PREFECT_PROFILE accordingly, so that instead that a new server will be created i will get an informative error message

raise ValueError(
            "No Prefect API URL provided. Please set PREFECT_API_URL to the address of a running Prefect server."
        )

I am interested to hear if you have additional ideas to prevent such a scenario.

[zzstoatzz]

hi @adityakalburgi - unfortunately this is likely not a "good first issue", we're in the midst of making some intentional changes to how settings work - thank you for your interest! feel free to follow along here

---

hi @david-gang - thank you for the issue! This is something we are actively working on simplifying, so your feedback here is very helpful

This issue was due to the fact that i didn't set the PREFECT_API_URL. Analyzing this issue and understanding the whole settings and profile mechanism , the settingscontext and the root_settings_context took me time.

I think we would fail as you expect if PREFECT_ALLOW_EPHEMERAL_MODE=false, as by default we assume that if you try to use a client without a PREFECT_API_URL set, that we should run an ephemeral server so your code works.

So the question for me is "why wasn't PREFECT_API_URL set in your pod?"

can you explain how things are happening in your case? i.e.

• are you using a k8s work pool
• are you overriding env on the work pool / deployment / flow run level?

my expectation is that if you're using a kubernetes work pool to run a deployment, the PREFECT_API_URL should be known by the worker, who injects that env onto the runtime pod - it seems to me like if that's not happening, that's the problem to solve

[david-gang]

@zzstoatzz I know. This was a stupid bug from my side not setting PREFECT_API_URL. But on the other hand starting the prefect server is an unexpected behavior. Just think on a theoretical application where you use postgresql and because you forgot to set the database url it starts a sqlite db which is 100% compatible with postgresql. wouldn't it be suprising?

my main issue is that i am writing the library code internally in my company and i want to make it easy for people doing errors to get an informative error message which guides them to fix the issue.

[david-gang]

@zzstoatzz I have a suggestion for an alternative solution. When you start the server in ephemeral mode instead of writing starting server, write maybe starting server in ephemeral mode. For more information read at http://