Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Accept None as a default in env_var #10485

Open
2 tasks done
dschneiderch opened this issue Jul 25, 2024 · 3 comments · May be fixed by #10629
Open
2 tasks done

[Feature] Accept None as a default in env_var #10485

dschneiderch opened this issue Jul 25, 2024 · 3 comments · May be fixed by #10629
Assignees
Labels
enhancement New feature or request triage

Comments

@dschneiderch
Copy link

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

env_var does not recognize None or none as a default.
If the environment variable is not defined I see Parsing Error Env var required but not provided: '{ENV_VAR}'

I also tried env_var('NAMESPACE',none) | as_native and env_var('NAMESPACE','none') | as_native and env_var('NAMESPACE','')|as_native

Expected Behavior

{% macro namespaced(prefix)  -%}

    {%- set _namespace = env_var('NAMESPACE',none) %}
    {%- if _namespace is none %}

        {{ prefix }}

    {%- else -%}

        {{ prefix }}_{{ _namespace | trim }}

    {%- endif -%}

{%- endmacro %}

In the macro above I would like to set _namespace to none and check if _namespace is none.
I have another macro that uses var() and this works as expected.

Steps To Reproduce

Set up a macro that access an env_var that does not exist then call that macro from a model, for example as part of {{ config(alias=namespaced("MY_TABLE")) }}

Relevant log output

No response

Environment

- OS: macos 14.5 (intel)
- Python: Python 3.10.14
- dbt --version                            
Core:
  - installed: 1.8.4
  - latest:    1.8.4 - Up to date!

Plugins:
  - bigquery: 1.8.2 - Up to date!

Which database adapter are you using with dbt?

bigquery

Additional Context

No response

@dschneiderch dschneiderch added bug Something isn't working triage labels Jul 25, 2024
@dbeatty10 dbeatty10 self-assigned this Jul 25, 2024
@dbeatty10
Copy link
Contributor

Thanks for reaching out @dschneiderch !

I see what you are saying about var("something", none) and env_var("something", none) behaving differently when the relevant (environment) variable is undefined and how that is surprising and doesn't work for your use-case.

In the case of env_var, it looks like it was intentionally coded to have the behavior you observed here and here.

So I'm going to change this to a feature request for further consideration.

@dbeatty10 dbeatty10 added enhancement New feature or request and removed bug Something isn't working labels Jul 25, 2024
@dbeatty10 dbeatty10 changed the title [Bug] env_var doesn't accept None as a default [Feature] Accept None as a default in env_var Jul 25, 2024
@kentkr kentkr linked a pull request Aug 29, 2024 that will close this issue
5 tasks
@kentkr
Copy link
Contributor

kentkr commented Sep 5, 2024

@dbeatty10 do you have an inkling as to why this behavior might be intentional?

It's logical to think a user might not know an env var needs to be set. But the solution in #10629 can delineate between an env var that is not set vs set to None.

As for my use case this would be beneficial. We created a custom ref macro that handles custom schema names depending on the env. For dev the env var is set to None.

Our solution was to account for different types of none (none, 'none', 'None', '', ...) in the macro. Pretty simple but not desired.

Thanks!

@dbeatty10
Copy link
Contributor

Thank you for that context -- helpful 🧠

@dbeatty10 do you have an inkling as to why this behavior might be intentional?

I don't know why it was intentional!

So we are in a Chesterton's fence scenario that will take some more research and conversations to learn more. But this isn't very high priority for us right now, so we haven't been able to dedicate the time towards it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request triage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants