Skip to content

table_transformers support for "template header" or "template variables" #1179

Description

@thorfi

Is your feature request related to a problem? Please describe.

We are generating many template column transformers, in a number of these we re-use data constants or block code. It would be helpful to have shared header blocks that get applied in template blocks.

Describe the solution you'd like

Something like this:

transformations:
  table_transformer_template_header: |-
    {{- $global_sequence_one := (list 1 2 3 4 5 4 3 2 1) -}}
    {{- $global_sequence_two := (list 6 4 2 0) -}}

  table_transformers:
    - schema: public
      table: logs
      column_transfomer_template_header: |-
        {{- $table_sequence_one := (list "a" "b" "c" "x" "y" "z") -}}
        {{- $table_sequence_two  := (list "q" "r" "s") -}}
      column_transformers:
        log_message:
          name: template
          parameters:
            template:
              {{- index $table_sequence_one (index $global_sequence_one 2) -}}

Meaning that the effective template code would be:

      column_transformers:
        log_message:
          name: template
          parameters:
            template: |-
              {{- $global_sequence_one := (list 1 2 3 4 5 4 3 2 1) -}}
              {{- $global_sequence_two := (list 6 4 2 0) -}}
              {{- $table_sequence_one := (list "a" "b" "c" "x" "y" "z") -}}
              {{- $table_sequence_two  := (list "q" "r" "s") -}}
              {{- index $table_sequence_one (index $global_sequence_one 2) -}}

Describe alternatives you've considered

Currently I just generate template code and dump duplicate blocks everywhere in the config file...

Alternatively (or additionally), a config section that sets variables would be good

transformations:
  table_transformer_template_variables:
    global_sequence_one: [ 1, 2, 3, 4, 5, 4, 3, 2, 1 ]
    global_sequence_two:
       - 6
       - 4
       - 2
       - 0
  table_transformers:
    - schema: public
      table: logs
      column_transfomer_template_variables:
        table_sequence_one: [ "a", "b", "c", "x", "y", "z"]
        table_sequence_two:
          - "q"
          - "r"
          - "s"
      column_transformers:
        log_message:
          name: template
          parameters:
            template:
              {{- index $table_sequence_one (index $global_sequence_one 2) -}}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions