v0.5 — Adapter-specific dependencies, hooks with arguments, and --full-refresh support

v0.5 — Adapter-specific dependencies, hooks with arguments, and --full-refresh support
Options to choose from – Photo by Tina Floersch on Unsplash

With the 0.5 release,  we make the fal package lighter by installing only the dependencies needed for your adapter,

tl;dr: change pip install fal for pip install fal[<adapter>] where <adapter> is your preferred adapter used in pip install dbt-<adapter>.

Adapter-specific dependencies (#504)

Until now, fal installed dependencies for all adapters that we support. Starting with v0.5, fal installs only the dependencies you actually need to run your scripts. This is done with pip install fal[<adapter>], an example would be pip install fal[snowflake].

We realized that the size of our dependencies are getting bigger as we add specific improvements to each adapter. We wanted to modularize these for a lighter package footprint. Here are the adapters that we support so far:

P.S. Not seeing your favorite adapter here? Open a GitHub issue or contact us in Discord!

Hooks with arguments (#487)

There are some exciting development happening around the fal pre and post hooks. The first one is making it easy to pass arguments to hooks without having to parse the entire meta object.

From now on, you can set arguments for a hook in schema.yml with the with keyword:

models:
  - name: model_a
    meta:
      fal:
        pre-hook:
          - pre_hook_1.py       # this still works
          - path: pre_hook_2.py # has no arguments
          - path: add.py
            with:               # arguments to pass
              left: 2
              right: 3
schema.yml with different ways to setup a hook

You can then access the arguments in the add.py script through the context:

value = context.arguments['left'] + context.arguments['right']
print(f"That is {value}!")
add.py script

Enable --full-refresh flag for fal flow (#501)

You can now add the --full-refresh flag to your fal flow runs and fal will pass them through to dbt under the hood for you.

❯ fal flow run --full-refresh
# ...
Running command: dbt run --threads 1 --project-dir .../008_pure_python_models --full-refresh --select model_a
# ...
Running command: dbt run --threads 1 --project-dir .../008_pure_python_models --full-refresh --select model_b
# ...
Example of fal flow run --full-refresh

Other fixes and improvements

v0.5.0

Features

  • Structured hooks for passing arguments (#487)
  • Optimize write and read for snowflake (#483)
  • Implement isolated hooks (#489)
  • Offer adapter_response per model in CurrentModel class (#498)

Miscellaneous Tasks

  • Separate adapters dependencies as extras (#504)

v0.4.1

Bug Fixes

  • Make exit code of Python non-zero if there are errors (#481)
  • Filter source nodes from nodes to run (#490)

Features

  • Include before/after scripts in nodes failed/skipped report (#482)
  • Implement fal_results.json (#436, #484, #485)