Introducing fal flow

We started the fal project with a simple objective; we wanted to build a developer tool that enables running python scripts downstream of dbt models. Since we launched the initial capabilities of fal, we have received an overwhelming amount of requests to run python scripts before, after and in between different dbt runs. Last month we enabled running python scripts before a dbt run, and today we are announcing our biggest update yet.

fal flow is a new CLI command that enhances the dbt graph to include python nodes, and enables familiar dbt operations on the whole graph.

Consider the example DAG below. Triggering a run that includes all the downstream dependencies of the load_data.py script requires three different commands to be invoked separately. Determining the exact commands also involved some manual work, as we need to figure out the descendants of the origin node.

fal flow provides the same delightful developer experience as dbt, but now for the whole DAG. Running the same workflow above is a single fal flow command with familiar dbt graph selector operations.

Our new release opens the door for single command workflows that previously required several different tools. Triggering an EL run followed by a mix of dbt nodes and python scripts is now a single fal command. Complicated polyglot workflows with Python nodes in between dbt nodes are also made simple with fal flow. We are going to explore the capabilities of fal flow for the rest of our launch week. Stay tuned!