At Features & Labels, dbt is one of our favorite tools to use while managing our data pipelines. It is a category defining tool (see Analytics Engineering) that enables us to build data transformations without having to write boilerplate data engineering code + a ton of other benefits that are too long to list here. (For a comprehensive overview, go here)
dbt's SQL only design is powerful, but whenever we wanted to get out of SQL-land and connect to external services or get into Python-land for any reason, we had a hard time doing so without having to resort to a heavier weight orchestration tool such as Airflow.
Today, we wanted to share with you an open-source library that we have been working on called fal, which we built to enable Python workloads (sending alerts to Slack, building predictive models, pushing data to non-data warehouse destinations and more) right within dbt in a simple way.
Some examples of what you can do with fal:
- download dbt models using a familiar
refsyntax as pandas dataframes
- send slack messages on model runs
- do forecasts on metrics
- do sentiment analysis on support tickets
Here is a video of it in action.
For a full documentation of what is possible, check out our README.
We hope you find fal useful in your day to day data work. We will be posting many how-to guides over the next days.
Don't forget to give us a ⭐ on Github:Star fal on GitHub
You can also join us here on our Discord community to give us feedback or ask us any questions.