Features & Labels Blog

Notebooks as fal scripts and more

Today we're presenting more new features that our users might like: Jupyter notebooks as fal scripts, execute_sql magic function, dbt source test results.

Introducing Post Hooks

fal post hooks let you define Python scripts that don't need to write to the data warehouse.

Introducing fal Python Models

fal enables you to run Python with your dbt project. Until today, fal users could associate their dbt models with Python scripts to run before or after a model, but these scripts did not generate new assets on their own. We had recently added the ability to run these Python

Supporting Athena and DuckDB

We’re constantly working on making it as easy as possible to integrate fal into our users’ projects. So far we have supported PostgreSQL, Google BigQuery, Snowflake and AWS Redshift. Today we’re adding two more: AWS Athena and DuckDB. Star us on GitHub Amazon AthenaAmazon Athena is a query

Announcing fal Launch Week #2

We are excited to bring you our second Launch Week! Over the past couple of months we have been working on exciting new features to make running Python even more delightful. Similar to the previous launch week, each day of the week we are going to announce a new feature

How-to

Use fal to integrate SageMaker with dbt

We discuss a straightforward way to integrate your dbt project with SageMaker by using fal. We will train a SageMaker model, store model data in a dbt model, use a SageMaker model to make some test predictions and store prediction results in another dbt model.

Python or SQL? Why not both?

Python vs SQL is a frequent debate in the data community and it usually revolves around which language is better for data transformations. At Features and Labels, we think about it a little differently. In our opinion, today, all transformations that are supported by SQL should be done in SQL.

Parallelizing Python Workflows in fal

Today's release introduces parallelization of Python scripts in your fal runs. During a run, fal creates a directed acyclic graph (DAG) of links between models and scripts. The number of threads represents the maximum number of tasks fal handles simultaneously. When fal is running, it will read the configuration for

How-to

Run Airbyte and Fivetran from your dbt project using fal

You can now use fal to easily specify and trigger sync jobs on pipelines from both Fivetran and Airbyte.

Introducing fal flow

We started the fal project with a simple objective; we wanted to build a developer tool that enables running python scripts downstream of dbt models. Since we launched the initial capabilities of fal, we have received an overwhelming amount of requests to run python scripts before, after and in between

Announcing fal Launch Week #1

After our Unbundling of Airflow article, it has been a wild few weeks here at Features & Labels. Gorkem’s now-viral post resonated with tens of thousands of data people around the globe. Meanwhile, we have been hard at work cooking up the future of our open-source library fal, which

How-to

Going from dbt models to pandas DataFrames

One of our favorite features of dbt is the ability to build dependencies between data models using the ref function. With ref, you can build pipelines all in SQL, simply by referring to other tables in your select statements. But what if you want to refer to these models in

How-to

Populate dbt models with CSV data

A common source of raw data in ELT pipelines are CSV files. These text files hold data in multiple lines with headers and need to be parsed and loaded into data warehouses, ideally in an automated process. One way to load CSV data is by using the dbt seed command.

What’s new in fal 0.2.0

We’re constantly working on improving fal for our users by looking into issues and feature requests. You can create a Github issue, if you also would like something in fal to be improved. Today we are releasing a new version of fal with a set of features that add

The Unbundling of Airflow

Star us on GitHub In each subdomain of software, we have seen the same story over and over again. Products start small, in time, add adjacent verticals and functionality to their offerings and end up becoming a platform. Once these platforms become big enough, people start to figure out how

How-to

How to run Python with dbt Cloud using Github Actions and dbt Cloud API

dbt is our favorite tool to build data pipelines. It allows us to skip boilerplate data engineering code, focus only on SQL and helps us build with software engineering practices like reusability, auto generating docs and tests. We recently introduced our open source project fal which let us run Python

How-to

How to run sentiment analysis on your dbt models from Python

Sentiment analysis is the practice of determining whether a text / statement is positive, negative or neutral typically using natural language processing (NLP). Applying sentiment analysis on data such as reviews, tickets, feedback and survey responses can help you understand how your organization is doing in the eyes of your customers.

How-to

How to detect data anomalies in your dbt models

For time series data, anomalies are almost always present in some way or another. Be it a sudden change in rainfall for a town due to climate change or, in our example here, a sudden upwards trend in daily Covid-19 cases. Any system or business that runs on time series

How-to

How to reverse ETL a dbt model into Google Firestore

Google Firestore is a cloud-hosted NoSQL database that can be directly accessed by mobile and web applications through native SDKs. It has a rich set of use cases, including keeping track of inventories, user sessions and state mutations. Firestore is fast and enable low-latency querying of large quantities of data.

How to run forecasts on dbt models

Transforming and validating data is very easy with dbt. We can build complex models that span multiple dependencies. We can also create macros that can work across a dbt project and make data pipelines easy to maintain. Sometimes we want to do more with our dbt models, like for example

How-to

How to integrate dbt with Slack

Using Slack bots can be a great time saver. They can enhance whatever workflow they are a part of, be it project management, version control or simple content sharing. As we bring our dbt projects to production, Slack bots can help us stay on top of dbt runs and models.

Introducing fal: do more with dbt

At Features & Labels, dbt is one of our favorite tools to use while managing our data pipelines. It is a category defining tool (see Analytics Engineering) that enables us to build data transformations without having to write boilerplate data engineering code + a ton of other benefits that are too