Digdag

Create data lineage from Trino/Hive queries in digdag log with Python featured image

Create data lineage from Trino/Hive queries in digdag log with Python

What’s data lineage? Data lineage is something to describe “Where this data comes from and where it goes?” I learned this term in my previous job. They provided “Cloudera …

avatar
Aki Ariga
Read more

digdaglog2sql

Extract SQLs from digdag log to visualize SQL lineage

avatar
Aki Ariga
Read more

py> operator development guide for Python users

This article show how to develop a digdag Python workflow task efficiently.

Read more

cookiecutter-digdag

A template generates digdag workflows for SQL and Python

avatar
Aki Ariga
Read more