Create data lineage from Trino/Hive queries in digdag log with Python
What’s data lineage? Data lineage is something to describe “Where this data comes from and where it goes?” I learned this term in my previous job. They provided “Cloudera …
3 configs add recommend articles into your Hugo blog by GitHub Actions
Hugo has a feature to show keyword based related articles. Yeah, keyword based articles might be useful, for people who can manage keyword, category, etc, constantly. I’d love to …
py> operator development guide for Python users
This article show how to develop a digdag Python workflow task efficiently.
•
6 min read
How to release Python package from GitHub Actions
Recently, I changed my CI from Travis to GitHub Actions. GitHub Actions is handy and useful for testing, publishing Python packages.
How to test a new Docker image for digdag workflow on CircleCI?
Testing workflow runnability would be important when we build a complex workflow. digdag is a workflow engine which syntax is simple and…
The first conference of Operational Machine Learning: OpML ‘19
I attended OpML ’19 is a conference for “Operational Machine Learning” held at Santa Clara on May 20th.
Ruby for Data Science and Machine Learning
I attended RubyKaigi 2019 held at Fukuoka from Apr 18 to Apr 21. This year’s RubyKaigi was a really great opportunity for me to know the…
A recent update of tabula-py
This article is a repost of Patreon article published last December. I’m planning to bump up next version of tabula-py within few weeks.
Use Markdown document on brand new PyPI
Yesterday, PyPI was renewed to the next-generation site. It is modern and stylish one.
Python basics: package management
Python is a very famous programming language for machine learning. In this article, I will introduce basic Python environment.