This article show how to develop a digdag Python workflow task efficiently.

Photo by Hitesh Choudhary on Unsplash Recently, I changed my CI from Travis to GitHub Actions. GitHub Actions is handy and useful for testing, publishing Python packages. Testing Python code on GitHub Actions Migration from Travis is super easy, just writing a simple workflow like: https://github.com/chezou/tabula-py/blob/master/.github/workflows/pythontest.yml The benefits of GitHub Actions for Python are: We can use build matrix (e.g., OS and Python versions) like Travis Launch time of GitHub is faster than Travis Easy for additional dependency installation by using uses syntax, which uses another workflow For example, installing JDK can be written as:

Photo by Campaign Creators on Unsplash Testing workflow runnability would be important when we build a complex workflow. digdag is a workflow engine which syntax is simple and is able to run tasks with SQL, Python, Ruby, shell script, etc. digdag has Docker executor and it works like a charm with py>, rb>, and sh> operators. How to ensure a new Docker image runnable with existing digdag workflow? I’ll show the way to run through it on CircleCI.

I attended OpML ’19 is a conference for “Operational Machine Learning” held at Santa Clara on May 20th. OpML ‘19 _The 2019 USENIX Conference on Operational Machine Learning (OpML ‘19) will take place on Monday, May 20, 2019, at the…_www.usenix.org[](https://www.usenix.org/conference/opml19) The scope of this conference is varied and seems not to be specified yet, even if I attended it. I’ll borrow the description from the OpML website. The 2019 USENIX Conference on Operational Machine Learning (OpML ’19) provides a forum for both researchers and industry practitioners to develop and bring impactful research advances and cutting edge solutions to the pervasive challenges of ML production lifecycle management.

I attended RubyKaigi 2019 held at Fukuoka from Apr 18 to Apr 21. This year’s RubyKaigi was a really great opportunity for me to know the possibility of Data Science and Machine Learning for Ruby. Data Science and Ruby As many of you may know, Ruby is widely known for web application with such as Ruby on Rails, but there is another momentum of Ruby or non-Python language. Here is the list of the sessions about Data Science.

機械学習ではよく使われるJupyter Notebookですが、これを使ってプロダクションで動くコードを書くのは非常に難しいなと思い、皆どのよ

Table of Contents workflowをpushするまえにローカル環境で開発とテストをする方法 基本戦略:Pythonのタスクを合理的な粒度にまとめる 1. Treasure Da

ML in productionの課題について話しました

RTDとRPrstoを使ってRからTreasure Dataにアクセスしてみよう

さる5/30に『n月刊ラムダノート』創刊記念パーティーという名のコルーチン祭りを開催しました。 会の中で一番嬉しそうだった鹿野さん 同僚の Toru Takahashi さ

When executing machine learning pipelines for trainings and inferences, the systems and machine learning infrastructures vary depending …

Developping and testing for workflows productively is hard. In this session, I talk about how to develop heavy data dependent workflow …

Adopting a machine learning system is an essential step for enterprise companies to progress to the next stage of their business. …

This article covers very biginning guide for MLOps, i.e., What is MLOps? How do tech giants make Machine Learning systems? What …

First book for how to design Machine Learning systems and how to proceed Machine Learning projects. This book is originally written in …