Coffee and DataBy Marc-Olivier Arsenault

An independent engineering notebook

Coffee & Data.

Notes on building data systems at scale, making sense of the data they produce, and the people behind them.

On the reading table


6 min read

Top Down vs Bottom Up

Early in my career, I was lucky enough to have a mentor who had a unique way of simplifying complex ideas. One of the most crucial lessons he imparted was an understanding of…

Read the article

All the field notes.

22 articles / Newest first

2023

  1. Notes 6 min read

    Top Down vs Bottom Up

    Early in my career, I was lucky enough to have a mentor who had a unique way of simplifying complex ideas. One of the most crucial lessons he imparted was an understanding of…

  2. Data engineering 7 min read

    Data Warehouse Manifesto

    My approach to the Data Warehouse and how company like Shopify should think about them.

2022

  1. Data engineering 2 min read

    I am no longer a Data Scientist

    For the past 4.5 years I have been somewhat a data scientist at Shopify. From IC roles to different management roles, the bulk of the work was in data science. Prior to this,…

  2. Talks 2 min read

    Be Impactful

    When and How to say no Internal presentation I made at Shopify earlier this year.

  3. Data science 6 min read

    Dear DM

    A message for all the Decision Makers (DM) using data to make decisions.

2021

  2. Data science 7 min read

    Warning systems on data warehouse

    The story of how and why I built Whistleblower, a system that allows us to set warnings on any table in our environment.

2020

2019

  2. Data science 4 min read

    Spark & AI Summit 2019

    My review of the latest Spark and AI Summit hosted in San Francisco on April 24th and 25th 2019.

  3. Data science 4 min read

    Spark JOIN using REGEX

    A more technical post about how I end up efficiently JOINING 2 datasets with REGEX using a custom UDF in SPARK

2018

  1. Data science 4 min read

    The data science pyramid

    Let&8217;s not start with data science this time. Let&8217;s start with psychology. I am far from having any competence in this domain, but I remember in high school being…

  2. Data science 5 min read

    This is what I really do as a Data Scientist

    Data Science is getting very popular and many people are trying to jump into the bandwagon, and this is GREAT. But many assume that data science, machine learning, plug any…

  4. Data science 2 min read

    Distribution’s 5th Symphony

    I would like to share with you a small tool I have discovered this year which is very useful; Violin Plot!

2017

  1. Ericsson 3 min read

    Machine VS Human

    Lately at work we had the opportunity to propose new Uses Cases to develop around Ericsson Expert Analytic and data science in the operator's network. We propose to work on an…

  2. Data science 4 min read

    Data Science Development Environment (Part 1)

    Part of my task in the Data Science Team is to participate in the deployment of our development environment. This will be the first, of a series of articles about the…

  3. Data science 6 min read

    Kolmogorov–Smirnov test

    Lately, at work, we had to do a lot of unsupervised classification. We basically had to distinguish N classes from a sample population. We had a rough idea of how many classes…

  4. Notes 6 min read

    Home made alarm system central

    Last year, my wife and I purchased our first house. The house came with a very standard alarm system installed by one of the main distributors in Montreal. As soon as I moved…

  5. Notes 5 min read

    Memories should stay forever

    Not so long ago, I had the chance to become a father. And one of the … "privilege" of having a baby is the time you have to think during all these nights when he won’t let us…

  6. Notes 5 min read

    The real challenge in IoT security

    About Me Before writing about my main topic. I want to take a couple of lines to introduce myself. I am an IT Engineer working since I got my bachelor’s degree (4 years ago)…