For the past 4.5 years I have been somewhat a data scientist at Shopify. From IC roles to different management roles, the bulk of the work was in data science. Prior to this, at Ericsson, I was also doing data science. Not anymore.
I am embarking on a new role on our Analytics Engineering team here at Shopify. What do we do? Glad you asked.
Shopify’s new team, Analytic Engineering, is going to build a strong data foundation for our Data Scientist. Our goal is to build the best data warehouse so DS folks can be as fast and efficient as possible. This role is also called Data Engineering in other companies.
Wait why
Is data science the sexiest job of the 21 century? Yeah maybe, but turns out I am a lot more passionate about the eng side of data science. Building data models is simple in appearance, but a real interesting challenge.
-
You need to align with the business on entity and business process definition, which is a lot more complex than it looks. Sitting 7 different people, with 7 different skills, point of views and objectives and driving to a common agreement is a hard challenge. That being said, it is extremely important. As a company you will get a lot further if everybody works with the same primitive, even if they are not, over-optimized for every scenario.
-
Building data models at scale is an interesting challenge. You can’t build data models at the Shopify scale without being extremely thoughtful about scale, PII, data size, cost, etc.
Especially #2 makes me feel comfortable in my old shoes of software development.
2023
So for the next little while, I will be focusing my work and my post towards Data Engineering.
