Data engineering is now growing faster than data science. I got to thinking, what does this mean for the data domain as a whole. Is this a good thing? Why is it happening now?
For the most part, data engineering is upstream from data science. The work that they do building data warehouses and data pipelines allows data analysts, data scientists, and ml engineers to build models and create insights from data. Why is this domain, that should precede data science, growing in popularity AFTER data science did?
For better or for worse, the world doesn’t always work linearly. Data science became sexy because many of the top technology companies were making gobs of money with it. Non-Tech companies saw the value that data science and machine learning was providing and they soon followed suit.
Unfortunately for many of these companies following the trend, they didn’t have the data infrastructure to properly leverage data scientists.
I believe the massive uptick in data engineering’s popularity comes from many less technical companies realizing that they made a massive mistake. They needed to hire people to fix their data before hiring all these data scientists.
Comments