- Many of the largest technology companies are hiring for a new, in-demand job.
- This position, called analytics engineer, has become one of the hottest jobs in technology.
- Analytics engineers use a tool called dbt and can earn about $127,000, according to Glassdoor.
In the past few years, the new role in the world of data analytics has become one of the most important jobs in technology.
While data analysts and data scientists previously had to go through incredibly messy data sets to report insights, a new role called analytics engineer is tasked with cleaning up that data and making it more usable for them.
Analytics Engineers Wanted at Some of the Biggest Companies – Apple, Amazon,
, and Instacart all employ analytics engineers in some form. They specialize in using dbt, a tool that makes it easier to build cleaner datasets that analysts can use.
“Change your LinkedIn title from Analyst to Analytics Engineer, and you instantly get a 20% pay increase and a host of job offers,” said Jeremiah Lewin, CEO of Prefect, a data curation startup. “This is enabled in large part by DBT. It’s a great example of category creation, which companies suck at these days.”
Glassdoor, which collects salaries for specific roles, suggests that the typical salary for an analytics engineer is around $127,000. But it can vary — a job listing for a Senior Analytics Engineer at Discord says the role pays more than $200,000 a year, while one from CircleCI says it pays $106.250 to $125,000. (Companies must list salary ranges in New York and Colorado.)
As data analysts prepare to move into this new position, Tristan Handy, CEO of Dbt Labs, said the roles are very different. And while analytics engineers take on some of the tasks that data engineers typically perform, it’s not just about rebranding.
“Besides different job responsibilities come different job ladders and, most importantly, different compensation,” Handy said. “The data professionals who make this transformation are very well compensated for doing so.”
One startup company, Dbt Labs, is responsible for promoting the Analytics Engineer
A Philadelphia startup founded in 2016 is primarily responsible for the emergence of the Analytics Engineer.
Dbt Labs, which created the crucial instrument for the role, is worth $4.2 billion. Analysts can use dbt to translate complex and messy data sets into cleaner, easier-to-use tables in Snowflake or Redshift.
Data engineers have traditionally used Spark or Python to move data from multiple sources and create tables in Snowflake and other SQL repositories. Instead of learning Python, data analysts can use dbt and the language they know best, SQL, to do what data engineers do.
In the past few years, dbt has caught the eye of big companies, which are increasingly moving data to storage tools like Snowflake.
“In late 2018, early 2019, we started to see a real identity emerge around analysts who were frustrated with waiting in long lines in data engineering, so we started learning the necessary practices to do the work themselves,” Handy said. “Dbt has been part of that journey for a lot of people, but at that time there has also been a rise in companies trying to build DBT-like tools in-house to empower analysts and people looking for a new name for what they were doing.”
Bob Muglia, the former CEO of Snowflake, said the increasing prevalence of SQL has made tools like dbt – and the role of analytics engineer – exponentially more in demand.
“Dbt wins in this space,” he said. “They definitely built the biggest community.”
Analytics engineers are also often required to have experience with Airflow, a popular open source big data tool for scheduling data analysis tasks. Astronomer, the startup that helps manage Airflow, said it’s making it easier for analytics engineers to use it.
“Traditionally, Airflow has been great for people from traditional data engineering backgrounds, but we’re seeing this emerging for people who are less technical but want to process data, do analysis, build models,” said Laurent Paris, Senior Vice President of Research at Astronomy, Laurent Paris. “And we want to meet it even with Airflow.”
Although dbt is very popular and has designed an essentially new role, its creator wants to expand beyond cleaning and organizing data with SQL — Handy said he expects the role to grow as dbt launches new tools like core metrics tracking.