How He Got $600,000 Data Engineer Job

median 50th percentile person on my team was making 500 I do think that there's definitely some data engineering roles that are going to be uh replaced hi everybody welcome back to another video in today's video we're going to talk about data engineering with none other than Zach Wilson Zach welcome to my YouTube channel I'm so excited to have you thank you I'm very happy to be here Zach is currently in Seattle and we just hosted a Meetup together and after this Meetup we thought we would create some content for all of you so thank you

Zach for being here and I'm so excited to talk to you today yeah it's going to be great so let's jump in cuz I am pretty sure when you became a data engineer data engineering did not exist yeah like I actually started my career as like a data analyst kind of Tableau guy that's like what I did at my very first job and I recognized at least for that role back then this was like 2014 like 10 years ago and I recognized I was like I don't want to be doing this like for I'm done

like CU I feel like I mastered Tableau in like 9 months and I was like this is as much as this tool can do like you can't go further and so that's when I got kind of got bored of that role and I was like I need to go somewhere else and then like I read a stat stat was that 90% of the world's data had been created in the last 18 months I the same Stu yeah I'm telling you and I was like damn I need to get in there right and like so that's

when I joined this startup called think big analytics okay and the other one I got obsessed with like the yellow elephant Hadoop right that's like I was just very drawn to that elephant I was like I need to learn this elephant I remember I made a post back then where I was like this elephant is about to become my entire life right and I did not understand like how true that posted end up being but like then I worked at think big for a while uh I was not there very long because what happened was

think big got acquired by terata and terod is like this corporate company and like you had to like wear like freaking like business like like a button- down shirt like it was like very corporate very corporate I was there I was there for like 6 months 7 months and I was like I'm like I hate it here I don't like this this is not the company I got hired for I thought I was joining like some hip startup and yeah and I was like this is not it for me and left I left and did

like some software stuff for a little bit like 6 months where I literally like went from Utah to Washington DC did software engineering for 6 months and then I got a job at Facebook doing data Engineering in San Francisco so I moved again and yeah 2016 was a messy year for me where I was just driving all the time I felt like that's all I was doing that year but then once I got in at Facebook like that's when I felt like I actually was like doing real data engineering and I was like getting into

it more like that's we using like a lot of Hive back then it was a lot of Hive and this thing called Data swarm which is essentially airflow meta Netflix Airbnb and then entrepreneurship I want to talk about Netflix and Airbnb and meta like how was the culture different between the three oh they're very different So like um Facebook one of the things about is great about facebook/ meta is that like it's very friendly everyone is very collaborative in some ways I actually felt like it was too collaborative right where I was like I'm just

trying to get my work done and then because like then people like hey Zach I have a question hey Zach I have a question I'm like dude I get 30 minutes a day to work on things right like and S familiar and so like uh and then Netflix was not as much that way right well and one of the other things at least for me in my journey there was like I got hired at Netflix and when I was really young especially for Netflix cuz so when I joined Netflix I was 24 and the next

youngest person on my team was 35 so like there was an 11year gap between me and the next youngest person on my team and so like for me definitely at Netflix I like the culture was still very collaborative but also people like had their own lives their own families they have all their own like stuff like and all that stuff and like I like I for me in my relationship to work in Netflix though I just had imposter syndrome the whole time because it was just like I was like I don't belong here like I'm

working with all these really talented people like why am I here too right and so I like and I just felt like I'm like I have to work to like prove that I belong here right yeah for sure did you get to that point then like where you're imposter and went away yeah it was like um I felt like it was after about a year when I like when I especially when I got like my first big win with this like database thing I was working on like I was like okay okay I belong here

I belong here for I love that I love that I feel like especially like first year for anybody like who's just starting out it's always like cuz I had a similar experience first year such a big imposter syndrome and then after delivering some stuff I'm like no I think I got it like I can do it so I know you started when you got your first job you started at 80k M how did that progression go from 80k to meta Netflix and then Airbnb 600k that's like unheard of so how did you do that how

do people do that yeah it's a journey like and it was actually something that I even was surprised by myself especially like when I got that job making 80k at terod dat back when I had a wear a button- down shirt and I hated it like I my dream for myself was like I'm like dude if I can get to 200 I will be such a baller I will be like this is going to be my life and I thought 200 was like when I'm 35 right when I'm like 10 years deep 15 years deep

that's when I'm going to hit 200 I learned a couple things about it like a big thing about it is like you need to get into the system in big Tech that's a big part of it is like if you can get that experience in especially meta specifically I feel like they do a great job at like investing in their Engineers better than Amazon I feel cuz Amazon really like holds that senior promotion as like a you got to really like work incredibly hard to get senior whereas a meta like they like encourage people more

like my friend Ryan right he did from he went from Junior to staff at meta in three years he just got promoted every wow you have to do that at meta right they push you they really do like I mean it's good and bad I feel like cuz it's like it's good like if you want to grow but it's bad if you want like a life if you want to CH if you want a coast meta is not the place for you no that's Google or Airbnb or Airbnb both yeah both are really great so

for me like when I got in at Facebook so I went from 80 and then like a year later cuz I was making adk in 2015 at terod data and then when I got in at Facebook it was like 185 only actually ever been internally promoted one time in my career one time and it was at Facebook at Facebook cuz I got hired as a junior engineer at Facebook and I got promoted to L4 right that's the only promo I've ever actually gotten so going to L4 bumped me to like 230 is right and then

I was like I'm done I want to be a software engineer and I'm like then I go to Netflix Netflix is crazy Netflix is a crazy company they give you all cash yeah they give you all cash just up front here you go we're going to pay you as much as like the market will give you right and I actually made a mistake I made a big mistake at when I got my interview at Netflix I probably lost $100,000 from from this mistake so I accepted my initial offer at Netflix I accepted was 365k cuz

going from 220 to 365 I was overjoyed right like I actually learned though I learned that the median for my team the not the highest paid guy the median 50th percentile person on my team was making 500 right so and I didn't even know about that I just was like so excited to get in at Netflix I learned about that about six months deep into my into my time were they expecting you to negotiate is that why they gave you offer yeah I thought I was being smart so what I said was like I won't

accept anything less than what an E5 makes at meta right and they gave me exactly that right you basically yeah and like but they they would have given me more than $100,000 more like had I freaking like actually like done my research that's why you never give the first number never never never and good news though was about that was that like uh ultimately after about 6 seven months at Netflix I got on to a very like senior like borderline staff level project like this graph database project and I was able to negotiate up I

was able to negotiate up in my first year at Netflix from 365 to 550 right and then I was like damn those like I've never had a raise like that before I was like this is a crazy raise crazy especially if it's all cash yeah it's all cash right your bank account is probably freaking out taxes man don't talk to me about taxes on the topic of tech salary I analyzed entry-level data scientist salary using this powerful AI tool that you should know about it's called Julius AI think of it like a data analyst or

a data scientist that is available at your fingertips that can write python code for you and yes if you're wondering it is 10 times better than Chad GPT Advanced analysis don't believe it let me show you so the first thing we're going to do is find a file that we're going to use for data analysis next thing I do is I go to Julius so here I have an option to either upload the file or connect it to Google Sheets next thing I'm going to do is I'm going to ask it to filter the data

with zero or one year of experience and the coolest thing is that it writes code while doing it so if you're just learning python or R yes it can do R it has R capabilities as well so if you want to learn this is actually a great place to Learn Python and R while it's doing the data analysis for you the next thing I'm going to do is I'm going to ask you to create visualization of data scientist salary by company in Seattle and it has created the visualization for me basically looking at this plot

I can tell that Uber pays the highest amount of entry-level salary to data scientist followed by Amazon then Zillow and then Facebook so as you can see that it has not only created a visualization for me it has also given me additional prompts and things that I can ask you this took me less than 3 minutes imagine if you had access to it actually you do you can try Julius AI for free using the link in description thank you Julius AI for sponsoring this section of the video now let's go back to Zach's salary progression

to 600k M 20 19 2020 like they Netflix changed they changed a lot so the big thing they did so before they had two orgs they had data engineering and infrastructure and then they had data science and algorithms two orgs right and then what they did in 2019 is they just they fired the data engineering leaders all of them my VP my director my manager the whole chain just right acts right and then they were like oh yeah we're not hiring him we're not going to rehire those leaders you guys are all now part of

data science and for me I was like I don't know about that I don't think cuz especially because the leaders that they got they let they let go I really believed in MH so I became very disillusioned about working there because I was like also just living in fear cuz I was like okay these people who I really these leaders who I looked up to like they taught me a lot like I feel like they were like some of the best people I've ever worked with and like they're getting let go and I didn't really

get it and I'm like and I lost all motivation to work there so I was like I'm done I'm not going to work here anymore and then I left and I just like went on some soul searching for a bit cuz I like I realized at that point that work was my life that was my entire life that was everything right and so like after I quit there was some months there where I was like who Am I who am I as a person who is Zach right I was like I I really lost in

touch with like my own like person and then 2020 pandemic that year was wild that year was so wild and then ultimately I got in back at Airbnb like 9 months later right when I started uh at Airbnb that was also when I started making content CU I knew cuz when I quit Netflix it's the work life balance it's for it's true for me too yeah yeah for sure gole content yeah and for me when I quit Netflix I had a vision for myself that I was going to be a Creator cuz I vividly remember

when I quit Netflix the day I quit when I was like I we had like a going away lunch and when I was gone I was like when I like I gave a peace sign to all my co-workers and I'm like you're going to see me again everywhere that's what I said right I had a vision for myself then and I wish I would have overcome some of that like depression and sadness in 2020 to start earlier but I mean I'm happy I started when I did cuz like I was able to get in it

and like uh just start growing cuz it's all about like just getting that initial momentum of like I'm going to do this this is my new thing right then after a while it's like compound interest just keeps you motivated after a while you know so 600k at Airbnb yeah for sure and that's when you decided to leave all together and when was it this was about a year ago 2023 okay so for a year you have been an entrepreneur primarily focused on data engineering what like data engineering teaching uh that's my big Focus yeah okay

and how is that going do you miss corporate Ah that's a great question I miss some things about corporate yes I mean I actually gave an interview a couple weeks ago I had my own doubts like the entrepreneur journey is very full of doubts itself when you have employees and you need to make payroll and you have like all of this like pressure to like you have to make sales it's like if you don't make sales you're going to have to fire people right it's like there's like a lot of like consequences like when you're

an entrepreneur that are like when you're an engineer you just don't think about them you just think about solving the problem in front of you I feel like for me the things that I miss the most about corporate is the big one is working on a team actually like in an office I missed the office actually a lot that was actually one of the things that like I was like kind of seeking out where I'm like I need I need to like be around people and like work on problems together uh I think that's the

big thing I miss I kind of also miss just like being able to like not think about food and have to like having to like feed myself three times a day it's a lot of work when I worked at meta it was like they would give me breakfast lunch and dinner and it's just like automated and I don't have to think about it right that's also really nice so I know that's a very Tech bro take right there but that's like it is what it is right no it's a fair take for F I'm there

the things I don't miss are like feeling like uh I'm in a situation that is outside of my control like I feel like one of the things that's great about an entrepreneur is it's extreme accountability like if you are successful it's because of things you did if you aren't successful it's because of things that you did or didn't do so talking about data engineering uh really quick if somebody wants to get into Data engineering what should they learn where should they start what's your advice to them that's great uh I think there's a couple things

there like you have to learn the languages languages are the like critical It's like because if you don't know the languages you can't speak the stuff that's necessary so you SQL and python are going to be the most important if you're trying to be more like in the like Cutting Edge space then maybe learn like Scala or rust rust in particular is probably like in the future like give it like five years it's going to be bigger um and then uh then you need to know the tools the tools are like spark airf flow but

there's like 5 million competitors for airf flow now there's like air flow Mage prefect Dag data bricks workflows Azure workflows there's like a there's there's a lot of freaking ways to schedule a job right so and just learning Kon and how to schedule a job and then picking one flavor of that and then spark is so important though I think that spark is going to keep being bigger it's actually getting more adoption which I think is crazy because it's been around for so long it's been around for like 12 years and it's like and it

and it still feels like it's in the you know the early part of the adoption curve it hasn't like it hasn't leveled off yet it's still like it's still growing which boggles my mind I feel so lucky that I like caught that in 2016 where I'm like I'm going to learn spark and it's going to be awesome and I was right that's awesome yeah I think and then the last important skill though is data modeling because you got to remember that like just because you have a pipeline that is productive and good and efficient if

it produces data that is annoying to use then you you drop all your value at The Last Mile right and that that last mile of like what is the contract between you and the data scientist right like what is like what are the name what do you name your columns make sure you don't have stupid column names it's like there's a lot of these things are actually very basic but a lot of people get them wrong you like you'll see it in the wild even in big Tech at companies where they pay people 500k they

still get it wrong and so like getting that like modeling stuff right is very important and I think those are going to be the three ones like so orchestration spark and modeling are going to be the big things and then like where to start I mean my blog is pretty good so blog. dateng engineer. a lot of things there I have a lot of road maps that go a lot more in detail than the last two minutes that I've said but like that's going to be a big one and then there's like a big another

thing the Google Google the data engineer handbook so I have a GitHub repository that has over 8,000 stars that has all the resources that you need to get into Data engineering and yeah I built that like a couple months ago in the description awesome yeah so that's how I'd say to get in for sure okay awesome cool last thing and then we'll close the video um AI yeah should data Engineers be worried about AI I mean ah great question I think less worried than the current market makes it feel like it sometimes it feels like

you know there's like those things like Devon that are out there where it's like look this thing is can just do your job and it's like no it can't like like I mean but like there's uh there are spaces right like I feel you need to be able to leverage AI in a way like for example for me like when I uh have been building things like uh like you can solve problems like so much faster like if you just use AI like and like there it it's like a I think of it more of

like as like a super Google it's like a Google that gives you a an answer that might just be right and you can paste it in and it just works right but not all the time obviously you still need to know the nuances of things but like I think as a tool it's great and I think uh there are going to be roles I do think that there are I mean I I do think that there's definitely some data engineering roles that are going to be uh replaced and uh changed or like they're going to

morph right because like especially with like the combination of llm generated stuff and then you have like uh low code no code tools you know like five Tran and all those other kind of tools there and when those things marry and then you have like people who can create pipelines with like a sentence and five Tran then like there's going to be a lot of those roles that right now are done like by a data engineer that don't need to be done by a data engineer and that's why there's this new architecture that's coming out

called Data mesh where you have people who like cuz the data engineering pattern itself is in some ways kind of like a middle right because like you as a data engineer need to talk to the business and talk to the analysts and understand try to understand all the pieces of the puzzle when like you aren't as close to the business yourself like you are like kind of in the middle ultimately if the the business owners the people who are solving those problems directly like I don't it might be the PMS it might be some other

people if those people can write the pipeline themselves and manage and maintain the pipeline themselves it's a better ownership pattern it's a more stable ownership pattern as well and that's what data mesh is looking to solve and I ultimately think that data mesh will work even even though I've seen it fail I've seen it fail a bunch of times in companies but I think it's because we don't quite have the tooling yet and llm is one of those things that's going to make that more likely to happen yeah but for a lot of the data

engineering roles the ones that are not just like a select query and an aggregation or like something that's just a little bit more like anything doing with like Master data or machine learning or anything like in those areas those roles are very safe because those roles are just so nuanced and so difficult to get the the model right that like you need to have that person with that expertise so that's where if you want to like have a safer data engineering role lean more into uh like machine learning stuff lean more into Master data management

and because those roles are very safe I do not see those roles going away anytime soon awesome cool where can people find you yeah I mean I'm on YouTube data with Zach is probably going to be the best place you can also find me anywhere um on the internet my username is exactly e CZ l y and that's yeah those are going to be the main places awesome well thank you so much this is awesome and hopefully people watching they learned something watching from this video so thank for sure glad to be here yeah awesome

all right bye so good