This built off my previous post on what is a data engineer . Apparently, I am being reflective.
Of the many of the posts I see, seem to stress the core idea of if you don’t know Python are you even a data engineer? For many people in the data space, they start their career learning Python, often before they learn SQL. For me, that is hard to wrap my head around.
I’ll admit my bias.
I have taken numerous Python classes; Coursera, EdX, DataCamp…but much like the ad I keep seeing for FreeMote,
I learned it and promptly forget 90% of it. I was writing SQL all day long, and never using Python in my day to day, so it just atrophied. Every time I came to learn again, I had retained a little (10% ???), but generally started from scratch.
My block might be, going back to last post, I think of Python as a general purpose programming language, and I am not a coder, but a data guy, and SQL, is my lingua franca.
Or maybe the problem is the way it is positioned and taught.
Each of the classes above were for Data Scientists, and there are many others that are for Software Engineers. There are few out there that are specifically for Data Engineers. This one seems like a BIG gap to me. Tons of training on how to build a web site, or design a BI report, or do regression and machine learning algorithms, but little for data engineering as a pure skill.
Will knowing Python help me as a data engineer, absolutely.
Many jobs will require it or at least like to see it. Moreover, if you are in a company where they use Spark for big data, then mostly likely you will use some Python. Alternatively, if you are working closely with data scientists it will help to understand their language to better interoperate. Ultimately, the more languages and programming concepts you know, the more successful you will be when tomorrow a new language or methodology appears.
That said….I think YOU can be a Data Engineer and not know Python, and certainly not know it well, you just may not be as successful as you want to be. However, I don’t think you can and should be a data engineer and not know SQL.
Side note-- I am now learning Python again....maybe this time it will stick!
