Principal Developer – Data Integration Engineer

Company Description

At Western Digital, our vision is to power global innovation and push the boundaries of technology to make what you thought was once impossible, possible.

At our core, Western Digital is a company of problem solvers. People achieve extraordinary things given the right technology. For decades, we’ve been doing just that. Our technology helped people put a man on the moon.

We are a key partner to some of the largest and highest growth organizations in the world. From energizing the most competitive gaming platforms, to enabling systems to make cities safer and cars smarter and more connected, to powering the data centers behind many of the world’s biggest companies and public cloud, Western Digital is fueling a brighter, smarter future.

Binge-watch any shows, use social media or shop online lately? You’ll find Western Digital supporting the storage infrastructure behind many of these platforms. And, that flash memory card that captures and preserves your most precious moments? That’s us, too.

We offer an expansive portfolio of technologies, storage devices and platforms for business and consumers alike. Our data-centric solutions are comprised of the Western Digital®, G-Technology™, SanDisk® and WD® brands.

Today’s exceptional challenges require your unique skills. It’s You & Western Digital. Together, we’re the next BIG thing in data.

Job Description

ESSENTIAL DUTIES AND RESPONSIBILITIES:

• Develop, test, and deploy data integration pipeline jobs using chosen cloud based native tool according to Western Digital design principles.
• Troubleshoot existing pipeline jobs based on BASH, Python, Java and modify jobs for performance improvements.
• Develop new pipelines using Java/Spark/Scala on AWS EMR platform and schedule using AirFlow
• Design data tables for faster load and optimal query performance
• Debug SQL statements and stored procedures in Postgres and Redshift databases.
• Use git-based source control system bit bucket and continuous integration tools like Jenkins and deployment using Ansible.

Qualifications

REQUIRED EDUCATIONAL BACKGROUND

• Minimum of a bachelor’s degree in computer science or engineering. Master’s degree preferred.
• AWS developer certification will be preferred.
• Any certification on SDLC (Software Development Life Cycle) methodology, integrated source control system, continuous development and continuous integration will be preferred

REQUIRED PROFESSIONAL BACKGROUND

• Minimum of 8+ years of working in Data Integration/Engineering using any cloud/on-premises ETL tools
• Passionate about working with BASH, Python and Java. Experience in running Spark/Scala programs, Java UDFs in AWS EMR platform is preferred.
• Understanding on Unix/Linux operating system like awk, ssh, crontab, etc.,
• Ability to write transact SQL, develop and debug stored procedures and user defined functions in python.
• Working experience on Postgres and/or Redshift database is preferred. Concepts of MPP, Massively Parallel Processing well understood and practiced in solving business problems.
• Exposure to CI/CD tools like bit bucket, Jenkins, ansible, docker, etc. is preferred.
• Ability to understand relational database systems and its concepts.
• Understanding of Semiconductor manufacturing related datasets and concepts preferred

REQUIRED PERSONAL BACKGROUND

• Passionate about working with large datasets
• Ability to read, write and speak in English clearly
• Ability to work as a team
• Ability to work in multiple projects and cope up with tight deadline and time constraints
• Ability to freely express his own view and accept well known methodologies.

Additional Information

Western Digital is committed to providing equal opportunities to all applicants and employees and will not discriminate based on their race, color, ancestry, religion (including religious dress and grooming standards), sex (including pregnancy, childbirth or related medical conditions, breastfeeding or related medical conditions), gender (including a person’s gender identity, gender expression, and gender-related appearance and behavior, whether or not stereotypically associated with the person’s assigned sex at birth), age, national origin, sexual orientation, medical condition, marital status (including domestic partnership status), physical disability, mental disability, medical condition, genetic information, protected medical and family care leave, Civil Air Patrol status, military and veteran status, or other legally protected characteristics. We also prohibit harassment of any individual on any of the characteristics listed above. Our non-discrimination policy applies to all aspects of employment. We comply with the laws and regulations set forth in the Equal Employment Opportunity is the Law poster.

Western Digital thrives on the power and potential of diversity. As a global company, we believe the most effective way to embrace the diversity of our customers and communities is to mirror it from within. We believe the fusion of various perspectives results in the best outcomes for our employees, our company, our customers, and the world around us. We are committed to an inclusive environment where every individual can thrive through a sense of belonging, respect and contribution.

Western Digital is committed to offering opportunities to applicants with disabilities and ensuring all candidates can successfully navigate our careers website and our hiring process. Please contact us at [email protected] to advise us of your accommodation request. In your email, please include a description of the specific accommodation you are requesting as well as the job title and requisition number of the position for which you are applying.

#LI-AS1