Data Engineer

We are seeking a motivated and results-driven Data Engineer to join our Reporting & Analytics Development Team. In this role, you will play an instrumental part in delivering reporting and analytical solutions, contributing to projects involving data lake house architecture, performance optimization, ETL pipeline development, and collaboration on Machine Learning & AI initiatives. You will have a key role in modernizing our cloud infrastructure and tool stack while striving for maximum success within your team. Together, we will work cohesively to improve and achieve our objectives.

Job Responsibilities:

• Design and maintain data lake house architecture based on a variety of database engines such as MS SQL, Snowflake, Exasol, S3 etc.
• Help on planning the growth of the infrastructure; improving system resilience, performance and stability.
• Develop and maintain ETL pipelines from various sources like RDBMS, NoSQL, Kinesis and other batch or streaming sources and flat files.
• Utilize AWS serverless architecture patterns in system design (ECS, EMR, Lambda, etc)
• Automate infrastructure with Docker and CI/CD pipelines.
• Ensuring consistency of technology usage across the data domain, by continuously reviewing existing toolsets and code and suggesting re-use of components.
• Collaborate with data analysts and data scientists and contribute in developing Data Models and Data Workflows.
• Contribute to content recognition AI/ML projects and other initiatives.
• Work with stakeholders on requirements and negotiate the best outcome.

Requirements

• At least 3 years of relevant experience in data engineering.

• Experience in designing, building, and launching highly available distributed systems for data extraction, transformation, and loading.
• Proficiency with Python/Java, shell scripting, system diagnostic and automation tooling.
• Strong skills in writing, optimizing, and profiling SQL queries.
• Familiarity with cloud services, particularly AWS.
• Familiarity with Data Warehousing design principles and best practices.
• Familiarity with data modeling.
• Experience with No-SQL technologies.
• Familiarity with CI/CD process as well as tools and processes including Git, Jira etc.
• Experience with setup and configuration, setting up design best practices, coding best practices and configuration management process for the data engineering team as well as with BI tools.
• Detail-oriented with a strong sense of ownership and accountability.
• Excellent communication skills with the ability to interact with technical and non-technical stakeholders.