Synechron

Senior Data Engineer – Ab Initio

29 October 2024
Full Time
Apply Now
Deadline date:
£59000 - £109000 / year

Job Description

Overall Summary:

We are seeking a highly skilled Senior Data Engineer with extensive experience in Big Data technologies, particularly with Ab Initio. The ideal candidate will have a strong background in software development using Scala, Java, or Python, along with expertise in Hadoop, Spark, and Hive technologies. This role involves designing and implementing data processing solutions to derive insights from complex datasets.

Overall Responsibilities:

  • Design and implement robust data processing solutions using Big Data technologies (Hadoop, Spark, Hive).
  • Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions.
  • Perform data wrangling to clean, structure, and enrich raw data for analysis and reporting.
  • Develop ETL processes and manage data workflows using Ab Initio and other relevant tools.
  • Create and maintain documentation for data engineering processes and solutions.
  • Participate in code reviews and provide mentorship to junior data engineers.
  • Ensure the delivery of solutions aligns with business objectives and timelines.

Technical Skills:

Programming Languages:

  • Proficient in Scala, Java, or Python for building solutions in Big Data environments.

Big Data Technologies:

  • Experience with Hadoop (HDFS commands) and Spark (Spark commands).
  • Proficiency in Hive and HQL (including UI tools and metastore).

SQL Scripting:

  • Strong SQL skills with experience in HQL, SparkSQL, or any RDBMS (Teradata, Oracle, MySQL, SQL Server, Redshift).

Data Movement Tools:

  • Experience with Sqoop for import/export operations and managing jobs and metastore commands.

Streaming Technologies:

  • Familiarity with Kafka or Spark Streaming.

Ab Initio:

  • Working experience with Ab Initio technologies (GDE, Express>IT, Control Center).

Version Control & CI/CD:

  • Proficient in GitHub and other versioning tools.
  • Experience with CI/CD tools (e.g., Stash, TeamCity).

Job Scheduling:

  • Basic and advanced knowledge of AutoSys, including JIL creation.
  • Familiarity with other job scheduler tools.

Operating Systems:

  • Knowledge of Unix/Linux OS commands and shell scripting.

Data Wrangling:

  • Experience in cleaning, structuring, and enriching raw data.

Experience:

  • For Senior Data Engineer: 5+ years of experience as a Big Data Software Engineer.
  • For Data Engineer: 3+ years of experience as a Big Data Software Engineer.

Day-to-Day Activities:

  • Analyze and understand business requirements to design and implement data solutions.
  • Develop and maintain data pipelines and ETL processes to support analytics initiatives.
  • Monitor and optimize the performance of data processing jobs and workflows.
  • Collaborate with data scientists and analysts to ensure data availability and quality.
  • Participate in Agile ceremonies and contribute to team processes using tools like Jira.
  • Document processes and maintain version control for all data engineering solutions.

Qualifications:

  • Bachelor’s degree in Computer Science, Information Technology, or a related field.
  • Relevant certifications in Big Data technologies or cloud platforms are a plus.

Soft Skills:

  • Strong critical thinking and problem-solving abilities.
  • Excellent communication skills to collaborate with cross-functional teams and stakeholders.
  • Ability to work independently and as part of a team in a fast-paced environment.
  • Strong attention to detail and a commitment to delivering high-quality solutions.
  • Experience working within an ITIL Service Management environment is a plus.

S​YNECHRON’S DIVERSITY & INCLUSION STATEMENT
 

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice

Related Jobs