NielsenIQ

Manager, Data Engineering – (Big Data, Spark, Scala, Python, Cassandra, Elasticserach, AWS, Airflow, SQL)

5 April 2024
Apply Now
Deadline date:
£120000 - £194000 / year

Job Description

At Nielsen, we believe that career growth is a partnership. You ultimately own, fuel and set the journey. By joining our team of nearly 14,000 associates, you will become part of a community that will help you to succeed. We champion you because when you succeed, we do too. Embark on a new initiative, explore a fresh approach, and take license to think big, so we can all continuously improve. We enable your best to power our future. 
This position is responsible for participating as a team lead / developer in analyzing and designing highly-complex or business-critical applications, as well as developing, testing, and supporting application software.  

Responsibilities

  • Oversee the development of scalable, reliable, and cost-effective software solutions with an emphasis on quality, best-practice coding standards, and cost-effectiveness 
  • Participate as a team lead on projects, which includes training, coaching, and sharing technical knowledge with less experienced staff.
  • Rapidly identify and resolve technical incidents as they emerge
  • Build rapid technical prototypes for early customer validation of new technologies
  • Collaborate effectively with Data Science to understand, translate, and integrate methodologies into engineering build pipelines 
  • Collaborate with product owners to translate complex business requirements into technical solutions, providing leadership in the design and architecture processes.
  • Provide expert apprenticeship to project teams on technology strategy, cultivating advanced skill sets in application engineering and implementing modern software engineering practices
  • Lead and mentor a team of Software Developers and Senior Software Developers, providing guidance and support in their professional development
  • Stay informed about the latest technology and methodology by participating in industry forums, having an active peer network, and engaging actively with customers
  • Cultivate a team environment focused on continuous learning, where innovative technologies are developed and refined through collaborative effort

Key Skills

  • Domain Expertise
  • Bachelor’s degree in computer science, engineering plus 5-8 years of experience in information technology solutions development and 2-3 years managing teams. 
  • Proven experience in leading and managing software development teams.
  • Must have strong cloud Implementation expertise in cloud architecture. 
  • Must have the ability to provide solutions utilizing best practices for resilience, scalability, cloud optimization and security.
  • Basic project management skills. 
  • Technical Skills
  • 6+ years of experience: big data using Apache Spark in developing distributed processing. applications; building applications with immutable infrastructure in the AWS Cloud with automation technologies like Terraform or Ansible or CloudFormation, Cassandra & Elasticsearch.
  • Experience in Service-oriented architecture, Spark Streaming, and Git.
  • Experience in software development using programming languages & tools/services like: Java or Scala, Big Data, Hadoop, Spark, Spark SQL, Presto \ Hive,Cloud (preferably AWS), Docker, RDBMS (such as Postgres and/or Oracle), Linux, Shell scripting, GitLab, Airflow.
  • Experience in big data processing tools/languages using Apache Spark Scala.
  • Experience with orchestration tools: Apache Airflow or similar tools.
  • Strong knowledge on Unix/Linux OS, commands, shell scripting, python, JSON, YAML.
  • Agile scrum experience in application development is required. 
  • Strong knowledge  in AWS S3, PostgreSQL or MySQL.
  • Strong knowledge  in  AWS Compute: EC2, EMR, AWS Lambda.
  • Strong knowledge in Gitlab /Bitbucket .
  • AWS Certification is a plus.
  • Mindset and attributes
  • Exceptional verbal/written communication and interpersonal skills.
  • Strong leadership qualities and the ability to inspire and motivate a team.
  • Strong ability to translate business requirements into technical solutions and guide the team in execution.