DVT

Senior Data Engineers | Data Architects

31 October 2024
Full Time
Apply Now
Deadline date:
£119000 / year

Job Description

DVT is a software development, technology consulting and related services company, with offices in Cape Town, Johannesburg, Durban, UK, Kenya, Ireland and Switzerland with over 600 staff. We employ highly – skilled individuals in the fields of Software Development (.Net, Mobile and Java), Business Intelligence, Business Analysis, Agile Consulting, Solution Architecture and Project Management. We develop turnkey solutions and provide consulting services to over 100 clients, both locally and abroad.

DVT is currently on a recruitment drive for experienced Data Engineers, Data Science, and Machine Learning Engineers at all levels. The Senior Data Engineer will play a crucial role in designing, developing, and maintaining scalable data solutions and infrastructure for our clients. This position requires a deep understanding of data engineering principles, technologies, and tools, as well as strong problem-solving skills. The Senior Data Engineer will collaborate closely with cross-functional teams to ensure data pipelines are efficient, reliable, and aligned with business objectives within our clients.

DVT Benefits

  • Access to International Clients and Projects
  • Medical Aid and Life Cover through Discovery
  • Access to Discovery’s Healthy Company Programme
  • Mentorship Platform for technical and non-technical skills
  • High potential program for career acceleration for key employees
  • Yearly Personalized Personal Development Plans & Detailed Career Paths
  • Access to Udemy for business
  • Paid for Training Programs and Certifications
  • Personalized Business Coaching
  • Access to DVT’s Innovation Center and Innovation Teams & Projects
  • Access to the Databricks Learning Academy

Requirements


Job Description: Senior Data Engineer / Data Architect

Position Overview: We are seeking a highly skilled and experienced Senior Data Engineer and Data Architect to join our dynamic team. The ideal candidate will have a strong technical background in big data technologies, cloud-based data solutions, and data pipeline orchestration. This role requires excellent leadership abilities, analytical skills, and a passion for staying up-to-date with the latest trends in data engineering.

Technical Knowledge:

  • Proficient in big data technologies such as Apache Hadoop, Spark, and Kafka.
  • Strong programming skills, particularly in Python, Java, and SQL.
  • Solid experience in developing and optimizing data pipelines, architecture, and data sets.
  • Solid understanding of data warehousing concepts, ETL tools, and data modeling.
  • Good experience with cloud-based data solutions like AWS (EC2, S3, EMR, Redshift, Glue), Google Cloud (BigQuery, Dataflow, Pub/Sub).
  • Familiarity with data pipeline orchestration and workflow management tools: Airflow, Luigi.
  • Familiarity with containerization and orchestration platforms: Docker, Kubernetes.

Behavioral Competencies:

  • Shows promising leadership abilities.
  • Excellent analytical and problem-solving skills.
  • Ability to work in a fast-paced, dynamic, and collaborative environment.
  • Comfortable with ambiguity and able to adapt to change.
  • Ability to work independently and as part of a team.
  • Passion for learning new technologies and staying up-to-date with the latest trends in data engineering.
  • Excellent written and verbal communication and collaboration skills.
  • Strong business acumen.

Responsibilities:

  • Work independently on complex data engineering projects.
  • Develop and maintain optimal data pipeline architecture.
  • Collaborate with data scientists and business analysts to meet functional data requirements.
  • Assist the Lead Data Engineer in planning and implementing data engineering projects and be involved in project management and strategic decision-making.
  • Optimize and tune data pipelines for performance, scalability, and cost-effectiveness.
  • Ensure data quality and implement data validation and cleansing processes.
  • Monitor and troubleshoot data pipelines to identify and resolve issues promptly.
  • Evaluate and recommend appropriate data storage and processing technologies.
  • Develop and maintain documentation for data engineering processes, data flows, and system architectures.
  • Stay updated with emerging technologies, industry trends, and best practices in data engineering and analytics.
  • Provide technical guidance and mentorship to junior data engineers.

Minimum Experience Required:

  • 6+ years of experience in data engineering (as a Data Engineer), with a focus on building large-scale data processing systems.
  • Bachelor’s or Master’s degree in Computer Science, Information Systems, or a related field.

Who we are:

Related Jobs