Lead Data Engineer- Big Data,Spark,Kafka Streaming, Full Stack-14+yrs

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

The Data and Artificial Intelligence Platform (DAP) group is a vital part of Visa’s Technology organization, providing the technology and processes to manage Visa’s extensive data assets and deliver valuable information, products, and services to our customers. The Data as a Service (DaaS) team, a strong group of data engineers, focuses on building data pipelines to meet Visa’s data needs.

As a Lead Data Engineer, you will design and develop enterprise-level data pipelines and services. As part of the DaaS development team within the Data and AI Platform organization based in Bangalore, India, your initial responsibility will be the end-to-end development of the Data Distribution Management System, migrating applications from Mainframe. In the future, you will explore new opportunities at Visa by developing and driving data services and products, leveraging the potential of machine learning and artificial intelligence, particularly Generative AI.

If you’re excited about the Payment Data and AI space, eager to learn quickly, and looking to make a significant impact, this is the ideal place for you!

 Essential Functions:

• Lead the analysis, design, development, and implementation of various initiatives, with end-to-end ownership to deliver business functionalities.
• Develop high-quality code, participate in code reviews, and mentor junior developers to ensure timely deliveries.
• Ensure the team’s deliverables meet or exceed the prescribed defect SLA by delivering zero-defect code.
• Champion security best practices by owning component security and collaborating with cybersecurity teams to implement and verify secure coding techniques.
• Collaborate with partners from business and technology organizations to develop key deliverables for Data Services and Products, including scalability, optimization, operations, availability, and roadmap.
• Effectively communicate status, issues, and risks in a precise and timely manner.
• Develop new products iteratively by building quick Proofs of Concept (POCs) and converting ideas into real products.
• Be highly hands-on, leading and developing large-scale multi-tenant software components on the platform in an Agile-based methodology to provide self-service capabilities.
• Actively participate in all phases of the software development lifecycle: analysis, technical design, planning, development, testing/CICD, release, and post-production/escalation support.
• Apply appropriate software engineering patterns to build robust and scalable systems.
• Mentor and provide technical guidance to junior data engineers, fostering their professional growth and ensuring adherence to best practices.
• Be the champion on productivity improvement using Generative AI in all aspects of SDLC.

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Qualifications

Basic Qualifications

• 14+ years of work experience with a bachelor’s degree.
• 12+ years with an Advanced Degree (e.g., Masters, PhD).

Leadership: At least 3-5 years of experience leading engineering teams.

Technical Skills:

• Full Stack Software Development for large-scale, mission-critical applications.
• Mastery in distributed big data systems such as  Spark, Hive, Kafka streaming,  Hadoop, Airflow.
• Expertise in Scala, Java, Python, J2EE technologies, Microservices, Spring, Hibernate, REST APIs.
• Experience with n-tier web application development and frameworks like Spring Boot, Spring MVC, JPA, Hibernate.
• UI/Web development using Angular or React, JavaScript, JQuery, HTML, CSS is a plus.
• Proficiency with version control systems, preferably Git; GitHub Copilot experience is a plus.
• Proficient in API Development using SOAP or REST, JSON, and XML.
• Experience developing back-end applications with multi-process and multi-threaded architectures.
• Hands-on experience with building scalable microservices solutions using integration design patterns, Dockers, Containers, and Kubernetes.
• Experience in DevOps practices like CI/CD, Test Automation, Build Automation using tools like Jenkins, Maven, Chef, Git, Docker.
• Experience with data processing in cloud environments like Azure or AWS is a plus
• Data Product development experience is essential.
• Experience in Agile development methodologies like SCRUM.
• Result-oriented with strong analytical and problem-solving skills.
• Excellent verbal and written communication and presentation skills.

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.