TTT-Astra Data Engineer-Senior

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. 

Astra Data Engineer (Senior) – Job Description

Skill set

• 3 to 4 years of hands-on experience in data engineering using cloud services like Astra and Databricks.
• Experience in data management and maintenance of NoSQL databases like Astra and Cassandra.
• Thorough knowledge and working experience in developing No SQL data models, scripting and querying in CQL, data mining, clustering, partitioning and segmentation techniques.
• Exposure and working experience in running spark jobs (written in Scala) in Databricks including configuring jobs, monitoring clusters, and troubleshooting job failures.
• Basic understanding on the architecture of applications written in Java or .Net and their interaction with databricks jobs through API.
• Understand and have practical experience with infrastructure technologies and components like cloud services, servers, and networking concepts.
• Proven experience as a data engineer in a similar role involving large data volume applications.
• Understanding of CI/CD release pipelines and code maintenance architectures.
• Overall understanding of SDLC Methodologies and practices, agile working model and processes.
• Strong analytical skills with detailed problem-solving abilities and good communication skills (verbal and written).
   

Responsibilities

• Work with business and application development teams to analyse and understand business data requirements and translate them into data architectures.
• Partner with Sr. Data architects and infrastructure teams to create platform level database designs and arrive at database platform level decisions.
• Build, test, and maintain the database architecture for a large-scale data processing requirements using technologies such as Astra, Databricks, Apache Cassandra. Configure and administer replication, high availability, and encryption of data. 
• Perform daily monitoring activities on the database clusters and spark jobs across all environments in the infrastructure, proactively identify problem areas or red flags and communicate to stakeholders.
• Work with infrastructure teams on maintenance activities, tuning databases, and DBMS to achieve high performance and availability, use data monitoring and analysis tools, tune configuration parameters, alter physical designs, benchmark etc.  
• Perform development changes such as data evolutions, data level changes, schema changes and database migrations based on the requests received from the business team and development teams.
• Work with DataStax Support engineers on POCs for new requirements, support questions, investigation and troubleshooting of issues coming up across all environments.
• Participate in design and architecture discussions and recommend efficient data models in terms of performance and management of data.
• Own the release and deployment activities on the database side, track changes made per environment, automate deployments (scripts execution), perform testing on database releases.
• Collaborate with data scientists and architects on the team for debugging issues and providing on-going system maintenance. Carry out systematic and rational analysis to identify the root causes of issues. 
• Maintain data security, privacy, ensure systems follow the industry’s best practices, processes, and standards.
• Document database requirements inclusive of data and transaction volumes, scalability, security, and performance requirements.

EY | Building a better working world 

 
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.  

 
Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.  

 
Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.