Principal Data Engineer

Company Description

At Neilson, we’re reimagining the future of data-driven insights with our pioneering division, DataWorks. As a cutting-edge division within the Neilson Group, DataWorks harnesses the latest advancements in AI, machine learning, and predictive analytics to empower growth across diverse markets and deliver next-generation solutions in Business Intelligence, Analytics, Data Science, and Actuarial science. Our team, based in the UK and Australia, is at the forefront of digital transformation, using data to unlock powerful insights and drive competitive advantage on a global scale.

As part of a vibrant and forward-thinking environment, DataWorks combines the agility and innovation of a high-tech AI startup with the backing of a trusted, established provider. We develop sophisticated solutions with the latest in time-series forecasting, large language models (LLMs), and advanced machine learning, pushing the boundaries of what’s possible in analytics. Recognized for excellence in customer service, we’re committed to delivering a high-quality, fully integrated experience for our clients and partners worldwide.

We are seeking a highly technical Principal Data Engineer to define and drive our enterprise-wide data architecture and technical vision. In this role, you will architect scalable data solutions, establish technical standards, and lead the implementation of our next-generation data platform across the organization. This is a hands-on role requiring deep technical expertise with cutting-edge data tools within Microsoft’s suite and beyond. You will make key architectural decisions that shape our data infrastructure strategy while actively coding and implementing solutions.

Reporting directly to the Head of Data & Analytics with a dotted line to the Group Chief Commercial Officer, you will design distributed systems, create technical frameworks, and establish best practices that enable seamless data flows across our global organization. This position currently has 2 reports and you will be expected to foster a collaborative and high-performing environment while maintaining significant technical contribution.

Job Description

Key Responsibilities:

• Design and implement the enterprise-wide data architecture strategy, including hands-on development of proof-of-concepts for new technologies and frameworks
• Architect and actively develop complex distributed systems that can scale across multiple regions and handle diverse data workloads
•  Develop, manage, and optimize ETL processes using Azure Data Factory to ensure accurate data integration across global databases
• Design, build, and maintain Microsoft SQL Server databases, ensuring data is well-organized, accessible, and secure
• Lead by example through hands-on development and implementation of advanced data processing frameworks that can be reused across multiple teams
• Define technical standards and best practices for data engineering across the organization through practical implementation and documentation
• Evaluate emerging technologies through proof-of-concept development and make recommendations for adoption or retirement of data technologies
• Identify and implement performance optimization strategies for ETL processes and reports to ensure optimal user experience
• Monitor and troubleshoot complex data processes to ensure uninterrupted service to global teams, including working occasionally outside regular business hours to support multiple time zones
• Create and maintain comprehensive technical documentation, architectural diagrams, and best practices
• Ensure data governance and compliance principles are maintained across all architectural decisions
• Communicate technical information to non-technical stakeholders effectively, tailoring the messaging as needed for diverse audiences
• Mentor and guide team members in technical best practices and architectural principles
• Collaborate effectively with cross-functional teams, including data scientists, analysts, and business stakeholders

Required Technical Skills:

• Expert-level proficiency in SQL and database optimization techniques
• Hands-on expertise in Azure services, particularly Azure Data Factory, Azure SQL Database, and Azure Analysis Services
• Deep practical experience with Data Warehousing using Kimball and STAR schema
• Strong coding abilities in at least one programming language (Python, Java, or C#)
• Hands-on experience with distributed computing frameworks
• Practical experience implementing data streaming solutions
• Expert knowledge of performance tuning and optimization techniques

Qualifications

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related technical field
• Significant, demonstrable hands-on data engineering experience, with at least 5 years in architectural roles preferably
• Proven experience as a Principal or Senior Data Engineer, with a focus on Microsoft SQL Server, Azure Data Factory, Power Automate and Power BI
• Demonstrated track record of designing and implementing large-scale distributed data systems
• Experience creating and implementing technical frameworks adopted by multiple teams
• Strong background in systems design, including handling complex data processing at scale
• Experience evaluating and implementing emerging technologies in production environments
• Leadership experience, with a focus on mentoring and coaching team members
• Strong problem-solving and troubleshooting skills with a hands-on approach
• Excellent communication and collaboration skills
• Willing to work flexible hours to support a global business

What Sets You Apart:

• Contributions to open-source projects or internal shared libraries
• Experience with real-time data processing and streaming architectures
• Track record of solving complex technical challenges at scale
• History of successful technical mentorship
• Experience working in a global, distributed team environment

Additional Information

Why Join Us:

• Work in a dynamic and innovative team driving real business impact.
• Opportunities for professional growth and development in a leadership role.
• Collaborative culture and cross-functional engagement with various teams and stakeholders.
• Competitive salary and benefits.