Senior Data Engineer

Job Description:

• Build and maintain ingestion pipelines bringing data from EP’s business systems into Databricks
• Automate quality checks and source reconciliation to identify missing, inaccurate, or duplicate data
• Design and develop silver and gold data layers
• Deliver governed data products supporting reporting, cross-system matching, and identity resolution
• Own day-to-day platform operations, including job scheduling, monitoring, alerting, data freshness, access control, and cost optimization
• Maintain runbooks, test recovery procedures, and document knowledge for platform support
• Manage CI/CD pipelines and deployments across development, testing, and production environments using Azure DevOps
• Coordinate with EP’s IT vendor on Azure infrastructure needs outside the data platform
• Preserve systems of record, enforce user-appropriate access, and validate licensing and platform constraints before deployment
• Partner with business stakeholders to translate needs into practical, reliable data solutions
• Design and implement confidentiality, retention, least-privilege access, and audit controls for sensitive client and matter data
• Support bounded AI and partner pilots with masking, evaluation data, and human approval
• Evaluate the inherited Databricks platform, identify improvement opportunities, and shape the growing data function
• Partner with the Manager of Data Services and Products and business, application, IT, and security stakeholders on architecture and build-versus-buy decisions
• Perform other duties as assigned by management

Requirements:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field, or equivalent practical experience
• 4+ years of experience in data engineering
• 2+ years of hands-on experience building and operating Databricks solutions in production
• Experience with multi-task Lakeflow Jobs over Delta tables governed by Unity Catalog
• Experience deploying through Declarative Automation Bundles (formerly Databricks Asset Bundles) or equivalent infrastructure-as-code is strongly preferred
• Ability to explain a production pipeline built, design decisions, failures, and operational challenges
• Ability to communicate clearly with nontechnical partners and connect engineering decisions to business impact
• Proficiency in Python, including testing with pytest and packaging
• PySpark experience preferred
• Strong SQL and dimensional modeling expertise, including serving governed datasets to BI tools
• Experience building REST API and SaaS ingestion pipelines, including pagination, rate limits, schema drift, and credential rotation
• Experience with Git-based CI/CD workflows
• Azure DevOps is a plus
• Experience using AI-assisted development tools and reviewing, testing, and validating generated code before deployment
• Experience building Lakeflow pipelines, managed connectors, entity resolution, or record linkage is a plus
• Experience managing Azure resources supporting a data platform, ideally as infrastructure-as-code, is a plus
• All work must be performed within the United States

Benefits:

• An opportunity to do challenging, dynamic, highly rewarding work with an inspiring mission
• A team of intelligent, creative, and dedicated immigration professionals who value respectful, collaborative working relationships
• Inclusive firm culture celebrating diverse perspectives, backgrounds, identities, and experiences
• People-centered workplace policies and flexible work schedules, including remote work opportunities
• Competitive compensation
• Health, dental, vision, and life insurance benefits
• 401(k)-retirement savings plan with employer-matching
• Generous paid time off
• Casual attire for employees working in person at Troy or Ann Arbor offices
• Lots of free food for employees working in person at Troy or Ann Arbor offices
• Regular opportunities for exposure to international cuisine and culture for employees working in person at Troy or Ann Arbor offices