Data Engineer

Visão Geral

We are seeking an experienced Data Engineer to join our Enterprise Data Lake team. The Data Engineer is responsible for designing and implementing the data pipelines that underpin our Azure-based data lake solution. This includes developing and maintaining data models, pipelines and ‘data hub’-wide services inside the Data Lake. The Data Engineer is continuously assessing and improving the performance and scalability of the data architecture. This includes tuning and optimizing platform components for efficient data retrieval, as well as identifying and resolving bottlenecks.

WHAT YOU WILL DO

• Data Pipeline Development: Design, develop, and maintain ETL (Extract, Transform, Load) pipelines to ingest data from various sources into the data warehouse using Databricks and other relevant tools.
• Data Modeling: Design and implement data models that support the organization's analytical and reporting needs within the data warehouse environment.
• Performance Optimization: Optimize data pipelines and queries for performance and scalability, ensuring efficient data processing and retrieval within the data warehouse.
• Data Quality Assurance: Implement data quality checks and monitoring mechanisms to ensure the accuracy, completeness, and consistency of data stored in the data warehouse.
• Security and Compliance: Implement data security measures and adhere to compliance standards (e.g., GDPR, HIPAA) to protect sensitive data within the data warehouse.
• Documentation: Document data pipeline workflows, data models, and technical specifications to facilitate knowledge sharing and ensure maintainability of the data warehouse environment.
• Collaboration: Collaborate with other teams to understand data requirements and deliver solutions that meet business objectives
• Follow standard procedures for proper escalation of unresolved issues to the appropriate internal teams.

ABOUT YOU

• Desired Skills & Experience
• Data Analytics (Data Lake, Data Mesh, OLTP, DWH, Data Marts, BI) – deep technical knowledge (5+ years)
• Cloud (Azure, Databricks, Data Factory) – deep technical knowledge (3+ years hands-on experience)
• Data Modeling (relational DBs, dimensional modeling, data vault modeling) – deep technical knowledge (5+ years)
• Data Pipeline development (with scalability, automation, and CI/CD in mind) - 5+ years hands-on experience
• 5+ years development experience with SQL
• 3+ years development experience with Python
• Development Process (version control, CI/CD, testing, Agile) - 5+ years hands-on experience
• Mentoring junior team members
• Complete project and support work with little to no oversight
• Excellent problem-solving skills and attention to detail
• Strong communication and collaboration skills to work effectively in a team environment
• Experience in data architecture

TOTAL REWARDS

Our total rewards philosophy integrates programs for compensation, benefits, recognition, learning and development, corporate culture, corporate citizenship and work-life balance. While individual program componentsmay differ by country, some things remain constant:

• our commitment to rewarding results
• opportunities to work with talented and driven individuals at every level of our company who respect each other, treat each other fairly and hold one another accountable for our customers’—and our company’s—success

There's more ...

• inclusive employee resource groups
• generous medical, dental, vision and other great benefits
• paid parental and medical leave programs
• 401(k) with a company match componentand profit sharing
• 15 days of paid time off plus company holidays
• Hybrid work model with flexibility
• tuition reimbursementand student loan repayment assistance