Senior Data Platform Engineer

Company Overview

Details about the company, its industry, and culture are not specified.

Job Summary

The Senior Data Platform Engineer will be responsible for architecting, building, and scaling the data platform using Databricks or an equivalent framework. This role involves owning the reporting and analytics module development and contributing to building robust data pipelines, ensuring data reliability and governance while enabling analytics and machine learning workflows.

Responsibilities

• Design, build, and maintain scalable, performant data pipelines (both batch and streaming) on Databricks using Spark, Delta Lake, structured streaming, or equivalent frameworks.
• Implement and manage data architecture (ingestion, storage, transformations, modeling) following best practices (e.g., bronze/silver/gold or other layering).
• Optimize data pipelines for performance, cost, and reliability; oversee cluster configurations, job scheduling, and resource utilization.
• Collaborate with data science, product, and engineering teams to provide data access, enable self-serve analytics/machine learning, and ensure scalability and reliability for growing workloads.
• Provide technical leadership by participating in design reviews, mentoring junior and mid-level data engineers, and promoting data best practices across teams.
• Troubleshoot production issues, monitor data platform health, and proactively identify and resolve bottlenecks or failures.

Qualifications

• 4-6+ years of experience in data engineering, data platform, or similar roles.
• Strong hands-on expertise with Databricks, including Spark (PySpark/Scala), Delta Lake, job scheduling/workflows, and data transformations.
• Proficient in Python and SQL, with a strong understanding of distributed data processing, storage formats, and metadata management.
• Experience in building and maintaining large-scale data pipelines (both batch and streaming).
• Familiarity with cloud platforms such as AWS, Azure, or GCP, including storage (S3/ADLS), compute, networking, permissions, and cloud-native data architecture.
• Understanding of data governance, security, access controls, and compliance considerations.
• Ability to write clean, maintainable, and well-documented code while following best practices, testing standards, and version control protocols.

Preferred Skills

• Experience with data modeling (star schema, snowflake, dimensional modeling), data warehousing, or lakehouse design.
• Familiarity with Django and Django Rest Framework or similar API development frameworks.
• Exposure to data quality frameworks, observability, monitoring, and lineage tracking.
• Understanding of machine learning and AI pipelines, feature stores, real-time analytics, or streaming-first architectures.
• Strong communication skills, ability to collaborate with cross-functional teams, mentor others, and manage high-ownership tasks.

Experience

• 4-6+ years relevant experience in data engineering or related fields.

Environment

Details regarding the typical work setting, location, or physical/environmental conditions are not specified.

Salary

Salary information has not been provided.

Growth Opportunities

Information about potential career advancement opportunities within the company is not provided.

Benefits

Details about offered benefits such as insurance, paid leave, or work policies are not specified.