Data Engineer

Company Overview

Information about the company is not specified.

Job Summary

As a Data Engineer, you will play a crucial role in designing and implementing data pipelines and architectures that enable effective data management and analytics capabilities within the organization. You will work closely with stakeholders to ensure the enhanced availability and usability of data, contributing to overall organizational goals.

Responsibilities

• Develop and maintain robust data pipelines using Azure/AWS Databricks to ensure the effective flow of data across various systems.
• Implement data processing frameworks using Python, Scala, Spark, and PySpark.
• Utilize strong SQL and RDBMS expertise to design efficient database schemas and optimize queries for performance.
• Work with big data technologies including HIVE, HBase, Impala, and Parquet for data storage and retrieval.
• Integrate data from various sources using Sqoop, Kafka, and Flume.
• Schedule and automate data workflows using Airflow and ensure process reliability with Jenkins or Bamboo.
• Manage version control of the codebase through GitHub or Bitbucket and maintain repositories using Nexus.

Qualifications

• Technical Skills:

  • Proficiency in Azure/AWS Databricks.
  • Experience with Python, Scala, Spark, and PySpark.
  • Strong knowledge of SQL and RDBMS.
  • Familiarity with big data tools such as HIVE, HBase, Impala, and Parquet.
  • Experience with data integration tools (Sqoop, Kafka, Flume).
  • Understanding of workflow orchestration with Airflow.
  • Knowledge of CI/CD tools like Jenkins or Bamboo.
  • Experience with version control systems such as GitHub or Bitbucket.

• Soft Skills:

  • Excellent communication skills to interact with customers and stakeholders.

• Educational Qualifications:

  • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

Preferred Skills

• Relevant accredited certifications for Azure, AWS, Cloud Engineering, and/or Databricks.
• Knowledge of Delta Live Tables (DLT) would be beneficial.

Experience

• A minimum of 7 years of experience in Data Engineering or a related field is required.

Environment

• The position is based in Gurgaon; additional specifics about the work setting (remote, in-office, or hybrid) are not provided.

Salary

Salary information is not specified.

Growth Opportunities

Opportunities for career advancement within the company are not specified.

Benefits

Details of offered benefits such as insurance and paid leave are not provided.