Senior Hadoop Administrator / Big Data Engineer

Company Overview

(Not specified)

Job Summary

We are seeking a Senior Hadoop Administrator / Big Data Engineer with 6 to 8 years of relevant experience to lead a migration project from Oracle Cloud to an on-premises MySQL database solution. The ideal candidate will possess strong technical skills in big data technologies and will be responsible for end-to-end pipeline design, orchestration, and monitoring, ensuring seamless integration across multiple data sources.

Responsibilities

• Lead the migration project from Oracle Cloud to MySQL database on-premises, ensuring efficient execution.
• Develop and execute SQL scripts and Python migration processes to facilitate data transfer and transformation.
• Design and implement on-premises solutions on Linux operating systems, focusing on standalone databases with multi-source connectivity.
• Manage cluster configurations including primary/secondary setups, load balancing, and high availability (HA) implementations.
• Optimize performance through Spark tuning, Hive query optimization, and resource management in YARN/EMR.
• Build and deploy automated data pipelines within the on-premises environment, utilizing CI/CD methodologies.
• Integrate monitoring and governance frameworks using logging and security tools to ensure data integrity and compliance.

Qualifications

• Education: Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• Technical Skills:
  • Proficient in Hadoop (HDFS, YARN), Hive, and PySpark.
  • Strong experience in data ingestion and ETL processes for end-to-end pipeline design.
  • Expertise in MySQL and on-premises infrastructure management.
  • Advanced programming skills in Python, PySpark, and SQL.
  • Knowledge of HDFS and Hive Metastore for storage and management.
  • Familiarity with performance optimization techniques in big data environments.
• Soft Skills:
  • Strong analytical and problem-solving skills.
  • Excellent communication and teamwork abilities.

Preferred Skills

• Experience with security frameworks related to data governance and compliance.
• Familiarity with additional big data tools and technologies.

Experience

• 6 to 8 years of hands-on experience in big data engineering and Hadoop administration.

Environment

• Work is primarily on-site in Noida, with a Monday to Friday schedule (5 days work from office).

Salary

• Estimated salary range of ₹22 LPA to ₹26 LPA.

Growth Opportunities

(Not specified)

Benefits

(Not specified)