Big Data Developer

Company Overview

[Company Overview is not specified.]

Job Summary

The Big Data Developer plays a crucial role in designing, developing, and implementing big data solutions using various AWS services and technologies. This role is essential for efficiently managing large datasets and ensuring seamless data integration and processing within the organization's data framework.

Responsibilities

• Develop and implement scalable big data solutions using AWS services like EMR, S3, Glue, and RDS.
• Design data pipelines using Apache Airflow for orchestration and scheduling tasks.
• Collaborate with cross-functional teams to integrate microservices, APIs, and domain services.
• Maintain and optimize data storage solutions across relational and NoSQL databases.
• Implement continuous integration and continuous deployment (CI/CD) practices using tools such as Jenkins.
• Utilize Python (PySpark) for data processing and transformation tasks.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• 6-10 years of experience in big data development.
• Proficiency in Pyspark, Scala, or Java for data processing.
• Solid understanding of AWS services including but not limited to EMR, S3, Glue, RDS, and DynamoDB.
• Experience with CI/CD tools, preferably Jenkins.
• Familiarity with relational databases and NoSQL databases.
• Knowledge of containerization technologies such as Docker and Kubernetes (K8s).

Preferred Skills

• Experience in architecting and deploying microservices or API gateways.
• Familiarity with data warehousing solutions and ETL processes.
• Experience using additional AWS services beyond the core listed.

Experience

• Minimum of 6 years and up to 10 years of relevant experience in big data development and management.

Environment

• This position is located in Hyderabad, offering a C2C opportunity. The typical work setting may involve both remote and in-office options, depending on the company’s policies.

Salary

• The estimated salary range is between 1.3 to 1.7 LPM.

Growth Opportunities

[Growth opportunities are not specified.]

Benefits

[Benefits are not specified.]