Data Engineer (Java+Spark)

The Data Engineer (Java+Spark) role is essential for developing scalable and reliable data pipelines that support the organization’s data needs. This position requires a combination of technical expertise in Java and Spark, as well as the ability to communicate effectively with stakeholders to understand business requirements.

Responsibilities

• Develop and maintain data pipelines to aggregate and transform data from multiple sources.
• Implement and optimize Spark applications, utilizing components such as Spark Core, Spark SQL, RDD, DataSet, and DataFrames.
• Conduct performance tuning and optimization using techniques such as caching data in memory and broadcasting.
• Collaborate with cross-functional teams using Scrum methodology to ensure project goals are met.
• Engage with end clients through effective communication, both via email and phone, to ascertain business needs.
• Utilize version control systems, specifically Git Bash, to manage code changes and collaborate with team members.

Qualifications

• Education: Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Technical Skills:
  • Proficiency in Java and solid understanding of its ecosystem (Must have).
  • Experience in Data Engineering with a focus on Java and Spark (Must have).
  • Basic knowledge of Linux / Linux scripting (Must have).
  • Good understanding of Spark architecture, including Spark Core, Spark SQL, RDD, DataSet, and DataFrames (Good to have).
  • Experience with performance tuning and optimization techniques (Good to have).
  • Familiarity with Azure/Cloud DevOps concepts and CI/CD pipelines (Good to have).
  • Knowledge of deployment pipelines (Good to have).
  • Hands-on experience with Git Bash (Must have).
  • Experience working within a Scrum methodology environment (Good to have).
• Soft Skills:
  • Strong communication skills, capable of engaging with clients and team members effectively.

Preferred Skills

• Familiarity with Spark application architecture.
• Understanding of best practices in data pipeline development and deployment.

Experience

• Minimum of 5 years of relevant experience in data engineering, with a focus on Java and Spark.

Environment

• Positions available across all PSL locations.
• Full-time role with immediate start date.

Salary

• Experience 4-5 Years: 14 LPA
• Experience 5-8 Years: 16.5 LPA

Growth Opportunities

Growth opportunities within the company not specified.

Benefits

Benefits offered not specified.