Big Data Engineer

Job Summary

We are seeking a highly skilled Big Data Engineer to join our team. This role is essential for designing and implementing scalable big data solutions that drive insights and strategic decision-making. By optimizing data infrastructure, the successful candidate will empower the organization to leverage data to its full potential and enhance overall performance.

Responsibilities

• Design and develop scalable data processing frameworks using Apache Hadoop and Apache Spark.
• Collaborate with data scientists and analysts to implement Databricks solutions for advanced data analytics.
• Manage and optimize big data pipelines and architectures to ensure efficiency and reliability.
• Ensure data quality by implementing effective data cleansing and transformation processes.
• Troubleshoot and resolve issues related to big data processing and performance.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Strong hands-on experience with Apache Hadoop and Apache Spark.
• Proficient in using Databricks and other big data technologies.
• Excellent problem-solving capabilities and the ability to work effectively in a fast-paced environment.
• Strong communication skills and a collaborative mindset.
• Proficient understanding of complex data architectures and ETL processes.

Preferred Skills

• Experience with cloud data platforms such as AWS, Azure, or Google Cloud.
• Familiarity with data warehousing solutions and additional ETL processes.
• Programming expertise in languages like Python, Scala, or Java.

Experience

• Minimum of 5 years of relevant experience in big data engineering.

Environment

• Location: Bangalore (with flexibility to work remotely or from our Bangalore office based on project needs)
• Type: Contractual position
• Start Date: Immediately
• Additional Details:
  • Openings: 1
  • Application Deadline: April 29, 2025
  • System Deadline: 2025-05-30T18:30:00.000Z

Salary

• Estimated salary range: ₹12,00,000 - ₹19,00,000

Benefits

• Competitive contractual compensation.
• Opportunity to work with cutting-edge big data technologies in a dynamic and collaborative environment.

Tools

file_search

// Tool for searching files uploaded by the user. // // To use this tool, you must send it a message. To set the tool as the recipient for your message, include this in the message header: to=file_search.<function_name> // // For example, to call file_search.msearch, you would use: // <|im_start|>assistant to=file_search.msearch code<|im_sep|>{"queries": ["first query", "second query"]}<|ghissue|> // // Note that the above must match exactly. // // You must provide citations for your answers. Each result will include a citation marker that looks like this: citeturn7file4