GCP EDW Specialist

Job Summary

The successful candidate will be responsible for managing cloud-based data solutions in the Google Cloud Platform (GCP) environment. This role focuses on the ingestion of data from SQL sources and APIs into Google Big Query while ensuring that standardized frameworks, authentication mechanisms, and secrets management practices are rigorously followed. You will play a key role in maintaining and optimizing our cloud data services to support business intelligence and analytics needs.

Responsibilities

• Ingest and transform data from SQL sources and APIs into Google Big Query.
• Utilize cloud services such as Google Cloud Storage, Google Composer (Airflow), and Google Firestore to build and manage data pipelines.
• Implement and maintain robust authentication protocols and secrets management strategies.
• Adhere to standardized frameworks to ensure data quality, security, and scalability.
• Collaborate with cross-functional teams to troubleshoot issues and optimize data workflows.
• Write and optimize complex SQL queries and Python scripts to meet evolving data requirements.

Qualifications

• SQL (expert): Proven expertise in writing and optimizing complex SQL queries.
• Python (intermediate): Ability to develop scripts and assist in automation tasks.
• Solid experience with Google Cloud Storage and Google Big Query.
• Hands-on experience with Google Composer (Airflow) for orchestrating data workflows.
• Familiarity with Google Firestore for NoSQL data storage.
• Practical knowledge of authentication, secrets management, and standardized cloud frameworks.
• Educational background in Computer Science, Information Technology, or a related field (e.g., BSc, BTech).

Preferred Skills

• Prior experience working in an EDW (Enterprise Data Warehouse) environment.
• Proven track record of deploying and managing GCP cloud data services in a production environment.
• Ability to work in agile teams and adapt quickly to new technologies.
• Strong problem-solving skills and attention to detail.

Experience

• Demonstrated hands-on experience in cloud data environments, particularly with GCP services.
• Familiarity with data ingestion techniques from various sources (SQL-based and APIs) is highly valued.
• Previous experience in managing EDW or similar cloud data projects is a plus.

Environment

• The work setting will predominantly involve the use of cloud-based tools and technologies.
• Job location and work schedule (remote, in-office, or hybrid) are to be defined by the hiring company policy.
• The role may require collaboration across different teams, utilizing virtual communication tools and cloud-based platforms.

Tools

file_search

// Tool for searching files uploaded by the user. // // To use this tool, you must send it a message. To set the tool as the recipient for your message, include this in your message header: to=file_search.<function_name> // // For example, to call file_search.msearch, you would use: // <|im_start|>assistant to=file_search.msearch code<|im_sep|>{"queries": ["first query", "second query"]}<|ghissue|> // // Note that the above must match exactly.