Cloud Engineer - Software Engineer - Analyst

Who We Look For:

Goldman Sachs Engineers are innovators and problem-solvers who thrive in fast-paced global environments. We are seeking a motivated Cloud Engineer to support the WM Data Engineering ecosystem. In this role, you will be a key contributor to the migration and modernization of our on-premises legacy data pipelines and services to AWS. You will work at the intersection of software engineering and data architecture, translating technical blueprints into high-performance code. Your mission is to build secure, accessible, and cost-optimized data assets that power real-time client insights and advanced analytics in a cloud-native environment.

Key Responsibilities:

1. Data Pipeline Development & Migration:
   • ETL/ELT Execution: Build and maintain scalable data pipelines using AWS Glue, Amazon EMR, and Snowflake, transitioning legacy on-premises workloads to modern cloud-native architectures.
   • Microservices Transition: Execute the migration of on-premises microservices to AWS by containerizing workloads with Docker and deploying them to Amazon ECS.
   • Schema Evolution: Implement and manage open table formats, specifically Apache Iceberg, to ensure high-performance analytics and seamless schema evolution across the WM data lake.
2. Orchestration & Automation:
   • Workflow Management: Develop and schedule complex data workflows using Apache Airflow (MWAA) or AWS Step Functions, ensuring robust error handling and retry logic.
   • Infrastructure as Code (IaC): Deploy and manage cloud infrastructure using Terraform or AWS CDK, adhering to the "Infrastructure as Code" philosophy for all deployments.
   • CI/CD Integration: Maintain automated deployment pipelines to ensure consistent and auditable code promotion across development, UAT, and production environments.
3. Data Quality & Governance:
   • Automated Testing: Implement data validation to ensure data integrity and accuracy during and after the migration process.
   • Observability: Build monitoring dashboards and alerting mechanisms to track pipeline health, data latency, and SLA adherence.
4. Modernization:
   • Migration Support: Contribute to the migration of on-premises data workloads to AWS.
   • AI/ML Readiness: Help build the data foundations required for predictive modeling and generative AI applications.

Qualifications:

Technical Requirements

• Experience: 2+ years of hands-on experience in Data Engineering or Software Engineering, with a focus on cloud-based data solutions.
• Technical Skills:
  • Experience with modern data platforms like Snowflake and cloud-native AWS services.
  • Understanding of open-source table formats, specifically Apache Iceberg.
  • Proficiency in Java, Python, and SQL.
  • Hands-on experience with Spark, Kafka, and orchestration tools like Apache Airflow, Dagster, or dbt.
• Soft Skills: Strong problem-solving "builder" mindset and the ability to communicate technical concepts within a team environment.

Education

• Bachelor’s or Master’s degree in computer science, Engineering, Mathematics, or a related field.