WM Data Engineering – Senior Cloud Data Engineer - Vice President

Who We Look For:

Goldman Sachs Engineers are innovators and problem-solvers, building solutions for various divisions. We look for creative collaborators who evolve, adapt to change and thrive in a fast-paced global environment.

We are seeking a high-caliber, hands-on Senior Cloud Data Engineer. While you will provide architectural guidance, your primary impact will come from hands-on engineering: building production-ready data pipelines, containerizing microservices for Amazon ECS, and executing the technical migration of legacy on-premises systems to AWS.

Key Responsibilities:

1. Hands-on Pipeline & Microservices Migration:
   • Active Migration Execution: Directly execute the migration of legacy ETL and microservices to AWS. This includes refactoring monolithic code into containerized services and deploying them to Amazon ECS (Fargate/EC2).
   • Containerization & Orchestration: Build and maintain Docker images, write complex ECS Task Definitions, and configure service-to-service communication using Amazon ECS Service Connect and AWS Cloud Map.
   • Data Pipeline Engineering: Develop end-to-end data flows using AWS Glue (PySpark), Amazon EMR, and Snowflake. Implement "Lakehouse" patterns using Apache Iceberg to ensure data portability.
2. Infrastructure & Automation-as-Code
   • IaC Development: Write and maintain production-grade Terraform or AWS CDK modules to provision VPCs, ECS clusters, and RDS instances. Ensure all infrastructure is version-controlled and deployed via GitHub Actions or GitLab CI.
   • AI-Augmented Coding: Actively use AI coding assistants (e.g., GitHub Copilot) to refactor legacy SQL, generate unit tests, and automate the creation of boilerplate pipeline code.
   • Toil Reduction: Identify manual bottlenecks in the migration process and build custom automation tools in Python or Go to streamline data validation and schema conversion.
3. Technical Leadership & Reliability
   • Code Reviews & Standards: Lead rigorous peer code reviews, enforcing standards for performance, security (IAM least privilege), and maintainability.
   • Observability Implementation: Hands-on configuration of Amazon CloudWatch Container Insights, and OpenTelemetry to ensure deep visibility into migrated microservices and data jobs.
   • Performance Tuning: Directly optimize Spark job configurations, Snowflake warehouse sizing, and ECS auto-scaling policies to balance performance.

Qualifications:

Technical Requirements

• Experience: 8+ years of hands-on experience in Data Engineering and Cloud Infrastructure, with a focus on building and migrating production workloads.
• AWS ECS Expertise: Deep technical expertise in Amazon ECS (Fargate/EC2), including networking (ALB/NLB), task placement strategies, and container security.
• Data Platform Expertise: Proven experience with modern data platforms such as Snowflake (AI Data Cloud) and cloud-native services. Good understanding of open-source table formats, specifically Apache Iceberg, to enable interoperability, schema evolution, and high-performance analytics across multiple engines.
• Programming: Expert-level proficiency in Java, Python and SQL.
• Big Data & Orchestration: Hands-on experience with Spark, Kafka, and orchestration tools like Apache Airflow, Dagster, or dbt.
• Data Modeling: Deep understanding of data warehousing and modern data lakehouse architecture.

Leadership & Soft Skills

• Mentorship: Proven track record of upskilling junior engineers.
• Communication: Ability to explain complex technical concepts to non-technical stakeholders in the wealth management business.
• Problem Solving: A "builder" mindset with the ability to navigate ambiguity in a fast-paced environment.

Education

• Bachelor’s or Master’s degree in computer science, Engineering, Mathematics, or a related field.