Migration AWS data platform to Google Cloud

INITIAL SITUATION

Shift infrastructure as well as several existing data products from AWS and Cloudera to Google Cloud

SOLUTION

The goal is to shift a data platform from AWS to Google Cloud. Infrastructure is built up via Infrastructure as Code by terraform / terragrunt. Several data products have been shifted to Google Cloud native services: Data Pipelines based on Apache Spark, SQL-Workloads, Cloudera Pipelines which now are available within Google BigQuery.

BUSINESS VALUE

AWS could be decomissioned after 1 year

​

FRAMEWORK & TOOLS

Google BigQuery, Google Cloud Composer, Google Cloud IAM, Google Cloud SQL, Google Cloud Storage, Google Compute Engine, Google DataProc, git, GitLab, CI / CD, Scala, Python, Docker, SQL, IaC (Terraform, Terragrunt)