Modern data teams rely on well-designed pipelines to keep analytics and machine learning systems accurate and up to date. This course will teach you how to build scalable, reliable batch and streaming data pipelines on Google Cloud.
What you'll learn
Organizations increasingly struggle to manage diverse data sources, real-time ingestion needs, and growing expectations for clean, reliable data—making well-designed pipelines essential. In this course, (GCP-PDE) Ingesting and Processing the Data, you'll gain the ability to build scalable batch and streaming pipelines that ensure data quality and readiness for downstream analytics. First, you'll explore how to plan data pipelines by understanding sources, sinks, transformation logic, networking fundamentals, and encryption strategies. Next, you'll discover how to build these pipelines using Dataflow, Pub/Sub, BigQuery, Cloud Data Fusion, and other Google Cloud services, including both code and no-code approaches. Finally, you'll learn how to deploy and operationalize your pipelines with Cloud Composer, Cloud Workflows, CI/CD, and infrastructure as code. When you're finished with this course, you'll have the skills and knowledge of Google Cloud data ingestion and processing needed to create reliable, scalable, production-ready pipelines and tackle this subject on the Professional Data Engineer certification.