googlecloudplatform-test: Big data is easier than ever with Google Cloud Dataflow

Google Cloud Platform Blog

Product updates, customer stories, and tips and tricks on Google Cloud Platform

Big data is easier than ever with Google Cloud Dataflow

April 16, 2015

MapReduceFlumeJavaMillWheelGoogle Cloud DataflowGoogle Cloud Platform big data servicesthe cloud wayGoogle Cloud Dataflow available in betaGoogle Cloud Platform

Merge your batch and stream processing pipelines thanks to a unified and convenient programming model. The model and the underlying managed service let you easily express data processing pipelines, make powerful decisions, obtain insights and eliminate the switching cost between batch and continuous stream processing.

Finely tune the desired correctness model for your data processing needs through powerful API primitives for handling late arriving data. You can process data based on event time as well as clock time and gracefully deal with upstream data latency when processing data from unbounded sources.

Leverage a fully-managed service, complete with dynamically adaptive auto-scaling and auto-tuning, that offers attractive performance out of the box. Whether you’re a developer or systems operator, you no longer need to invest time worrying about resource provisioning or attempting to optimize resource usage. Automation, a fully managed service, and the programming model work together to significantly lower both CAPEX and OPEX.

Enjoy reduced complexity of managing and debugging highly parallelized processes with a simplified monitoring interface that’s logically mapped to your processing logic as opposed to how your code’s mapped to the underlying execution plane.

Benefit from integrated processing of data across the Google Cloud Platform with optimized support for services such as Google Cloud Storage, Google Cloud Datastore, Google Cloud Pub/Sub, and Google BigQuery.