Google Cloud Pub/Sub

Score9 out of 10

24 Reviews and Ratings

What is Google Cloud Pub/Sub?

Google offers Cloud Pub/Sub, a managed message oriented middleware supporting many-to-many asynchronous messaging between applications.

Categories & Use Cases

Message Oriented Middleware

Cézar Augusto Nascimento e Silva View profile

Solutions Architect, Tech Lead, Software Engineer in Information Technology at Stilingue (201-500 employees employees)

Use Cases and Deployment Scope

We used Google Cloud Pub/Sub to solve ETL/Streaming and real-time processing problems for high volumes of data. We used it either to fill datalakes, process and store in warehouses or data marts and also for processing events, either using JSON or protobuf.

This was integrated for many languages such as python, java, golang and kotlin. We had configured kubernetes auto scaling system based on some Google Cloud Pub/Sub metrics which worked very well. The main observed metrics for alerts and overall health indicator of our systems were both the size of each queue and the oldest message in queue, either indicating a high volume jam or some random specific error for a single message, respectively.

We had to handle idempotency since a duplicated message delivery is a possibility, this was usually paired and a Redis Cache to guarantee idempotency for a reasonable time window.

Pros

Data Streaming
Even Sourcing
Protobuf message format
Scalability
Easy to Use
Observability
Integrated Dead Letter Queue (DLQ) functionality

Cons

Deliver Once (idempotency) - currently in preview
Vendor locked to Google

Most Important Features

DLQ (Dead Letter Queues)
Scalability
Delivering backoff for failed messages

Return on Investment

Scalable System
Better Alerts (observability)
Auto Scaling

Alternatives Considered

Apache Kafka

Other Software Used

MongoDB, HashiCorp Consul, HashiCorp Vault, Istio

Usability

Mehdi Bettiche

Senior Consultant in Information Technology at Wavestone (1001-5000 employees employees)

Use Cases and Deployment Scope

We mainly use Google Cloud Pub/Sub with cloud functions and google scheduler to handle FinOps subjects. For example :
- Stopping a Google Kubernetes Engine (GKE) using a cloud scheduler that publishes to a pub/sub topic which then triggers a python function that changes GKE configurations.
- Stopping a Google SQL database using Google scheduler and Google Cloud Pub/Sub.

Pros

Pub/Sub UI
Pub/Sub subscriptions & message retrieval.
Integration with BigQuery & Cloud Storage.

Cons

Debugging
Error screening

Most Important Features

Handling messages with a managed solution.
Integration with other GCP services.

Return on Investment

Since it integrates in our FinOps solution, we have positive impact on the billing.

Alternatives Considered

Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS) and RabbitMQ

Other Software Used

Google Kubernetes Engine, Google Cloud SQL, Google Cloud Functions

Cameron Gable View profile

Software Engineer in Research & Development at Serimmune Inc. (11-50 employees employees)

Pros

A hands-off approach to publishing messages and subscribing to topics.
Easy to use APIs.
Useful, simple UI on cloud console to send messages for debugging, etc.

Cons

I can't think of missing functionality, but added functionality is always nice!

Return on Investment

Allowed the use of specialized VMs
Saves costs and time

Alternatives Considered

Google Compute Engine

Other Software Used

Google Cloud Storage, Stackdriver, Google Cloud Dataflow, Google Cloud SQL

Eric Mason

Lead Developer in Engineering at SaaSquatch (11-50 employees employees)

Pros

Highly scalable.
Queues messages for up to 7 days if not acknowledged.

Cons

Would be nice if the queue could be extended beyond 7 days.
We found it a bit tricky replay unacknowledged messages when needed.

Return on Investment

Pub/Sub has helped avoid data loss improving our customer value prop.
Pub/Sub has reduced development time that would otherwise be needed to build a highly scalable queue.

Alternatives Considered

RabbitMQ, Apache Kafka and Amazon Kinesis

Other Software Used

MongoDB, Redis, Google Cloud Storage

Verified User

Director in Engineering (501-1000 employees employees)

Use Cases and Deployment Scope

Fully managed messaging middleware useful for service integration. We have mainly used it for large scale logs from Kubernetes clusters. Pub/Sub proved to be easy to use, with a clear, user-friendly UI, well documented SDK, built-in elasticity and was the foundation for a real time data management system. Overall, very pleased with Google Cloud Pub/Sub.

Pros

Messaging
Scalable
No-ops
Secure

Cons

Only usable within GCP

Most Important Features

Elasticity
Ease of use
Security

Return on Investment

Positive ROI

Alternatives Considered

Amazon Managed Streaming for Apache Kafka (Amazon MSK), Apache Kafka and Amazon Simple Notification Service (SNS)

Other Software Used

Amazon SageMaker, Amazon Forecast, Google BigQuery