Apache Kafka vs. Oracle GoldenGate

For brokering messages, Confluent Kafka is well suited since it offers a managed solution ready to use. Scenarios where the solution is not very well suited are for example, where pricing is an issue. The solution costs quite a lot for basic usage (for example: for 3 clusters, pricing is above 100k$ a year).

Incentivized

Oracle [GoldenGate] is well suited for data replication where real-time results are important, such as data reporting. There is potential to have the users query the replicated database, which can have separate resources set up to be compressed or using lower performing or cheaper resources. As mentioned, columns with LOBs can become a headache, as they will need to be synchronized in their own group many times. Tables containing data in CHAR, VARCHAR, NUMBER, DATE, or other non-LOB columns, however, are able to be replicated quite seamlessly. [Oracle GoldenGate] does provide a way to review errors that are encountered, so that is a positive if their is scrutiny around data integrity.

Incentivized

• Apache Kafka is able to handle a large number of I/Os (writes) using 3-4 cheap servers.
• It scales very well over large workloads and can handle extreme-scale deployments (eg. Linkedin with 300 billion user events each day).
• The same Kafka setup can be used as a messaging bus, storage system or a log aggregator making it easy to maintain as one system feeding multiple applications.

Incentivized

• OGG can be used and integrated with heterogeneous databases as well as homogeneous databases.
• OGG is easy setup as well as easy learning curve
• OGG supports large databases
• OGG support many uses cases (topologies) such as unidirectional replication, bidirectional, peer-to-peer, Broadcast, Cascading replication etc.
• OGG can be used not only for CDC (change capture), can be used to full load

Incentivized

• The Kafka Tool is a community-made Java application that looks and feels from the past century.
• Logging can be confusing. This certainly shows when we have to do troubleshooting.
• Hybrid scenarios - pub/sub, but there are services in and outside a Kubernetes cluster. Then there are a ~3 options, but only 2 (the harder ones) are production-safe.

Incentivized

• The earlier versions of Golden Gate were complex to use as there was no front end. All that works was the command line.
• The cost of the product can be a detriment to most companies.
• The product has been augmented over the years and has so many features that it can be overwhelming.

Incentivized

Kafka has suited our use case very well so far. Going forward we are planning to expand our platform manifold so the load on Kafka and our reliance on Kafka is going to increase only.

Oracle GoldenGate is a core part of our business.

Incentivized

Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file

Incentivized

Once set up, it's very easy to use and keep running, it's getting to that point that can make it cumbersome to some. Also, depending on the data that you want to replicate, the configuration files can become quite cumbersome to maintain. Learning curve can be high for some who are not as experienced with databases and transactions.

Incentivized

Oracle GoldenGate has only gone down for us when the data being sent was bad.

Incentivized

Oracle GoldenGate's performance is great for our use cases. Your mileage may vary.

Incentivized

Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.

Incentivized

Oracle Support for Oracle GoldenGate has been quite responsive and quite helpful in the few situations where we've needed it. Furthermore, the documentation on Oracle GoldenGate is so good that we often do not need to contact support with issues as the fix is already documented and able to be run by us without needing to open a ticket.

Incentivized

We've had Oracle consultants come as well for training days to talk about new features, parts of Oracle GoldenGate we may not be using and things of that nature. The consultants they send are great as they're very knowledgeable about all things Oracle GoldenGate and great resources for any questions or concerns you may have with the product.

Incentivized

We used Oracle University for our Oracle Golden Gate Training and it was top notch. We were able to turn our whole DBA team to Oracle GoldenGate newbies to Oracle GoldenGate troubleshooting experts in a matter of a few days, while this obviously did not come cheap, the company felt that it was worth the investment.

Incentivized

If Oracle GoldenGate is new to your organization, expose as many DBAs as possible to it. Having your whole team fluent in it will overcome early operational hurdles and allow it to more quickly become an accepted and supported part of your supported platform for your team that will enable the business to use it to its fullest.

Incentivized

Apache Kafka is built for scale. From high throughput and real-time data streaming, it has a strong advantage over RabbitMQ with its low latency. This put Apache Kafka at the forefront as the platform of choice for large datasets messaging and ensuring scalability when data scale up tremendously. RabbitMQ however has its strengths in traditional messaging. Routing and message delivery reliability are the bedrock of RabbitMQ and this is where RabbitMQ excels. In my previous workplace, RabbitMQ was of choice as reliability matters more than scale. In two words. Apache Kafka for scale, RabbitMQ for reliability. And for cloud deployment and large dataset messaging in what I am doing now, Apache Kafka is the default choice.

Oracle is tightly integrated with Oracle Exadata and is a better fit for using Oracle databases.Both products support heterogeneous databases but using Exadata, it just seemed to make better sense to use the Oracle GoldenGate product. We also take advantage of the Oracle GoldenGate Veridata product to compare datasets and make sure they are still in sync.

Incentivized

Have never had any issues with scaling Oracle GoldenGate itself, however Oracle GoldenGate Monitor does have scaling issues, but with Oracle GoldenGate now able to be monitored by Oracle Enterprise Manager, this is no longer an issue, in my opinion.

Incentivized

• Positive: bursts of traffic on special holidays are easy to handle because Kafka can absorb and buffer all the messages we need to process long enough to let an understaffed set of back-end services catch up on processing. Hard to put a number to it but we probably save $5k a month having fewer machines running.
• Positive: makes decoupling the web and API services from the deeper back-end services easier by providing topics as an interface. This allowed us to split up our teams and have them develop independently of each other, speeding up software development.
• Negative: our engineers have made mistakes such as accidentally dropping a few thousand messages due to the CLI being confusing to use, and as a result a customer lost some of their precious data. I'd say that was more our fault than Kafka's though.

Incentivized

• Oracle GoldenGate guarantees our standing-by database in passive data center, is always in "pre-warmed" status, as the data is always in-sync with the master database in our active data center. At any time, the standing-by database can kick in in case the active DC encounters the serious issue. This significantly increases the IT application's high availability and promise to "business as usual".

Incentivized