Apache Hive vs. Azure Synapse Analytics

Apache Hive shines for ad-hoc analysis and plugging into BI tools. Its SQL-like syntax allows for ease of use not for only for engineers but also for data analysts. Through our experience, there are probably more desirable tools to use if you are planning on integrating Hive into your processing pipeline.

Incentivized

In terms of a well-suited scenario - the Azure Synapse can be used to capture data from multiple sources (especially from onPrem sources apart from Dataverse) and update the transformed data based on the given conditions (eg: refresh data based on the specified date/time ranges). Also, the transformed data can simply be transferred to Azure Data Lake for further processing by utilizing other analytics tools such as PowerBI.

Incentivized

• Hive syntax is almost like SQL, so for someone already familiar with SQL it takes almost no effort to pick up Hive.
• To be able to run map reduce jobs using json parsing and generate dynamic partitions in parquet file format.
• Simplifies your experience with Hadoop especially for non-technical/coding partners.

Incentivized

• The combination of SQL/unstructured data
• Keeping things "complicated, but simple"; [heterogeneous] data formats seen as just SQL tables to business experts used to use Power BI, Excel, and any other traditional SQL-oriented BI tools
• Integration options using "Synapse pipelines", the application of ADFs
• The greatly integrated solution of independent things (Spark MPP cluster, MPP SQL Servers, ADFs) - all sitting under one roof. Great job!
• Integration with super-fast, globally replicated data. I really appreciate the integration of NoSQL databases (namely Core API and Mongo API under Cosmos DB) with purely batch-processed BI data

Incentivized

• Use Hive for analytical work loads. Write once and read many scenarios. Do not prefer updates and deletes.
• Behind scenes Hive creates map reduce jobs. Hive performance is slow compared to Apache Spark.
• Map reduce writes the intermediate outputs to dial whereas Spark operates in in-memory and uses DAG.

Incentivized

• With Azure, it's always the same issue, too many moving parts doing similar things with no specialisation. ADF, Fabric Data Factory and Synapse pipeline serve the same purpose. Same goes for Fabric Warehouse and Synapse SQL pools.
• Could do better with serverless workloads considering the competition from databricks and its own fabric warehouse
• Synapse pipelines is a replica of Azure Data Factory with no tight integration with Synapse and to a surprise, with missing features from ADF. Integration of warehouse can be improved with in environment ETl tools

Incentivized

Since I do not know the second data warehouse solution that integrate with HDFS as well as Hive.

Hive is a very good big data analysis and ad-hoc query platform, which supports scaling also. The BI processes can be easily integrated with Hadoop via the Hive. It can deal with a much larger data set that traditional RDBMS can not. It is a "must-have" component of the big data domain.

Incentivized

The data warehouse portion is very much like old style on-prem SQL server, so most SQL skills one has mastered carry over easily. Azure Data Factory has an easy drag and drop system which allows quick building of pipelines with minimal coding. The Spark portion is the only really complex portion, but if there's an in-house python expert, then the Spark portion is also quiet useable.

Incentivized

Apache Hive is a FOSS project and its open source. We need not definitely comment on anything about the support of open source and its developer community. But, it has got tremendous developer support, awesome documentation. I would justify the fact that much support can be gathered from the community backup.

Incentivized

Microsoft does its best to support Synapse. More and more articles are being added to the documentation, providing more useful information on best utilizing its features. The examples provided work well for basic knowledge, but more complex examples should be added to further assist in discovering the vast abilities that the system has.

Incentivized

We have used a simple but necessary function such as merging certain data tables, which although they may be from different areas, complement each other or are necessary, you can use metadata if what you need is to validate the origin of your information and what impact it has, is also feasible.

Incentivized

They're all part of the Microsoft Azure family, so they are not exactly competitors. They overlap in functionality, but they're targeted at different levels of customers. Azure Data Factory is an excellent stand-alone PaaS (included in Synapse Analytics) for writing, scheduling, and monitoring pipelines. Azure SQL Database (and all the Azure SQL family) is excellent for traditional, SQL-based data warehouses, especially if you're migrating from on-premises. Combined with Azure Data Factory (that can run SSIS packages), it's a perfect solution for a simple path to the cloud. Azure Databricks is effectively the only internal "competitor" to Synapse Analytics but targeted more to a "platform-agnostic" audience. On the other hand, Synapse is more of a proprietary mix of products that are more tightly related to Microsoft technologies.

Incentivized

• Good ROI for being able to access data easily across the network, we have large amounts of data and this is a good system to access it
• Good ROI for being easy to learn how to use for new employees, not much time spent which saves costs
• Good ROI for being able to integrate with Spark and other applications, hence data can be analyzed through programs

Incentivized

• It definitely has a positive impact on ROI. We are able to use it to generate MORE revenue through predictive analytics and pricing optimization.
• Because of the SQL Data Warehouse design, we're able to set up some self service reporting tools which allow our users to generate reports ad hoc instead of having a full time employee creating these by hand.
• Having visibility into the data is very useful for management to make good business decisions.

Incentivized