AWS Glue vs. RapidMiner

When the data which requires ETL has different formats, schema, and volume, this service suits them best. So, when the volume is not consistent (typical use-case of healthcare and online shopping), AWS Glue can be the prime choice. When the data is available in both batch and streaming mode, the developer needs to generate a separate codebase. This increases the source code management efforts. So, prefer to go with Glue when the nature of the data is the same (either batched or streamed).

Incentivized

RapidMiner is the best tool to build models on textual data. It is rich in ML algorithms and reduces the need to manually tune the parameters. It automatically optimizes them, thus providing a better solution. RapidMiner again extends great capability for data preparation, its insane connections to almost every data source pulls in the data easily into one environment. And it can comfortably perform data cleaning and process tasks over that. RapidMiner is not so good with image, audio or video data. These data points cannot be used directly in their raw form. They must be transformed into some intermediate form for performing analytics over it. Moreover, there are no connectors to directly pull data from their varied sources. For example, we don't have a connector to read audio data directly from a switch and then convert it to text (although Google speech API is available for audio to text conversion.)

Incentivized

• After data cleansing, the team also implemented the best practices for using AWS platform services as a Data Lake, such as job bookmarking for AWS Glue jobs, proper delimiter for the AWS Glue crawlers, partitioning in AWS S3, and transformation to parquet file for compression and faster querying time in Amazon Athena.
• Data modernization through combining data from multiple sources into a functioning datasets, rebuilding DW, and resctructuring data sources.
• Aims to lessen customer complaints, eliminate manual data extraction requests via SR from different data sources, and Increase accuracy, consistency and speed up reconciliation process.

Incentivized

• RapidMiner Studio offers a superb user interface with an intuitive workflow paradigm that is very easy to learn.
• RapidMiner Studio’s operators make it a complete and powerful tool for data preprocessing, data visualization, and data mining/analytics.
• RapidMiner Studio provides excellent documentation, countless worked examples, training and support via a large user community.
• Every problem is solved using a sequence of operators.
• Statistical analysis capabilities offered with the T-Test, ANOVA, Grouped ANOVA, and ANOVA Matrix operators.
• Textual data mining operators.
• Web-based and cloud computing capabilities.
• Visualization capabilities.
• Marketplace Extensions – especially Finance And Economics.
• Process portability.

Incentivized

• It’s integration with other cloud vendors is bit difficult
• If it can support non SQL based databases as well, it would be powerful.
• Real time data synchronisation in data source is missing

Incentivized

• Wish the tool was more efficient in terms of processing power. The tool takes a lot of CPU processing power, even for a small process on a small data set
• Wish there were more options on charts and graphs to visualize the data

Incentivized

Very fast and user-friendly tool

Incentivized

I personally found it very usable for a data engineer's day job, particularly for performing ETL and managing the data pipelines.

Incentivized

Very use to use and learn

Incentivized

Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.

Incentivized

The cataloging of data objects is the best in the case of AWS Glue. We use AWS Glue in all of our data pipelines to sync external and internal data sources and to automatically produce SQL-based ETL based on AWS Glue catalog objects. Integration with Amazon products is the other advantage.

Incentivized

The other product like RapidMiner Studio that I have used is WEKA. I decided to use RapidMiner because almost all modelling methods and feature selection methods from the Weka machine learning library are available within RapidMiner. Furthermore, RapidMiner Studio is a visual workflow and therefore it is easier to demonstrate and visualise the processes involves in getting the desired results. Visualization of workflow enhances teaching and learning. RapidMiner is rich with algorithms and online learning materials that can assist students in their self-directed learning on data preparation, machine learning, deep learning, text mining, and predictive analytics. Moreover, RapidMiner repository has more than 1500 machine learning algorithms and functions that students can explore for any case study and assignments. The RapidMIner is also an open platform that can seamlessly integrates with other applications programmed with other programming languages like R and Python.

Incentivized

• Positive Impact :- after ETL we can able to do some kind of automation
• Negative :- At some point of time it can hamper the cost but not really

Incentivized

• We saved over $100k on our direct mail program by not mailing to those unlikely to respond to our mailings based on our predictive analysis.
• Our CX team has saved countless hours by automating call scripts to isolate key phrases and code each call.

Incentivized