AWS Glue vs. Paxata

When the data which requires ETL has different formats, schema, and volume, this service suits them best. So, when the volume is not consistent (typical use-case of healthcare and online shopping), AWS Glue can be the prime choice. When the data is available in both batch and streaming mode, the developer needs to generate a separate codebase. This increases the source code management efforts. So, prefer to go with Glue when the nature of the data is the same (either batched or streamed).

Incentivized

Paxata can be highly useful to someone who doesn't like/have any experience with writing codes to treat data before using it as input into BI dashboards. Paxata can accelerate data cleaning in environments where a large amount of unclean data is generated and business decisions on the go are required. It performs really well while dealing with natural language.

Incentivized

• After data cleansing, the team also implemented the best practices for using AWS platform services as a Data Lake, such as job bookmarking for AWS Glue jobs, proper delimiter for the AWS Glue crawlers, partitioning in AWS S3, and transformation to parquet file for compression and faster querying time in Amazon Athena.
• Data modernization through combining data from multiple sources into a functioning datasets, rebuilding DW, and resctructuring data sources.
• Aims to lessen customer complaints, eliminate manual data extraction requests via SR from different data sources, and Increase accuracy, consistency and speed up reconciliation process.

Incentivized

• Visualize distributions in large data sets effectively which enable the user to quickly spot outliers and treat them appropriately
• Provides recommendation to merge datasets based on matching column values
• The cluster and edit feature in my opinion is its most powerful feature and reduces cardinality in column with text

Incentivized

• It’s integration with other cloud vendors is bit difficult
• If it can support non SQL based databases as well, it would be powerful.
• Real time data synchronisation in data source is missing

Incentivized

• Doesn't provide recommendation on how to impute values
• There is a lag quite often
• We can say whether a column has errors or quality issues in the first look

Incentivized

I personally found it very usable for a data engineer's day job, particularly for performing ETL and managing the data pipelines.

Incentivized

Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.

Incentivized

The cataloging of data objects is the best in the case of AWS Glue. We use AWS Glue in all of our data pipelines to sync external and internal data sources and to automatically produce SQL-based ETL based on AWS Glue catalog objects. Integration with Amazon products is the other advantage.

Incentivized

Paxata is a much better tool when it comes to handling natural language but Talend provides recommendations on how to impute missing values and outliers. Paxata provides recommendations on dataset tie-ups and joins but Talend doesn't provide any such recommendations. In paxata you can visualize distribution of data in a column and filter them by dragging and selecting the section you'd like to retain

Incentivized

• Positive Impact :- after ETL we can able to do some kind of automation
• Negative :- At some point of time it can hamper the cost but not really

Incentivized

• It saves time to clean data
• It reduces the requirement of too many data engineer/stewards and hence adds positive impact on the return of the business

Incentivized