AWS Glue vs. Toad Data Point

When the data which requires ETL has different formats, schema, and volume, this service suits them best. So, when the volume is not consistent (typical use-case of healthcare and online shopping), AWS Glue can be the prime choice. When the data is available in both batch and streaming mode, the developer needs to generate a separate codebase. This increases the source code management efforts. So, prefer to go with Glue when the nature of the data is the same (either batched or streamed).

Incentivized

Toad Data Point is the clear tool of choice if the end-user is interested in reports that are relatively simple to build using SQL code and export to Excel. It is less useful if the analyst also needs to run statistical models on the data and visualize the data for those functions I usually use RStudio or Jupyter Notebook which incorporates those features much more seamlessly.

Incentivized

• After data cleansing, the team also implemented the best practices for using AWS platform services as a Data Lake, such as job bookmarking for AWS Glue jobs, proper delimiter for the AWS Glue crawlers, partitioning in AWS S3, and transformation to parquet file for compression and faster querying time in Amazon Athena.
• Data modernization through combining data from multiple sources into a functioning datasets, rebuilding DW, and resctructuring data sources.
• Aims to lessen customer complaints, eliminate manual data extraction requests via SR from different data sources, and Increase accuracy, consistency and speed up reconciliation process.

Incentivized

• Ease of use. It is a very easy product to use without any particular training.
• Multiple data source connection. It easily connects to multiple data sources which allows you to use different data sources to run reports.
• Seamless application. Once you set everything up-- whether it is an FTP or export of data-- you can set it and forget it!

Incentivized

• It’s integration with other cloud vendors is bit difficult
• If it can support non SQL based databases as well, it would be powerful.
• Real time data synchronisation in data source is missing

Incentivized

• The program can error out when reconnecting to a database from a timeout but that is easily fixed by restarting the program.
• If you aren't familiar with database tools the UI can be overwhelming to some.

Incentivized

I personally found it very usable for a data engineer's day job, particularly for performing ETL and managing the data pipelines.

Incentivized

I find Toad Data Point easy to use for both the novice and the experienced business analyst. If all you desire is to access data and create spreadsheets...this is a snap. Toad Data Point actually has cool data analysis features built into it. The newer workflow interface makes automating steps a snap

Incentivized

Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.

Incentivized

The cataloging of data objects is the best in the case of AWS Glue. We use AWS Glue in all of our data pipelines to sync external and internal data sources and to automatically produce SQL-based ETL based on AWS Glue catalog objects. Integration with Amazon products is the other advantage.

Incentivized

TOAD excels at connecting to divergent data sources, but appears geared more to DBAs than to regular query users. Microsoft's offerings excel against Microsoft SQL Server, but sometimes struggle with other data sources. However, SSMS and vs code excel at many developer productivity/workflow enhancements. vs code, in particular, has a lively extension system that allows it to be tailored for development/querying/model building/etc. That flexibility comes at a cost - the learning curve is steep for new users. The tradeoff between complexity and power may not be good for some environments/users/situations.

Incentivized

• Positive Impact :- after ETL we can able to do some kind of automation
• Negative :- At some point of time it can hamper the cost but not really

Incentivized

• It is the least common denominator - not particularly optimized for our environment or workflows.
• Hangs or slowdowns add anywhere from 5% - 7% for projects utilizing large/complicated data setts. (This could be due to other IT-imposed constraints and not entirely due to TOAD.)
• Trying to perform some operations requires reading documentation and experimenting in order to figure out the TOAD-specific approaches and commands.
• It just works (when we understand it). Updates don't break things and things don't suddenly start behaving differently. Best of all, we don't mysteriously lose functionality.

Incentivized