AWS Glue vs. Pentaho

When the data which requires ETL has different formats, schema, and volume, this service suits them best. So, when the volume is not consistent (typical use-case of healthcare and online shopping), AWS Glue can be the prime choice. When the data is available in both batch and streaming mode, the developer needs to generate a separate codebase. This increases the source code management efforts. So, prefer to go with Glue when the nature of the data is the same (either batched or streamed).

Incentivized

Pentaho is very well suited to perform data extraction & data mining from various cloud storage & transform that data using various available data models. However, the software struggles when it comes to visualizing the extracted data in an appealing manner & can be difficult for end-users to get an understanding of data tables created using those models.

Incentivized

• After data cleansing, the team also implemented the best practices for using AWS platform services as a Data Lake, such as job bookmarking for AWS Glue jobs, proper delimiter for the AWS Glue crawlers, partitioning in AWS S3, and transformation to parquet file for compression and faster querying time in Amazon Athena.
• Data modernization through combining data from multiple sources into a functioning datasets, rebuilding DW, and resctructuring data sources.
• Aims to lessen customer complaints, eliminate manual data extraction requests via SR from different data sources, and Increase accuracy, consistency and speed up reconciliation process.

Incentivized

• Integrate and synchronize with big data easily
• Import data from any sources and different databases
• Managing data in on-premise, hybrid and cloud environments.
• Compatibility and flexibility of the platform with any type of scenario and any business or industry
• Various tools in the software suite to transformation of data
• Simple interface appearance and creative UI graphics

• It’s integration with other cloud vendors is bit difficult
• If it can support non SQL based databases as well, it would be powerful.
• Real time data synchronisation in data source is missing

Incentivized

• I think the relative obscurity of the tool is a downside, not as many developers, consultants or peers you can tap into.
• Lack of a solid user community held us back, looking at Power BI and Qlik, they have huge user communities that help each other out. Would have liked that here.
• Smaller company means smaller sales force, and the lack of a local presence made it hard to only interact online with the account rep. Other companies have someone local who often stops by with pre-sales developers to just pitch in free of charge when they have time.

Incentivized

I will use Pentaho until I find a better tool with a better, easier to use report designer client. For now, Pentaho has been the most powerful reporting tool for our clients because of its ability to connect to Odoo, integrate in Odoo (reports are accessible in Odoo) and the flexibility in report design and parameter integration

Incentivized

I personally found it very usable for a data engineer's day job, particularly for performing ETL and managing the data pipelines.

Incentivized

Even if Pentaho requires less technical skills to develop a pipeline or ETL project, its learning curve can be a bit slow since there are many ways to do the same thing as in any other platform. However, in Pentaho, some things can be confusing some moments for non-technical teams.

Incentivized

Amazon responds in good time once the ticket has been generated but needs to generate tickets frequent because very few sample codes are available, and it's not cover all the scenarios.

Incentivized

We are an Enterprise customer. They handle problems INSTANTLY when they are critical, including initiation an immediate WebEx screen share call when needed. Smaller/less-critical problems are handled within 1-2 days -- and NEVER fall off their radar, no matter how small. As needed, we can also leverage "professional services" from them -- much of which is included in our Enterprise contract. Finally, when a problem I have discovered turns out to be a bug..they create a JIRA for the fix, and make me a watcher. I love seeing notes come in showing me status updates of bugs filed because of something I found. They really are TOP-NOTCH.

Course Taken: DI1000 Pentaho Data Integration Fundamentals Setup A week before your class started, the instructor will start sending out class material and lab setup instructions. This is helpful so that you understand how the environment is laid out and can start reviewing the content. Ultimately it saved about a 1/2 day trying to setup with 10 other people online which was great! The Course The 3-day course was laid out like many other technical classes with 15-30 minutes instruction and 15-60 minutes of lab exercises. The instructor was very knowledgeable with the functionality from version to version and answered questions as we went along. I was amazed at some of the functionality that was available that I was not using at the time and quickly implemented changes to many existing transformations and jobs. The novice users seemed to catch on quickly and more experienced users explained how some of the functionality was used in their home environments. Towards the end there was enough time so that we were able to ask very directed questions about our own environments. Overall, I really found the class to be informative and deliver enough information to be dangerous. My skills improved and I was able to design better and efficient transformations for the HIE. Course Description: https://training.pentaho.com/instructor-led-training/pentaho-data-integration-fundamentals-di1000

Get the right people in before starting implementation. Start small and build as you go approach is time consuming and involves lot of rework. Evangalize within the organization the capabilities and limitations equally so that correct delivery expectations are set. Set expectations with the Customer that the tool cannot replace proprietary software in terms of stability/usability and that timelines could change given the new ness of the product.

Incentivized

The cataloging of data objects is the best in the case of AWS Glue. We use AWS Glue in all of our data pipelines to sync external and internal data sources and to automatically produce SQL-based ETL based on AWS Glue catalog objects. Integration with Amazon products is the other advantage.

Incentivized

Perhaps Snowflake and SalesForce have some components which align with the Pentaho tools. The Pentaho tools have integrations with these technologies to add more value to the final users. Perhaps the only weakness I can honestly find in the Pentaho tools right now is the lack of a powerful web interface for data transformations. There is a web component from which you can access existing data transformations created with the Pentaho Data Integration tool. Still, the web component only allows visualization of the data transformation and remote execution. A complete web interface with remote execution would be excellent, and I'm sure that we might see something like this available at some point in the future.

Incentivized

• Positive Impact :- after ETL we can able to do some kind of automation
• Negative :- At some point of time it can hamper the cost but not really

Incentivized

• Pentaho has improved our overall business process.
• Pentaho has helped the Managers and Directors to analyze the numbers going up and down from time to time.
• We have a started a big project using Pentaho that is going to include all the business processes in the organization.

Incentivized