dbt is an SQL development environment, developed by Fishtown Analytics, now known as dbt Labs. The vendor states that with dbt, analysts take ownership of the entire analytics engineering workflow, from writing data transformation code to deployment and documentation. dbt Core is distributed under the Apache 2.0 license, and paid Teams and Enterprise editions are available.
N/A
SAP PowerDesigner
Score 8.0 out of 10
N/A
SAP PowerDesigner (formerly from Sybase) is an enterprise data architecture modeling tool, used to Build a blueprint of the current enterprise architecture and visualize the impact of change before it happens.
dbt is very flexible and can fit into most data pipelines. This is a pro for most organizations that aren't fully bought into one platform (Google Cloud, etc.)
Matillion is graphical versus dbt, which is SQL code-based (that, of course, is a matter of personal preference and not an objective advantage). The integrated testing, documentation generation, lineage, etc., were additional criteria that led us to choose dbt.
I actually don't know what the alternative to dbt is. I'm sure one must exist other than more 'roll your own' options like Apache Airflow, say, bu tin terms of super easy managed/cloud data transforms, dbt really does seem to be THE tool to use. It's $50/month per dev, BUT …
Snaplogic is great at the Extraction and Load processes of ETL. It can pull data from anywhere, even behind firewalls. So if you need to get data from various APIs, databases, files, S3, SFTP, etc it is easy to do so. However, it requires special knowledge in order to build …
Most ETL pipeline products have a T layer, but dbt just does it better. The transformation is on steroids compared to the others. Also, just allows much more Adhoc solutions for very specific projects. Those ETL tools are probably better on the T part if you don't need too many …
Airflow can accomplish the same work as dbt (data build tool), however, dbt's (data build tool) development workflow and UI can open up data transformation and modeling work to non-data engineering teams. Looker might also be able to define data models via LookML with a …
Vice President, Chief Architect, Development Manager and Software Engineer
Chose SAP PowerDesigner
The version of ErWin we had didn't support the repository for document sharing and data dictionary sharing. Our version of PD does so we were able to leverage that and have a central repository that is shared among the team members. That really helped to give us consistency …
Oracle SQL Developer Data Modeler : Unfortunately this tool only supports Oracle Databases as a target database, but has many features similar to SAP Sybase PowerDesigner. ERWIN Data Modeler: Has some issues when switching from Conceptual Model to Physical Model, Impact …
dbt (Data Build Tool) is best suited for doing the data transformation. dbt is just a transformation tool and it is not suitable for building a data pipeline which requires extraction of data and loading. dbt is well suited for SQL based transformation logic and it is less appropriate when transformation logic requires python.
SAP PowerDesigner allows our team of data modelers to work and collaborate from a single repository and single data dictionary. This helps enforce consistency as data elements are referenced in other objects. Prior to our use of PD, we might have an element named "ppt" in one table, "participant" in another table and "part" in a third table. By forcing everything to be used from the data dictionary, we avoid these situations because everyone has to go to the dictionary. And we are able to easily do peer reviews on models before they are released because we are collaborating through the use of the repository.
Slow load times of the dbt cloud environment (they're working on it via a new UI though)
More out-of-the-box solutions for managing procedures, functions, etc would be nice to have, but honestly, it's pretty easy to figure out how to adapt dbt macros
dbt is very easy to use. Basically if you can write SQL, you will be able to use dbt to get what you need done. Of course more advanced users with more technical skills can do more things.
We did have to reach out to support to learn how to properly utilize the repository feature and share the data model across many developers. Support was able to help us get this set up correctly. The downside was it took us several weeks before we gave up and contacted support. We should have done that earlier. I would say, however, the documentation wasn't clear on how to do this. So support was a great big help!
Matillion is graphical versus dbt, which is SQL code-based (that, of course, is a matter of personal preference and not an objective advantage). The integrated testing, documentation generation, lineage, etc., were additional criteria that led us to choose dbt.
The version of ErWin we had didn't support the repository for document sharing and data dictionary sharing. Our version of PD does so we were able to leverage that and have a central repository that is shared among the team members. That really helped to give us consistency across our databases. PD is easy to use, but getting the repository set up properly was a little tricky.