Databricks Data Intelligence Platform

Databricks is the primary data platform where we land, standardize, clean, transform, and clean our data sources. We utilize the Workflows feature to automate reoccurring tasks and have built internal applications around the reusable workflows. We use the dashboard feature internally to allow customer success teams and business analysts to keep tabs on the performance and outputs of our products. The workloads are orchestrated in Databricks but executed within our own AWS accounts, allowing us to stay compliant with our stringent security requirements.

• Greater democratization to data sources.
• Migration took a while, as we were largely a Pandas shop.

Snowflake

Notion, Datadog

I use Databricks Lakehouse Platform in my Data Scienc & AI consulting company to help various business entities with data-driven solutions. The platform can handle large and complex data sets and enable us to build and deploy applications using the latest technologies. The opennness of Databricks allows us to seamlessly integrate and adapt to our clients requirements :

* Creating dashboards with Tableau, Redash, Qlik,

* Feed their CRM tool like Salesforce, SAP,

* developing chatbots for Knowledge Management

* Serve ML models behind API endpoints.

Databricks Lakehouse Platform is a versatile and open product that saves us a lot of time, help us control cloud cost and human resources energy !

• Unity Catalog
• Collaborative Spark Notebook supporting python, SQL, Scala, R
• Serverless Endpoints
• mlflow integration

• Data Science environment is ready in a matter of minutes, not days.
• Much better cost control
• Easy onboarding for all clouds

Azure Synapse Analytics and Snowflake

Bitbucket, Azure Data Factory, Tableau Desktop

• Data Warehouse
• Spark computations
• Allows SQL, Scala and R to collaborate on notebooks

• A comprehensive data warehouse for transactions and calculations.
• Cost effective on just using one tool that does most we ask for.
• Fast business insights with data availability

Snowflake

Microsoft Azure, Sift Digital Trust & Safety Suite, Automation Anywhere

This product is used for Data Science project development, from data analysis/wrangling to feature creation, to training, to finetuning and to model test and validation, and finally to deployment. While Databricks is used by many users, we also use GitHub and code Q/A to promote a code in production. This is one of the advantages of Databricks is the integration part, not only Git but whether you use it on Azure or AWS, you can also leverage the power of the integrated Machine Learning in those platforms, such as auto ml or Azure ML.

• Make Data Scientist more efficient

• Scalable processing
• Notebook-based codebase
• Scheduled jobs

• Supports end-customer dashboards
• Provides a fast analysis platform
• Supports BI dashboards for engineering teams