Databricks Community Edition Upgrade To Paid Plan AWS Setup

Databricks Community Edition Upgrade To Paid Plan AWS Setup

Databricks has a free community edition and different paid plans. The paid plans can be integrated with AWS, Microsoft Azure, or Google Cloud. This tutorial covers:

  • How to sign up for the free Databricks Community Edition?
  • How to upgrade to the paid plan using AWS?
  • How to monitor Databricks and AWS costs?

To learn about how to sign up for the 14-day full trial directly and set up AWS, please refer to my tutorial Databricks AWS Account Setup.

Resources for this post:

Databricks Community Edition Upgrade To Paid Plan AWS Setup – GrabNGoInfo.com

Step 1: Databricks Free Community Edition Sign Up

To sign up for the Databricks free community edition, go to https://community.cloud.databricks.com and click the blue Sign Up.

Fill out the form and click the orange GET STARTED FOR FREE button.

On the Choose a cloud provider page, click Get started with Community Edition.

A window for verification will pop up. Click Verify. After finishing the verification, you will be asked to validate your email.

An email titled “Welcome to Databricks! Please verify your email address” from Databricks will be sent to your email. Clicking the link in the email brings you to a page to set your password.

After setting the password, you will see the Databricks homepage, and the free community edition is ready to use!

Step 2: Databricks Paid Plans Selection

Databricks free community edition has most of the commonly used features, but there are some features only available for the paid plans. For example, only the paid plans have the workspace, cluster, pool, jobs, and table access control.

When there is a need to use the features that are not available in the community edition, we can upgrade to a paid plan.

To switch to a paid plan, click Settings, then Manage Account on the lower-left corner of the page.

We need to sign in again to manage the account.

After logging in the account, it’s time to select a subscription plan.

  • Standard plan provides a basic platform for data analytics and machine learning workloads. It does not have access to Databricks SQL, autoscaling, and role-based access control.
  • Premium plan has access to Databricks SQL, cloud native security, and autoscaling. It’s the most popular plan.
  • Enterprise plan provides advanced compliance and security for mission critical data. It’s commonly used by large companies.

We selected the premium plan because it has all the commonly used functionalities for data science and machine learning projects.

Clicking the blue Continue button starts the 14-day free trial. The rate after the free trial is available by clicking the blue list rates.

Step 3: Databricks Workspace Setup

After clicking the blue Continue button, we can see a page about account, workspace, and tools structure.

After clicking the blue Get started button, we need to confirm:

  • An AWS account has been created.
  • We have the password for the newly created Databricks account.
  • We have a name in mind for the workspace.

If you have all three items available, click the blue Confirm button.

Now it’s time to set up the first workspace! Give your workspace a name and select an AWS region from the dropdown list.

After clicking the blue “Start quickstart” button, a Databricks workspace page is open.

Step 4: AWS Acount Setup

A new tab is open for logging in AWS console. We will work on the AWS setup, then return to the Databricks workspace page.

After logging in AWS account, we will create a stack. Most information for creating the stack has been pre-populated. The Databricks account password needs to be entered.

Under Workspace configuration, if the Data bucket name is not pre-populated, enter star (* ) to access data in any of your S3 buckets.

Scroll to the end of the page, under Capabilities, check “I acknowledge that AWS CloudFormation might create IAM resources with custom names.” and click the orange Create stack button.

The status shows CREATION_IN_PROGRESS.

When the stack creation is completed, the status changes to CREATE_COMPLETE.

Step 5: Databricks Workspace Login

Now let’s return to the Databricks workspaces page. We can see that the workspace called “demo” is created.

Clicking the blue word Open will open a new tab for logging in Databricks.

We will see the Databricks homepage after entering the username and password. On the upper right corner of the page, there is a message “Free trial ends in 14 days. Continue with a pay-as-you-go subscription by providing your billing information.”

Databricks is successfully connected to the AWS account for the premium plan now!

Step 6: Verify Current Plan and Add Billing Information

To verify the plan selection and AWS account connection, click Settings, then Manage Account on the lower-left corner of the page.

We need to sign in again to manage the account.

Go to the Settings gear icon on the left-hand side, we can see that the current plan is Premium. We can add billing information here.

Step 7: Track Costs For DBU and AWS

To check the DBU usage and costs, click the usage icon on the left-hand side.

The AWS costs need to be monitored in the AWS account. Please follow my previous tutorial on AWS Cost Control Using Budget Monitor Alert to monitor the AWS cost.

Summary

In this tutorial, you learned:

  • How to sign up for the free Databricks Community Edition?
  • How to upgrade to the paid plan using AWS?
  • How to monitor Databricks and AWS costs?

For more information about data science and machine learning, please check out my YouTube channel and Medium Page or follow me on LinkedIn.

Recommended Tutorials

Leave a Comment

Your email address will not be published. Required fields are marked *