Databricks AWS Account Setup

Databricks AWS Account Setup

Databricks is a unified data analytics platform for data engineering, data science, machine learning, and data analytics. It has integration with AWS, Microsoft Azure, and Google Cloud. In this tutorial, we will talk about how to sign up for the 14-day free trial and set up AWS account for Databricks. You will learn:

  • Who needs the AWS account setup?
  • How to sign up for the Databricks free trial?
  • How to set up the AWS stack for Databricks?
  • How to choose between different plans?
  • How to monitor Databricks and AWS costs?

Resources for this post:

Let’s get started!

If you prefer the video version of the tutorial, please check out the video below.

Databricks AWS Account Setup – GrabNGoInfo.com

Step 1: Who needs AWS Account Setup?

Databricks has free and paid plans. The free plan is called Databricks Community Edition. The paid plan has different tiers with different pricing structures and functionality.

  • The free community edition provides access to a micro-cluster, a cluster manager, and a notebook environment. Users do not need to pay for the Databricks platform or the AWS costs.
  • The paid plan has three tiers: standard plan, premium plan, and enterprise plan. The plan and pricing details can be found at https://databricks.com/product/aws-pricing.

Only the paid plans need users to set up the AWS account. The free plan AWS account setup is managed by Databricks.

Step 2: Sign Up For Full Trial

Databricks provides a 14-day full trial. We can start with the free community edition and upgrade to the paid version when the functionalities in the paid versions are needed, or sign up for the free trial at https://databricks.com/try-databricks. In this tutorial, we will sign up for the 14-day full trial by filling out the form below and clicking the orange GET STARTED FOR FREE button.

Databricks Free Trial – GrabNGoInfo.com

To learn how to upgrade from the community edition to the paid plans, check out my tutorial Databricks Community Edition Upgrade To Paid Plan AWS Setup. I will leave the link in the description below.

Step 3: Choose Cloud Provider

After clicking the orange GET STARTED FOR FREE button, we need to choose a cloud provider from AWS, Microsoft Azure, and Google Cloud Platform. We choose AWS here because this tutorial is for Databricks AWS account setup.

Step 4: Verify Email and Login Databricks

Click the blue Get started button, and you will be asked to validate your email.

An email titled “Welcome to Databricks! Please verify your email address” from Databricks will be sent to your email. Clicking the link in the email brings you to a page to set your password.

Step 5: Databricks Subscription Plan Selection

After setting up the password, it’s time to select a subscription plan.

  • Standard plan provides a basic platform for data analytics and machine learning workloads. It does not have access to Databricks SQL, autoscaling, and role-based access control.
  • Premium plan has access to Databricks SQL, cloud native security, and autoscaling. It’s the most popular plan.
  • Enterprise plan provides advanced compliance and security for mission critical data. It’s commonly used by large companies.

We selected the premium plan because it has all the commonly used functionalities for data science and machine learning projects.

Clicking the blue Continue button starts the 14-day free trial. The rate after the free trial is available by clicking the blue list rates.

Step 6: Databricks Workspace Setup

After clicking the blue Continue button, we can see a page about account, workspace, and tools structure.

After clicking the blue Get started button, we need to confirm:

  • An AWS account has been created.
  • We have the password for the newly created Databricks account.
  • We have a name in mind for the workspace.

If you have all three items available, click the blue Confirm button.

Now it’s time to set up the first workspace! Give your workspace a name and select an AWS region from the dropdown list.

After clicking the blue “Start quickstart” button, a Databricks workspace page is open.

Step 7: AWS Account Setup

A new tab is open for logging in AWS console. We will work on the AWS setup, then return to the Databricks workspace page.

After logging in AWS account, we will create a stack. Most information for creating the stack has been pre-populated. Enter the Databricks account password.

Under Workspace configuration, if the Data bucket name is not pre-populated, enter star (* ) to access data in any of your S3 buckets.

Scroll to the end of the page, under Capabilities, check “I acknowledge that AWS CloudFormation might create IAM resources with custom names.” and click the orange Create stack button.

The status shows CREATION_IN_PROGRESS.

When the stack creation is completed, the status changes to CREATE_COMPLETE.

Step 8: Databricks Workspace Login

Now let’s return to the Databricks workspaces page. We can see that the workspace called “demo” is created.

Clicking the blue word Open will open a new tab for logging in Databricks.

We will see the Databricks homepage after entering the username and password. On the upper right corner of the page, there is a message “Free trial ends in 14 days. Continue with a pay-as-you-go subscription by providing your billing information.”

Databricks is successfully connected to the AWS account for the premium plan now!

Step 9: Verify Current Plan and Add Billing Information

To verify the plan selection and AWS account connection, click Settings, then Manage Account on the lower-left corner of the page.

We need to sign in again to manage the account.

Go to the Settings gear icon on the left-hand side, we can see that the current plan is Premium. We can add billing information here.

Step 10: Track Costs For DBU and AWS

To check the DBU usage and costs, click the usage icon on the left-hand side.

The AWS costs need to be monitored in the AWS account. Please follow my previous tutorial on AWS Cost Control Using Budget Monitor Alert to monitor the AWS cost.

Summary

This tutorial talked about how to sign up for the Databricks 14-day free trial and connect to AWS account. You learned:

  • Who needs the AWS account setup?
  • How to sign up for the Databricks free trial?
  • How to set up the AWS stack for Databricks?
  • How to choose between different plans?
  • How to monitor Databricks and AWS costs?

For more information about data science and machine learning, please check out my YouTube channel and Medium Page or follow me on LinkedIn.

Recommended Tutorials

Leave a Comment

Your email address will not be published. Required fields are marked *