Skip to main content

Overview

Users can request for access to a Databricks environment for processing datasets requested on DataHive. Databricks is an Apache Spark environment for processing large datasets, useful for complex data processing or analysis that can't be done on a local machine (GSIB/SOE).

We highly recommend using Databricks if the expected output of the data exceeds 1 million rows or it is greater than 5GB in size.

Pre-requisite​

  1. Opt-in for this feature in your data request form.

Request Databricks Access

  1. The data request with Databricks opted-in has to be in a fulfilled state.

  2. A TechPass account. This account is used to log into Databricks. Sign up from here.

    • In the TechPass sign up page fill in your details as follows:
      • Email Address: Key in your .gov.sg email address
      • Mobile number: Your Singapore (+65) phone number
      • Is onboarding to SEED required?: No

Getting access to Databricks​

After the data request has been fulfilled, you can access the datasets directly on Databricks.

Click on Go to Databricks to access the workspace provisioned:

Access Databricks Workspace