Query Azure Data Lake Storage Data in DataGrip

DataGrip is a database IDE that allows SQL developers to query, create, and manage databases. When paired with the CData JDBC Driver for Azure Data Lake Storage, DataGrip can work with live Azure Data Lake Storage data. This article shows how to establish a connection to Azure Data Lake Storage data in DataGrip.

Create a New Driver Definition for Azure Data Lake Storage

The steps below describe how to create a new Data Source in DataGrip for Azure Data Lake Storage.

1. In DataGrip, click File -> New > Project and name the project
2. In the Database Explorer, click the plus icon () and select Driver.
3. In the Driver tab:
   • Set Name to a user-friendly name (e.g. "CData Azure Data Lake Storage Driver")
   • Set Driver Files to the appropriate JAR file. To add the file, click the plus (), select "Add Files," navigate to the "lib" folder in the driver's installation directory and select the JAR file (e.g. cdata.jdbc.adls.jar).
   • Set Class to cdata.jdbc.adls.ADLS.jar
   Additionally, in the advanced tab you can change driver properties and some other settings like VM Options, VM environment, VM home path, DBMS, etc
   • For most cases, change the DBMS type to "Unknown" in Expert options to avoid native SQL Server queries (Transact-SQL), which might result in an invalid function error
4. Click "Apply" then "OK" to save the Connection

Configure a Connection to Azure Data Lake Storage

1. Once the connection is saved, click the plus (), then "Data Source" then "CData Azure Data Lake Storage Driver" to create a new Azure Data Lake Storage Data Source.
2. In the new window, configure the connection to Azure Data Lake Storage with a JDBC URL.

   Built-in Connection String Designer

   For assistance in constructing the JDBC URL, use the connection string designer built into the Azure Data Lake Storage JDBC Driver. Either double-click the JAR file or execute the jar file from the command-line.

   java -jar cdata.jdbc.adls.jar

   Fill in the connection properties and copy the connection string to the clipboard.

   Authenticating to a Gen 1 DataLakeStore Account

   Gen 1 uses OAuth 2.0 in Azure AD for authentication.

   For this, an Active Directory web application is required. You can create one as follows:

   1. Sign in to your Azure Account through the .
   2. Select "Azure Active Directory".
   3. Select "App registrations".
   4. Select "New application registration".
   5. Provide a name and URL for the application. Select Web app for the type of application you want to create.
   6. Select "Required permissions" and change the required permissions for this app. At a minimum, "Azure Data Lake" and "Windows Azure Service Management API" are required.
   7. Select "Key" and generate a new key. Add a description, a duration, and take note of the generated key. You won't be able to see it again.

   To authenticate against a Gen 1 DataLakeStore account, the following properties are required:

   • Schema: Set this to ADLSGen1.
   • Account: Set this to the name of the account.
   • OAuthClientId: Set this to the application Id of the app you created.
   • OAuthClientSecret: Set this to the key generated for the app you created.
   • TenantId: Set this to the tenant Id. See the property for more information on how to acquire this.
   • Directory: Set this to the path which will be used to store the replicated file. If not specified, the root directory will be used.

   Authenticating to a Gen 2 DataLakeStore Account

   To authenticate against a Gen 2 DataLakeStore account, the following properties are required:

   • Schema: Set this to ADLSGen2.
   • Account: Set this to the name of the account.
   • FileSystem: Set this to the file system which will be used for this account.
   • AccessKey: Set this to the access key which will be used to authenticate the calls to the API. See the property for more information on how to acquire this.
   • Directory: Set this to the path which will be used to store the replicated file. If not specified, the root directory will be used.
3. Set URL to the connection string, e.g., jdbc:adls:Schema=ADLSGen2;Account=myAccount;FileSystem=myFileSystem;AccessKey=myAccessKey;InitiateOAuth=GETANDREFRESH
4. Click "Apply" and "OK" to save the connection string

At this point, you will see the data source in the Data Explorer.

Execute SQL Queries Against Azure Data Lake Storage

To browse through the Azure Data Lake Storage entities (available as tables) accessible through the JDBC Driver, expand the Data Source.

To execute queries, right click on any table and select "New" -> "Query Console."

In the Console, write the SQL query you wish to execute. For example: SELECT FullPath, Permission FROM Resources WHERE Type = 'FILE'

Download a free, 30-day trial of the CData JDBC Driver for Azure Data Lake Storage and start working with your live Azure Data Lake Storage data in DataGrip. Reach out to our Support Team if you have any questions.