Skip to content

Snowflake Git Integration

In this tutorial we will show you how to integrate Git into your Snowflake account. We will walk through two main subjects. The first is pulling the repo into Snowflake and navigating around. The second is creating a stored procedure, streamlit dashboard, and run a query from files that lives on github.

Credit for the orginial tutorial goes to Mark Boos!

Video

Requirements

You will need a github repo and the ability to create a personal access token. We will show how to create the token in this tutorial.

Download

Setup

In this section we will upload the example code to a repository and then setup Snowflake:

Github Repository

Start by creating a repo. Create repo

Call it "tutorial". It can be public or private. Naming

Please do not upload the repo folder itself. Just the files in it.

Upload the example repository. Naming

Your good to go. Naming

Personal Access Token

We will need a persoanl access token to allow Snowflake to work with our Git repository. First lets navigate to the token page. Basic Navigation

Click on settings. Navigation Settings

Next developer settings. Navigation Developer Settings

We'll be using a classic token. Select Classic Token

Next we'll enter in a name, the experation of the token, and select "repo" for the scope of the permissions. Select Permission

We'll copy our token, it will be used in the following Snowflake step. Copy Token

Snowflake

Lets now use snowflake to connect to our repository.

If you don't have a database, schema or warehouse yet. 
use role sysadmin;

-- Create a database to store our schemas.
create database raw;

-- Create the schema. The schema stores all our objectss.
create schema raw.git;

/*
    Warehouses are synonymous with the idea of compute
    resources in other systems. We will use this
    warehouse to call our user defined function.
*/
create warehouse if not exists developer 
    warehouse_size = xsmall
    initially_suspended = true;

use database raw;
use schema git;
use warehouse developer;


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
use role accountadmin;

create or replace secret github_secret
    type = password
    username = '<Github Username>' /* (1)! */
    password = '<Personal Access Token>'; /* (2)! */

create or replace api integration git_api_integration
    api_provider = git_https_api
    api_allowed_prefixes = ('<Base Github URL>') /* (3)! */
    allowed_authentication_secrets = (github_secret)
    enabled = true;

create or replace git repository tutorial
    api_integration = git_api_integration
    git_credentials = github_secret
    origin = '<REPOSITORY URL>'; /* (4)! */

  1. Copy Token

  2. Copy Token

  3. Copy Token

  4. Copy Token

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
use role accountadmin;

create secret github_secret
    type = password
    username = 'sfc-gh-dwilczak'
    password = 'huifuhf.....f94894h2';

create api integration git_api_integration
    api_provider = git_https_api
    api_allowed_prefixes = ('https://github.com/sfc-gh-dwilczak')
    allowed_authentication_secrets = (github_secret)
    enabled = true;

create git repository tutorial
    api_integration = git_api_integration
    git_credentials = github_secret
    origin = 'https://github.com/sfc-gh-dwilczak/tutorial';
status
Git Repository TUTORIAL was successfully created.

Examples

Now that we have our Snowflake and Git repository setup lets go through a few examples.

Lets navigate and show parts of our repo in Snowflake.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
-- Show repos added to snowflake.
show git repositories;

-- Show branches in the repo.
show git branches in git repository tutorial;

-- List files.
ls @tutorial/branches/main;

-- Show code in file.
select $1 from @tutorial/branches/main/examples/app.py;

-- Fetch git repository updates.
alter git repository tutorial fetch;

Run a File

Lets execute a file on Snowflake that lives in our repositroy.

1
2
-- Run the files in Snowflake.
execute immediate from @tutorial/branches/main/examples/hello.sql;
GREETING
Hello World from a SQL query

Stored Procedure

Lets create a snowflake stored procedure from a file that lives in the repository.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
-- Create snowpark procedure
create or replace procedure hello()
    returns string
    language python 
    runtime_version= '3.8'
    packages=('snowflake-snowpark-python')
    imports=('@tutorial/branches/main/examples/hello.py')
    handler='hello.main';

call hello();
Hello
Hello World!

Streamlit

Lets create a Streamlit application in Snowflake using a file in our repository.

1
2
3
4
5
-- Create streamlit application from file.
create or replace streamlit streamlit_application
    root_location = @raw.git.tutorial/branches/main/examples
    main_file = '/app.py'
    query_warehouse = 'developer';
status
Streamlit STREAMLIT_APPLICATION successfully created.

Lets navigate to the streamlit in snowflake tab and select our dashboard that we just created. Navigate to Streamlit

Look at our simple streamlit dashboard that was create from our file in our Git repository! Simple Streamlit Dashboard