Image for post
Image for post

Integrating Google Drive/Colab with GitHub

Whether you’re uploading large amounts of data to Drive for storage, utilizing Google Colab’s cloud computing to train a large model, or simply prefer the easy file sharing that the Google atmosphere enables between users, you’re going to encounter some stickiness in your workflow when incorporating Google Colab + Github. This is often the case for me when I’m trying to train a large neural network inside of a notebook (.ipynb file) and would like to access some of Google’s GPU and TPU(current free access to 12GB for up to 12 hrs runtime ).

While you could move your files from your local machine → Drive, make your edits, and download the edited files to overwrite the local copy and then push back up to GitHub, there is an easier way. While the following tutorial focuses specifically on integrating .ipynb files from Github with Google Colab, the same processes still apply for importing any work from a GitHub repository into Google Drive.

The Process

To run notebooks in Google Colab, first you are going to have to clone down your original repository directly from GitHub. To do this, I suggest creating a separate ‘terminal.ipynb’ notebook to hold all of your terminal commands (you can run shell commands from notebooks just the same as python scripts).

The first line in terminal.ipynb should be the following line, which connects the notebook to google drive.

from google.colab import drive
drive.mount('/content/drive')

You will be asked to follow a link, and copy an authentication key to paste in your notebook

Image for post
Image for post

Next, cd into the home of your drive by navigating to ‘My Drive’, escaping the space with a backslash. Bash commands run from notebook cells require a magic character in front to signal that it’s a bash command. In Jupyter, that special character is a ! (called a bang), but notice here we’re using a % instead.

%cd drive/My\ Drive/

After that, clone (or fork and clone) down your repository from GitHub. You will see it appear as a folder in the home of your Google Drive. (replace link with link to repo you are trying to clone)

!git clone https://github.com/sam-thurman/bsds500.git
Image for post
Image for post

After making changes to a file inside the repo, go to the drop down file menu in the upper left corner, and select ‘Save a copy in GitHub’.

Image for post
Image for post

From there you can select the GitHub repository to push the file into, specify a file path if you want to put it in a folder, add a commit message, and even specify a branch to push your work to. When you’re done, simply press okay.

Image for post
Image for post

In GitHub, if you are pushing a new file, you will see it appear in the repo in GitHub under the specified file path. If you are updating a file’s contents, you will notice that the file will have been updated with your changes, and commit message.

Written by

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store