There are projects where you need to manage data not coming from a regular database but rather from sources such as repositories. In situations like this, you need to integrate with various service providers (GitHub, GitLab, etc.)
While there are multiple ways to go around this topic, today, I will focus on how to do that using GitHub's REST API. For this task, I chose to play with the integration using a pre-existent Node JS app.
You will find only the integration logic in the code snippets below rather than a full-fledged working app.
Getting started
The first thing I did was install GitHub's library, Octokit REST is one of the libraries you can use for this. An alternative to this is Octokit Core (more functionalities; yet these are not required for what I'm trying to achieve today).
Once installed, I generated a personal access token. This one is needed for making the requests. To generate a token, go to Personal Access Tokens page on GitHub and simply add a new one to the list (only select repo).
The last thing I did before getting into the action was to fork this repo.
Connecting to your Repository
With all of the pre-requisites ready, I created a new .js file using the configs below.
const { Octokit } = require('@octokit/rest');
const octokit = new Octokit({
auth: 'my-personal-access-token',
});
const owner = 'my-username';
const repo = 'Hello-World';
author = {
name: 'My Name',
email: 'myemail@domain.com',
};
const url = '/repos/{owner}/{repo}/{path}';
const ref = 'heads/master';
I then authenticated a new octokit instance, used to make calls to GitHub. The rest of the configs were needed on a global scope for various actions (more on that, down below).
Pulling data
Easiest thing to do with the current setup is to retrieve the contents of the repo. To do that, I created a new function where I made a request to octokit, specifying the path of the wanted content:
const getContents = async () => {
const { data } = await octokit.request({
owner,
repo,
url,
method: 'GET',
path: 'contents',
});
console.log(data)
}
getContents();
Et voila! The request returns a list with one object corresponding to the README file from the forked repo. If I had a more complex structure in the repo and wanted to retrieve data from a specific folder, I could have changed the path to something like this: 'contents/src/components/component1'; where src is at the root and component1 contains one or more files.
Going back to my object, we can see that it contains information such as links, name, path & type of the file, and a property sha. This represents the hash of the file, and you can think of it as an ID. SHAs play a significant role when it comes to working with this API, so you'll see that property being used over and over again.
[
{
name: 'README',
path: 'README',
sha: '980a0d5f19a64b4b30a87d4206aade58726b60e3',
size: 13,
url: 'https://api.github.com/repos/my-username/Hello-World/contents/README?ref=master',
html_url: 'https://github.com/my-username/Hello-World/blob/master/README',
git_url: 'https://api.github.com/repos/my-username/Hello-World/git/blobs/980a0d5f19a64b4b30a87d4206aade58726b60e3',
download_url: 'https://raw.githubusercontent.com/my-username/Hello-World/master/README',
type: 'file',
_links: {
self: 'https://api.github.com/repos/my-username/Hello-World/contents/README?ref=master',
git: 'https://api.github.com/repos/my-username/Hello-World/git/blobs/980a0d5f19a64b4b30a87d4206aade58726b60e3',
html: 'https://github.com/my-username/Hello-World/blob/master/README'
}
}
]
Pushing data
Now, if I want to push a file to the repo, things get complicated. To do that, I need to:
- make sure I'm "up to date" with the branch I'm on (currently
master) by getting the latest commit (git pull)
- create a new list of files that will be added to the commit (
git add .)
- create a new commit (
git commit)
- push the changes (
git push origin HEAD)
The way to achieve is by first pulling the data. To do that, I got the ref to the latest commit:
const pushContents = async () => {
const commits = await octokit.repos.listCommits({
owner,
repo,
});
const latestCommitSHA = commits.data[0].sha;
}
Only the SHA of the latest commit is needed, so I extracted it from the commits variable.
Next, I faked the files which are going to be committed. These are later added to a git tree (the hierarchy structure between files in a Git repository). For that, I followed the format implied by the API.
const pushContents = async () => {
...
const files = [{
mode: '100644',
path: 'src/file1.txt',
content: 'Hello world 1',
},{
mode: '100644',
path: 'src/file2.txt',
content: 'Hello world 2',
}];
}
The format above explained:
- mode: type of the item, 100644 = file
- path: location, name and extension of the file
- content: this one is optional, whatever you want to add inside the file (if the file already has content, you'll need to create a blob)
With the files set, I went ahead and created the tree.
const pushContents = async () => {
...
const {
data: { sha: treeSHA },
} = await octokit.git.createTree({
owner,
repo,
tree: files,
base_tree: latestCommitSHA,
});
}
The tree needs to include the files and the sha of my latest commit (to keep track of the data existing inside the repo). If you want to overwrite the repo completely, omit base_tree.
With the new tree in place, I created a new commit. I specified the sha of the tree, the sha of the latest commit, and my commit message. Without specifying the latest commit in parents, it is like trying to push without being up to date.
const pushContents = async () => {
...
const {
data: { sha: newCommitSHA },
} = await octokit.git.createCommit({
owner,
repo,
author,
tree: treeSHA,
message: 'Changes via API',
parents: [latestCommitSHA],
});
}
Once I had the commit ready, the only thing left to do was to push the changes.
const pushContents = async () => {
...
const response = await octokit.git.updateRef({
owner,
repo,
ref,
sha: newCommitSHA,
});
}
Done! Two new files have been added to my repo, under the folder src. Below you can find the full example:
const pushFiles = async () => {
const commits = await octokit.repos.listCommits({
owner,
repo,
});
const latestCommitSHA = commits.data[0].sha;
const files = [{
mode: '100644',
path: 'src/file1.txt',
content: 'Hello world 1',
},{
mode: '100644',
path: 'src/file2.txt',
content: 'Hello world 2',
}];
const {
data: { sha: treeSHA },
} = await octokit.git.createTree({
owner,
repo,
tree: files,
base_tree: latestCommitSHA,
});
const {
data: { sha: newCommitSHA },
} = await octokit.git.createCommit({
owner,
repo,
author,
tree: treeSHA,
message: 'Changes via API',
parents: [latestCommitSHA],
});
const result = await octokit.git.updateRef({
owner,
repo,
ref,
sha: newCommitSHA,
});
};
Next steps
Updating the existing files or deleting ones follows the same principle. To do that, target the tree, add what's changed, then commit and push. For easiness, make a function out of each step you need to go through, which return the SHAs.
