gitdir

gitdir

Javascript / node.js code to read a single directory from a GitHub or GitLab repository. All the files (not directories) and their contents, are returned in an array or map. Useful if you want to grab a small part of some external repository for your project.

basic usage

var gitdir = require('gitdir');
...
gitdir(repositoryName, directory, options, callback);

repositoryName combines the user and the repo, and the delimiter is important!

• for GitHub use JohnDoe/CoolRepository
• for GitLab, use JohnDoe%2FCoolRepository

e.g. to read the root directory of the Github npm repository,

   gitdir("npm/npm", "", {}, function(err, data) {
      // data contains an array of objects with file information
   });

For GitHub, the file information is similar to what you'd get from Get Contents

e.g. https://api.github.com/repos/{repositoryName}/contents/

except:

1. Usually only files (not dirs) are included (see options.keepAll below)
2. A new field, "contents", has the contents of the file.

Example for a single file npm/npm/LICENSE (contents truncated for this example)

{
   "name": "LICENSE",
   "path": "LICENSE",
   "sha": "0b6c2287459632e4aaf63bd7d53eb9ba054b57ea",
   "size": 9742,
   "url": "https://api.github.com/repos/npm/npm/contents/LICENSE?ref=latest",
   "html_url": "https://github.com/npm/npm/blob/latest/LICENSE",
   "git_url": "https://api.github.com/repos/npm/npm/git/blobs/0b6c2287459632e4aaf63bd7d53eb9ba054b57ea",
   "download_url": "https://raw.githubusercontent.com/npm/npm/latest/LICENSE",
   "type": "file",
   "_links": {
      "self": "https://api.github.com/repos/npm/npm/contents/LICENSE?ref=latest",
      "git": "https://api.github.com/repos/npm/npm/git/blobs/0b6c2287459632e4aaf63bd7d53eb9ba054b57ea",
      "html": "https://github.com/npm/npm/blob/latest/LICENSE"
   },
   "contents": "The npm application\nCopyright (c) npm, Inc. ..."
}

For GitLab, the information is similar to what you'd get from List repository tree

e.g. https://gitlab.com/api/v4/projects/{repositoryName}/repository/tree?private_token={options.private_token}&ref={options.branch}, with the addition of a contents and a download_url, which was used to fetch the contents.

e.g. gitlab-com%2Fwww-gitlab-com/LICENCE, you get:

{
   "id": "e186012554b42685b8e3b9bd52f3658f2d1d215c",
   "name": "LICENCE",
   "type": "blob",
   "path": "LICENCE",
   "mode": "100644",
   "download_url":"https://gitlab.com/api/v4/projects/gitlab-com%2Fwww-gitlab-com/repository/files/LICENCE/raw?private_token=&ref=master",
   "contents": "Copyright (c) GitLab B.V. \n"
}

Options

Option	Default	Notes
gitlab, GitLab	false	false => GitHub
body_key	"contents"	name of the new key holding the file contents ("body")
blob	false	GitLab only. Get content from blob blobs/:sha instead of raw file files/:file_path
branch	"master"	note - not tested much yet...
deleteDownloadURL	false	delete the download URL from the data, in case in contains a private_token
fileFilter	null	see below
gitlab_API_root	"https://gitlab.com/api/v4"	change if you have your own GitLab server
github_API_root	"https://api.github.com"	change if you have your own GitHub server
keepAll	false	keep all results (including directories). Normally only files are kept.
map	false	if true, return a map (with key = path) instead of array of file information
private_token	""	needed if you are fetching a private repository
recur	false	not yet supported
user_agent	"github.com/MorganConrad/gitdir"	required for the API call, be polite

options.fileFilter determines which files will be included

• if missing, include all files, except those ending in ".jar".
• if a string or Regex, only include matching filePaths.
• if a user-provided-function, include the file when filter(filePath, fileInfo) returns true. e.g. If you want to use multimatch

Caveats, TODOs

1. the private_token is a security weakness, and, even worse, may appear in the results.

• You should probably use the deleteDownloadURL option.
• Haven't tried it with GitHub cause I don't have any private repos there.
• For GitLab, you should use a more granular "Personal Access Token" instead.

1. Branch isn't well tested.
2. Recursion into directories might be added later.
3. Not sure about encoding for non-text contents.