Skip to content

Commit

Permalink
Merge pull request #78 from augmentable-dev/askgithub
Browse files Browse the repository at this point in the history
first pass at adding ability to query the GitHub API
  • Loading branch information
patrickdevivo authored Dec 13, 2020
2 parents 268322c + 5809ea8 commit c4b68ea
Show file tree
Hide file tree
Showing 39 changed files with 1,406 additions and 91 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
run: |
git clone https://github.com/libgit2/libgit2.git ~/libgit2
cd ~/libgit2
git checkout v1.0.1
git checkout v1.1.0
mkdir build && cd build
sudo cmake .. -DBUILD_CLAR=0
sudo cmake --build . --target install
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
coverage.out
.env
4 changes: 3 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,7 @@
],
"go.toolsEnvVars": {
"GOFLAGS": "-tags='sqlite_vtable'"
}
},
"go.testTimeout": "120s",
"go.testEnvFile": "${workspaceFolder}/.env"
}
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
gotags = "sqlite_vtable,static,system_libgit2"
gotags = "sqlite_vtable,sqlite_json1,static,system_libgit2"

vet:
go vet -v -tags=$(gotags) ./...
Expand Down
129 changes: 118 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ It can execute queries that look like:
-- how many commits have been authored by [email protected]?
SELECT count(*) FROM commits WHERE author_email = '[email protected]'
```

There's also preliminary support for executing queries against the GitHub API.

More in-depth examples and documentation can be found below.

## Installation
Expand Down Expand Up @@ -101,7 +104,11 @@ See `-h` for all the options.

### Tables

#### `commits`
#### Local Git Repository

When a repo is specified (either by the `--repo` flag or from the current directory), the following tables are available to query.

##### `commits`

Similar to `git log`, the `commits` table includes all commits in the history of the currently checked out commit.

Expand All @@ -120,7 +127,16 @@ Similar to `git log`, the `commits` table includes all commits in the history of
| parent_count | INT |
| tree_id | TEXT |

#### `files`
##### `stats`

| Column | Type |
|-----------|------|
| commit_id | TEXT |
| file | TEXT |
| additions | INT |
| deletions | INT |

##### `files`

The `files` table iterates over _ALL_ the files in a commit history, by default from what's checked out in the repository.
The full table is every file in every tree of a commit history.
Expand All @@ -136,7 +152,7 @@ Use the `commit_id` column to filter for files that belong to the work tree of a
| executable | BOOL |


#### `branches`
##### `branches`

| Column | Type |
|--------|------|
Expand All @@ -145,7 +161,7 @@ Use the `commit_id` column to filter for files that belong to the work tree of a
| target | TEXT |
| head | BOOL |

#### `tags`
##### `tags`

| Column | Type |
|--------------|------|
Expand All @@ -158,14 +174,105 @@ Use the `commit_id` column to filter for files that belong to the work tree of a
| message | TEXT |
| target_type | TEXT |

#### `stats`
#### GitHub Tables

| Column | Type |
|-----------|------|
| commit_id | TEXT |
| file | TEXT |
| additions | INT |
| deletions | INT |
**This functionality is under development and likely to change**

The following tables make GitHub API requests to retrieve data during query execution.
As such, you should ensure the `GITHUB_TOKEN` environment variable is set so that API requests are authenticated.
Unauthenticated API requests (no `GITHUB_TOKEN`) are subject to a stricter rate limit by GitHub, and may take longer to execute (query execution will try to respect the applicable rate limit).

##### `github_org_repos` and `github_user_repos`

These tables can be queried as table-valued functions expecting a single parameter, like so:

```sql
-- return all repos from a github *org*
SELECT * FROM github_org_repos('augmentable-dev')

-- return all repos from a github *user*
SELECT * FROM github_user_repos('augmentable-dev')
```

| Column | Type |
|-------------------|----------|
| id | INT |
| node_id | TEXT |
| name | TEXT |
| full_name | TEXT |
| owner | TEXT |
| private | BOOL |
| description | TEXT |
| fork | BOOL |
| homepage | TEXT |
| language | TEXT |
| forks_count | INT |
| stargazers_count | INT |
| watchers_count | INT |
| size | INT |
| default_branch | TEXT |
| open_issues_count | INT |
| topics | TEXT |
| has_issues | BOOL |
| has_projects | BOOL |
| has_wiki | BOOL |
| has_pages | BOOL |
| has_downloads | BOOL |
| archived | BOOL |
| pushed_at | DATETIME |
| created_at | DATETIME |
| updated_at | DATETIME |
| permissions | TEXT |

##### `github_pull_requests`

This table expects 2 parameters, `github_pull_requests('augmentable-dev', 'askgit')`:

```sql
SELECT count(*) FROM github_pull_requests('augmentable-dev', 'askgit') WHERE state = 'open'
```

| Column | Type |
|---------------------------|----------|
| id | INT |
| node_id | TEXT |
| number | INT |
| state | TEXT |
| locked | BOOL |
| title | TEXT |
| user_login | TEXT |
| body | TEXT |
| labels | TEXT |
| active_lock_reason | TEXT |
| created_at | DATETIME |
| updated_at | DATETIME |
| closed_at | DATETIME |
| merged_at | DATETIME |
| merge_commit_sha | TEXT |
| assignee_login | TEXT |
| assignees | TEXT |
| requested_reviewer_logins | TEXT |
| head_label | TEXT |
| head_ref | TEXT |
| head_sha | TEXT |
| head_repo_owner | TEXT |
| head_repo_name | TEXT |
| base_label | TEXT |
| base_ref | TEXT |
| base_sha | TEXT |
| base_repo_owner | TEXT |
| base_repo_name | TEXT |
| author_association | TEXT |
| merged | BOOL |
| mergeable | BOOL |
| mergeable_state | BOOL |
| merged_by_login | TEXT |
| comments | INT |
| maintainer_can_modify | BOOL |
| commits | INT |
| additions | INT |
| deletions | INT |
| changed_files | INT |

### Example Queries

Expand Down
15 changes: 9 additions & 6 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"github.com/augmentable-dev/askgit/pkg/askgit"
"github.com/augmentable-dev/askgit/pkg/tui"
"github.com/gitsight/go-vcsurl"
git "github.com/libgit2/git2go/v30"
git "github.com/libgit2/git2go/v31"
"github.com/spf13/cobra"
)

Expand Down Expand Up @@ -103,15 +103,18 @@ var rootCmd = &cobra.Command{
if err != nil {
handleError(err)
}
if cui {
tui.RunGUI(repo, dir, query)
return
}

ag, err := askgit.New(dir, &askgit.Options{
UseGitCLI: useGitCLI,
UseGitCLI: useGitCLI,
GitHubToken: os.Getenv("GITHUB_TOKEN"),
})
handleError(err)

if cui {
tui.RunGUI(ag, query)
return
}

rows, err := ag.DB().Query(query)
handleError(err)

Expand Down
9 changes: 8 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,23 @@ go 1.13
require (
github.com/DATA-DOG/go-sqlmock v1.5.0
github.com/gitsight/go-vcsurl v1.0.0
github.com/google/go-github v17.0.0+incompatible
github.com/google/go-querystring v1.0.0 // indirect
github.com/jroimartin/gocui v0.4.0
github.com/kr/text v0.2.0 // indirect
github.com/libgit2/git2go/v30 v30.2.2
github.com/libgit2/git2go/v31 v31.3.4
github.com/mattn/go-runewidth v0.0.9 // indirect
github.com/mattn/go-sqlite3 v1.14.4
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e // indirect
github.com/nsf/termbox-go v0.0.0-20201107200903-9b52a5faed9e // indirect
github.com/olekukonko/tablewriter v0.0.4
github.com/spf13/cobra v1.1.1
golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a // indirect
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
golang.org/x/sync v0.0.0-20190423024810-112230192c58
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
gopkg.in/yaml.v2 v2.3.0 // indirect
)

replace github.com/mattn/go-sqlite3 v1.14.4 => github.com/patrickdevivo/go-sqlite3 v1.14.6-0.20201211024840-146d4a910383
12 changes: 8 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Z
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
Expand Down Expand Up @@ -118,17 +122,15 @@ github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/libgit2/git2go/v30 v30.2.2 h1:6TzX6xDpLH24gexY6dcAt+56mFbg8gs9p2C8ZThKNGc=
github.com/libgit2/git2go/v30 v30.2.2/go.mod h1:bEqWPWaJjDpnkerA2FlyUdsuhc5/3UPBjYJ6SV0X3gY=
github.com/libgit2/git2go/v31 v31.3.4 h1:uwm+k+cgxIpyJtHxafYxdIDaO3EAdZJfGPO6pxSxA+c=
github.com/libgit2/git2go/v31 v31.3.4/go.mod h1:mnc0hPGPs0nDi9INrurTpioeRzje9DvSXqON/+JEhwY=
github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54=
github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-sqlite3 v1.14.4 h1:4rQjbDxdu9fSgI/r3KN72G3c2goxknAqHHgPWWs8UlI=
github.com/mattn/go-sqlite3 v1.14.4/go.mod h1:WVKg1VTActs4Qso6iwGbiFih2UIHo0ENGwNd0Lj+XmI=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
Expand All @@ -152,6 +154,8 @@ github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn
github.com/olekukonko/tablewriter v0.0.4 h1:vHD/YYe1Wolo78koG299f7V/VAS08c6IpCLn+Ejf/w8=
github.com/olekukonko/tablewriter v0.0.4/go.mod h1:zq6QwlOf5SlnkVbMSr5EoBv3636FWnp+qbPhuoO21uA=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/patrickdevivo/go-sqlite3 v1.14.6-0.20201211024840-146d4a910383 h1:yIH5Hs1SaI3AbLVf5lmUHLBDcMUjTxq2MlnnWfr7SuI=
github.com/patrickdevivo/go-sqlite3 v1.14.6-0.20201211024840-146d4a910383/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181zc=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
Expand Down
Loading

0 comments on commit c4b68ea

Please sign in to comment.