Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[spruce] Update control plane operations #167

Merged
merged 9 commits into from
Nov 22, 2023
2 changes: 1 addition & 1 deletion .github/workflows/merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Setup
uses: ./.github/actions/setup
- name: Generate TypeDoc documentation
Expand Down
125 changes: 4 additions & 121 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Setup
uses: ./.github/actions/setup
- name: Prettier
Expand All @@ -35,123 +35,6 @@ jobs:
- name: TypeDoc
uses: ./.github/actions/build-docs

run-integration-tests:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was part of the integration tests work which is also included here. This block of logic was moved to testing.yml so it could be more easily reused.

name: Run integration tests
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
jest_env: ['node', 'edge']
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup
uses: ./.github/actions/setup
- name: Run tests
env:
CI: true
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
run: npm run test:integration:${{ matrix.jest_env }}

example-app-semantic-search:
name: 'Example app: semantic search'
runs-on: ubuntu-latest
steps:
- name: Checkout pinecone-ts-client
uses: actions/checkout@v4
with:
path: pinecone-ts-client
- name: Checkout semantic-search-example
uses: actions/checkout@v4
with:
repository: pinecone-io/semantic-search-example
path: semantic-search-example
- name: Install and build client
shell: bash
run: |
cd pinecone-ts-client
npm install --ignore-scripts
npm run build
- name: Install and build sample app
shell: bash
run: |
cd semantic-search-example
npm install --no-cache
npm install '../pinecone-ts-client'
cat package.json
- name: Run example tests with dev version of client
env:
CI: true
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
PINECONE_INDEX: 'semantic-search'
shell: bash
run: |
cd semantic-search-example
npm run test

unit-tests:
name: Run unit tests
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup
uses: ./.github/actions/setup
- name: Run tests
env:
CI: true
run: npm run test:unit -- --coverage

typescript-compilation-tests:
name: TS compile
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
tsVersion:
[
'~4.1.0',
'~4.2.0',
'~4.3.0',
'~4.4.0',
'~4.5.0',
'~4.6.0',
'~4.7.0',
'~4.8.0',
'~4.9.0',
'~5.0.0',
'~5.1.0',
'~5.2.0',
'latest',
]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: 18.x
registry-url: 'https://registry.npmjs.org'
- name: Install npm packages
run: |
npm install --ignore-scripts
shell: bash
- name: Build typescript for pinecone
run: npm run build
shell: bash
- name: install, compile, and run sample app
shell: bash
env:
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
run: |
set -x
cd ..
cp -r pinecone-ts-client/ts-compilation-test ts-compilation-test
cd ts-compilation-test
npm install typescript@${{matrix.tsVersion}} --no-cache
npm install '../pinecone-ts-client' --no-cache
cat package.json
cat package-lock.json
npm run tsc-version
npm run build
npm run start
testing:
uses: './.github/workflows/testing.yml'
secrets: inherit
6 changes: 5 additions & 1 deletion .github/workflows/release-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,16 @@ on:
- cron: '0 0 * * *'

jobs:
testing:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think our Python client runs through testing on the release workflows, felt like it made sense to add it, but let me know otherwise.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. We should test all the time. The testing prior to release is really just a final safeguard in case somebody is YOLOing something to main outside the normal process. So I'm very 👍 👍 on this change.

uses: './.github/workflows/testing.yml'
secrets: inherit

version-and-release:
name: Release dev build to NPM
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Setup
uses: ./.github/actions/setup
Expand Down
10 changes: 4 additions & 6 deletions .github/workflows/release-prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,18 @@ concurrency:
cancel-in-progress: true

jobs:
# integration-tests:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seemed old, we can parameterize the running of tests if desired.

# if: ${{ github.event.inputs.runTests == 'true' }}
# uses: ./.github/workflows/PR.yml
# secrets: inherit
testing:
uses: './.github/workflows/testing.yml'
secrets: inherit

version-and-release:
name: Release to NPM
# needs: integration-tests
runs-on: ubuntu-latest
outputs:
tagName: ${{ steps.npm-release.outputs.release_tag }}
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Setup
uses: ./.github/actions/setup
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/release-spruce-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ on:
- cron: '0 0 * * *'

jobs:
testing:
uses: './.github/workflows/testing.yml'
secrets: inherit

version-and-release:
name: Release dev build to NPM
runs-on: ubuntu-latest
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/release-spruce.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
workflow_dispatch: {}

jobs:
testing:
uses: './.github/workflows/testing.yml'
secrets: inherit

version-and-release:
name: Release dev build to NPM
runs-on: ubuntu-latest
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Setup
uses: ./.github/actions/setup
- name: Cleanup
Expand Down
143 changes: 143 additions & 0 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
name: Testing
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the logic that was pulled out of pr.yml.


on:
workflow_call: {}

jobs:
unit-tests:
name: Run unit tests
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup
uses: ./.github/actions/setup

- name: Run tests
env:
CI: true
run: npm run test:unit -- --coverage

integration-tests:
name: Run integration tests
runs-on: ubuntu-latest
strategy:
fail-fast: false
max-parallel: 1
matrix:
jest_env: ['node', 'edge']
pinecone_env: ['prod', 'staging']

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup
uses: ./.github/actions/setup

- name: Run integration tests (prod)
if: matrix.pinecone_env == 'prod'
env:
CI: true
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
PINECONE_CONTROLLER_HOST: 'https://api.pinecone.io'
run: npm run test:integration:${{ matrix.jest_env }}

- name: Run integration tests (staging)
if: matrix.pinecone_env == 'staging'
env:
CI: true
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY_STAGING }}
PINECONE_CONTROLLER_HOST: 'https://api-staging.pinecone.io'
run: npm run test:integration:${{ matrix.jest_env }}

typescript-compilation-tests:
name: TS compile
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
tsVersion:
[
'~4.1.0',
'~4.2.0',
'~4.3.0',
'~4.4.0',
'~4.5.0',
'~4.6.0',
'~4.7.0',
'~4.8.0',
'~4.9.0',
'~5.0.0',
'~5.1.0',
'~5.2.0',
'latest',
]
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: 18.x
registry-url: 'https://registry.npmjs.org'
- name: Install npm packages
run: |
npm install --ignore-scripts
shell: bash
- name: Build TypeScript for Pinecone
run: npm run build
shell: bash
- name: install, compile, and run sample app
shell: bash
env:
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
run: |
set -x
cd ..
cp -r pinecone-ts-client/ts-compilation-test ts-compilation-test
cd ts-compilation-test
npm install typescript@${{ matrix.tsVersion }} --no-cache
npm install '../pinecone-ts-client' --no-cache
cat package.json
cat package-lock.json
npm run tsc-version
npm run build
npm run start

example-app-semantic-search:
name: 'Example app: semantic search'
runs-on: ubuntu-latest
steps:
- name: Checkout pinecone-ts-client
uses: actions/checkout@v4
with:
path: pinecone-ts-client
- name: Checkout semantic-search-example
uses: actions/checkout@v4
with:
repository: pinecone-io/semantic-search-example
path: semantic-search-example
- name: Install and build client
shell: bash
run: |
cd pinecone-ts-client
npm install --ignore-scripts
npm run build
- name: Install and build sample app
shell: bash
run: |
cd semantic-search-example
npm install --no-cache
npm install '../pinecone-ts-client'
cat package.json
- name: Run example tests with dev version of client
env:
CI: true
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
PINECONE_INDEX: 'semantic-search'
shell: bash
run: |
cd semantic-search-example
npm run test
15 changes: 10 additions & 5 deletions src/__tests__/pinecone.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,16 @@ jest.mock('../control', () => {
return {
...realControl,
describeIndex: () =>
jest
.fn()
.mockImplementation(() =>
Promise.resolve({ status: { host: fakeHost } })
),
jest.fn().mockImplementation(() =>
Promise.resolve({
name: 'fake-index',
dimension: 1,
metric: 'cosine',
host: fakeHost,
spec: { serverless: { cloud: 'aws', region: 'us-east-1' } },
status: { ready: true, state: 'Ready' },
})
),
deleteIndex: () => jest.fn().mockImplementation(() => Promise.resolve()),
};
});
Expand Down
Loading
Loading