Skip to content

Commit

Permalink
sendMail API (#254)
Browse files Browse the repository at this point in the history
* rename User table NewsletterSubscriber

* move users to newsletterSubscribers in api

* init uses /api/newsletterSubscribers

* working on migration [wip]

* fix migrations for indexes

* fix messed up prisma migration

* signup and session

* use iron-session for session management

* email validator, signup page

* login page

* login

* auth demo

* refactor session stuff into util

* sendMail api

* /api/logout route
analytics page requires login

* api keys

* nonworking

* add iron-session to cli and rename cookies

* session, logout

* refactor

* working github oauth

* add oauth_tokens table

* add prisma to cli

* email and password validations on signup

* ui for login

* add Config table to cli db

* add signup link to header for testing

* start of Organization in CLI

* global prisma

* fix types

* finish oauth callback maybe

* fix logix

* add iron session to callback route

* working on api

* return the user that sent the oauth token

* settings

* sendMail with api keys

* nodemail opts

* validate api key

* add cache control header (30s) to validate

* fix apikey response parse

* integration test for /api/apiKeys/:k/validate

* try ankane/setup-postgres@v1

* try user postgres

* try petersugihara

* try with service

* MM_DEV=1

* migrate in cli too

* try checking in generated client

* rm packages/cli/prisma/generated from git

* generate web client locally too

* beautiful green

* fix posthog test to work when MM_DEV=1

* MM_DEV=1 works for setup test

* refactor

* use the right type

* refactor

* user.test.ts for users api in web

* test db url

* .envrc

* database cleaning

* test create user and login

* use test db's in github

* test passwords for github actions

* signup: if no redirectTo, take them to settings

* invariants

* separate e2e tests from regular jest tests

* woops, needed to fix yarn test

* rename e2e -> integration

* comment

* try prisma

* use DATABASE_URL=$DATABASE_URL_TEST WEB_DATABASE_URL=$WEB_DATABASE_URL_TEST

* hmm

* move tests that use the db into integration folder

* fix lint errors

* rename

* fix merge conflict

* comment out tests

* uncomment out tests

* - remove cli and web prisma migrations
- update cli and web schemas
- i didn't run prisma migrate dev yet

* delete oauth files

* move apiKeys to cli

* more schema update

* migrate

* update package script names

* move users, session api to cli

* remove oauth redirect

* remove signup page

* remove signup link from cli

* move settings page from web to cli and delete login page from web

* signup page in cli

* signup in cli

* find first user or 404

* working

* remove loggin

* login

* fix useRef types

* text

* properly type session user

* styling

* auth

* sync table names for integration tests

* working on integration tests

* integration tests

* /api/apiKeys should only return active api keys

* working api/sendMail test

* do not open

* rm unused import

* prettiiier

* refactor fetch to apiSendMail

* only look at ts/tsx in CLI tsconfig.json include

* fix types

* next build works in cli

* validateSessionSetup when running instead of loading

* web has no login, session, signup atm

* better ts

* delete unused mailingApi.ts

* jest integration should only run .test.ts files (e.g. not util files)

* refactor integration apiCreateUser

* fix integration test

* remove more oauth from migration

* wait for databases to be empty, all integration tests passing

* apiKeys

* fix cliPrisma types, delete with prisma... 100ms -> 30ms

* refactor

* logout integration test

* fix fn name

* generate prisma client in server build

* add more user api tests

* MAILING_DATABASE_URL, runInBand :(

* fix type errors, build works

* - move apiIntegrationTestUtil to __integration__
- ignore __integration__ in .npmignore and .mailing build step

* ignore __integration__ also in tsconfig?

* fix link:emails

* cd into .mailing before running prisma commands

* added prisma to package.json files to copy into node_modules

* email-validator import

* move packages to the right place

* unused_no_db_session_password123 idea

* return 404 if MAILING_SESSION_PASSWORD is undefined (sessions not in use or forgot to define

* refactoring apiIntegrationTestUtil.ts

* refactor helper functions for api integration tests

* don't need ?

* correct ci:test command to user ci:cli

* assert ports are free

* fix ci:test:next

* debug ci

* try import fetch from "node-fetch";

* start the web server

* ci:test set db url for web

* fail-fast false

* also run yarn prisma generate

* fix next.js web deploy vercel
run yarn prisma generate && yarn prisma migrate deploy in build

* psql -> mysql; tests run locally

* db name

* Revert "db name"

This reverts commit d806b45.

* Revert "psql -> mysql; tests run locally"

This reverts commit 1be2c92.

* session should 404 if MAILING_SESSION_PASSWORD is not set

* px-[12px] -> px-2; py-[8px] -> py-2

* border-[1px] -> border

Co-authored-by: Peter Sugihara <[email protected]>
Co-authored-by: Elliot Hursh <[email protected]>
  • Loading branch information
3 people authored Oct 17, 2022
1 parent e30ff34 commit 2ed3d46
Show file tree
Hide file tree
Showing 62 changed files with 4,788 additions and 3,260 deletions.
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
node_modules
**/dist/**
**/generated/**
8 changes: 7 additions & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,12 @@
],
"rules": {
"semi": "off",
"@next/next/google-font-display": "off"
"@next/next/google-font-display": "off",
"react/no-unknown-property": [
2,
{
"ignore": ["jsx"]
}
]
}
}
26 changes: 26 additions & 0 deletions .github/workflows/jest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,32 @@ jobs:
jest:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node-version: [16, 18]
services:
# Label used to access the service container
postgres:
# Docker Hub image
image: postgres
# Provide the password for postgres
env:
POSTGRES_PASSWORD: postgres
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
# Maps tcp port 5432 on service container to the host
- 5432:5432
env:
MM_DEV: 1
MAILING_DATABASE_URL_TEST: postgresql://postgres:postgres@localhost:5432/mailing_cli_test
WEB_DATABASE_URL_TEST: postgresql://postgres:postgres@localhost:5432/mailing_test
MAILING_WEB_SESSION_PASSWORD: test-YEAed65EasPz7zbGF4fuoMTDka7C
MAILING_SESSION_PASSWORD: test-5toTch6whgDvErN1nPg0hbDq6Do6
steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
Expand All @@ -26,3 +50,5 @@ jobs:
run: yarn ci:test
- name: Run CI Next.js tests
run: yarn ci:test:next
- name: Run CI Integration tests
run: yarn ci:test:integration
8 changes: 4 additions & 4 deletions .rubocop.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
AllCops:
NewCops: enable
Exclude:
- 'vendor/**/*'
- '**/vendor/**/*'
- 'node_modules/**/*'
- '**/node_modules/**/*'
- "vendor/**/*"
- "**/vendor/**/*"
- "node_modules/**/*"
- "**/node_modules/**/*"

Metrics/AbcSize:
Max: 50
Expand Down
1 change: 1 addition & 0 deletions jest.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ const config: Config = {
"<rootDir>/.mailing/",
"tmp-testMailQueue.json",
],
testMatch: ["<rootDir>/packages/**/__test__/**/*.test.[jt]s?(x)"],
};

export default config;
37 changes: 37 additions & 0 deletions jest.integration.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import type { Config } from "jest";

const config: Config = {
setupFilesAfterEnv: ["<rootDir>/testSetup.integration.ts"],
testMatch: ["<rootDir>/packages/**/__integration__/**/*.test.[jt]s?(x)"],
preset: "ts-jest",
testEnvironment: "node",
maxConcurrency: 1,
maxWorkers: 1,
// TODO: keep testTimeout low and use jest.setTimeout for long ones
testTimeout: 60000,
transform: {
"^.+\\.(ts|tsx)$": "ts-jest",
"^.+\\.(js)$": "babel-jest",
},
testPathIgnorePatterns: [
"scripts",
"<rootDir>/emails/",
"<rootDir>/.mailing/",
"<rootDir>/packages/cli/.mailing",
"<rootDir>/packages/cli/src/pages/.*/__test__",
"<rootDir>/packages/cli/src/components",
],
watchPathIgnorePatterns: [
"<rootDir>/emails/",
"<rootDir>/packages/cli/src/emails/",
"<rootDir>/packages/cli/src/generator_templates/ts/emails/",
"<rootDir>/.mailing/",
"tmp-testMailQueue.json",
],
};

// Point to the correct DB in dev
process.env.MAILING_DATABASE_URL = process.env.MAILING_DATABASE_URL_TEST;
process.env.WEB_DATABASE_URL = process.env.WEB_DATABASE_URL_TEST;

export default config;
13 changes: 9 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
],
"scripts": {
"lint": "eslint .",
"dev": "cd packages/cli && rm -rf .mailing && src/dev.js",
"dev": "cd packages/cli && yarn dev",
"dev:js": "yarn link:emails:js && yarn dev",
"dev:init": "cd packages/cli && rm -rf previews_html; rm -rf .mailing emails mailing.config.json; src/dev.js",
"dev:export-previews": "yarn link:emails; cd packages/cli && rm -rf .mailing && src/dev.js export-previews",
Expand All @@ -20,11 +20,16 @@
"build": "preconstruct build",
"test": "yarn link:emails; yarn jest",
"test:next": "yarn link:emails; yarn jest -c jest.next.config.ts",
"ci:test": "MAILING_DATABASE_URL=$MAILING_DATABASE_URL_TEST WEB_DATABASE_URL=$WEB_DATABASE_URL_TEST start-test ci:cli :3883 ci:web :3000 test",
"ci:test:next": "MAILING_DATABASE_URL=$MAILING_DATABASE_URL_TEST start-server-and-test ci:cli :3883 test:next",
"ci:test:integration": "./scripts/assert-free-ports.sh; MAILING_CI=true MAILING_DATABASE_URL=$MAILING_DATABASE_URL_TEST WEB_DATABASE_URL=$WEB_DATABASE_URL_TEST start-test ci:cli :3883 ci:web :3000 unsafe:test:integration",
"ci:test:integration:watch": "./scripts/assert-free-ports.sh; MAILING_CI=true MAILING_DATABASE_URL=$MAILING_DATABASE_URL_TEST WEB_DATABASE_URL=$WEB_DATABASE_URL_TEST start-test ci:cli :3883 ci:web :3000 unsafe:test:integration:watch",
"ci:cli": "cd packages/cli && yarn ci:server",
"ci:web": "cd packages/web && yarn ci:server",
"unsafe:test:integration": "yarn link:emails && yarn jest -c jest.integration.config.ts",
"unsafe:test:integration:watch": "yarn unsafe:test:integration --watch --runInBand",
"test:e2e": "bundle && bundle exec ruby e2e/cli.rb run-all",
"ci:test": "start-server-and-test ci:server http://localhost:3883 test",
"ci:test:next": "start-server-and-test ci:server http://localhost:3883 test:next",
"ci:test:e2e:build": "bundle && bundle exec ruby e2e/build.rb",
"ci:server": "cd packages/cli && rm -rf .mailing && src/dev.js --quiet",
"prepublish": "yarn build",
"postinstall": "husky install && preconstruct dev",
"watch": "preconstruct watch",
Expand Down
1 change: 1 addition & 0 deletions packages/cli/.npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
node_modules
.next
__test__
__integration__
10 changes: 10 additions & 0 deletions packages/cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"theme.js",
"postcss.config.js",
"tsconfig.json",
"prisma",
"public",
"dist"
],
Expand All @@ -33,6 +34,7 @@
"@next/eslint-plugin-next": "^12.2.0",
"@testing-library/react": "^13.4.0",
"cypress": "^10.3.1",
"fetch-cookie": "^2.1.0",
"node-mocks-http": "^1.11.0"
},
"peerDependencies": {
Expand All @@ -42,10 +44,12 @@
},
"dependencies": {
"@babel/parser": "^7.18.11",
"@prisma/client": "^4.4.0",
"@reecelucas/react-use-hotkeys": "^1.3.5",
"@swc/core": "^1.2.229",
"@swc/register": "^0.1.10",
"@tailwindcss/line-clamp": "^0.4.2",
"@types/bcrypt": "^5.0.0",
"@types/fs-extra": "^9.0.13",
"@types/html-minifier-terser": "^6.1.0",
"@types/jsdom": "^20.0.0",
Expand All @@ -59,13 +63,16 @@
"@types/prompts": "^2.0.14",
"@types/yargs": "^17.0.10",
"autoprefixer": "^10.4.8",
"bcrypt": "^5.1.0",
"chalk": "^4.1.2",
"chokidar": "^3.5.3",
"classnames": "^2.3.1",
"dotenv": "^16.0.1",
"email-validator": "^2.0.4",
"esbuild": "^0.15.5",
"fs-extra": "^10.1.0",
"html-minifier-terser": "^7.0.0",
"iron-session": "^6.2.1",
"jsdom": "^20.0.0",
"lodash": "^4.17.21",
"mailing-core": "^0.8.4",
Expand All @@ -77,6 +84,7 @@
"postcss": "^8.4.16",
"posthog-node": "^2.0.2",
"prettier": "^2.7.1",
"prisma": "^4.4.0",
"prompts": "^2.4.2",
"react-tiny-popover": "^7.1.0",
"tailwindcss": "^3.1.8",
Expand All @@ -86,6 +94,8 @@
"yargs": "^17.5.1"
},
"scripts": {
"dev": "rm -rf .mailing && yarn prisma migrate dev && src/dev.js",
"ci:server": "NODE_ENV=test rm -rf .mailing && yarn prisma generate && yarn prisma migrate deploy && src/dev.js > test.log",
"ci:server:nohup": "rm -rf .mailing && nohup src/dev.js --quiet > /dev/null 2>&1 &"
},
"keywords": [
Expand Down
15 changes: 15 additions & 0 deletions packages/cli/prisma/__mocks__/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { PrismaClient } from "../generated/client";
import { mockDeep, mockReset, DeepMockProxy } from "jest-mock-extended";

import prisma from "..";

jest.mock("..", () => ({
__esModule: true,
default: mockDeep<PrismaClient>(),
}));

beforeEach(() => {
mockReset(prismaMock);
});

export const prismaMock = prisma as unknown as DeepMockProxy<PrismaClient>;
12 changes: 12 additions & 0 deletions packages/cli/prisma/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { PrismaClient } from "./generated/client";

declare global {
// eslint-disable-next-line no-var
var prisma: PrismaClient<any>;
}

const prisma = global.prisma || new PrismaClient();

if (process.env.NODE_ENV === "development") global.prisma = prisma;

export default prisma;
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
-- CreateTable
CREATE TABLE "User" (
"id" TEXT NOT NULL,
"email" TEXT NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"password" TEXT NOT NULL,
"organizationId" TEXT NOT NULL,

CONSTRAINT "User_pkey" PRIMARY KEY ("id")
);

-- CreateTable
CREATE TABLE "Organization" (
"id" TEXT NOT NULL,
"name" TEXT NOT NULL,

CONSTRAINT "Organization_pkey" PRIMARY KEY ("id")
);

-- CreateTable
CREATE TABLE "ApiKey" (
"id" TEXT NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"active" BOOLEAN NOT NULL DEFAULT true,
"organizationId" TEXT NOT NULL,

CONSTRAINT "ApiKey_pkey" PRIMARY KEY ("id")
);

-- CreateIndex
CREATE UNIQUE INDEX "User_email_key" ON "User"("email");

-- CreateIndex
CREATE INDEX "User_organizationId_idx" ON "User"("organizationId");

-- CreateIndex
CREATE INDEX "ApiKey_organizationId_idx" ON "ApiKey"("organizationId");

-- AddForeignKey
ALTER TABLE "User" ADD CONSTRAINT "User_organizationId_fkey" FOREIGN KEY ("organizationId") REFERENCES "Organization"("id") ON DELETE RESTRICT ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "ApiKey" ADD CONSTRAINT "ApiKey_organizationId_fkey" FOREIGN KEY ("organizationId") REFERENCES "Organization"("id") ON DELETE RESTRICT ON UPDATE CASCADE;
3 changes: 3 additions & 0 deletions packages/cli/prisma/migrations/migration_lock.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Please do not edit this file manually
# It should be added in your version-control system (i.e. Git)
provider = "postgresql"
41 changes: 41 additions & 0 deletions packages/cli/prisma/schema.prisma
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema

generator client {
provider = "prisma-client-js"
output = "./generated/client"
}

datasource db {
provider = "postgresql"
url = env("MAILING_DATABASE_URL")
}

model User {
id String @id @default(cuid())
email String @unique
createdAt DateTime @default(now())
password String
organizationId String
Organization Organization @relation(fields: [organizationId], references: [id])
@@index([organizationId])
}

model Organization {
id String @id @default(cuid())
name String
User User[]
ApiKey ApiKey[]
}

model ApiKey {
id String @id @default(cuid())
createdAt DateTime @default(now())
active Boolean @default(true)
organizationId String
Organization Organization @relation(fields: [organizationId], references: [id])
@@index([organizationId])
}
9 changes: 6 additions & 3 deletions packages/cli/src/commands/init.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import prompts from "prompts";
import fetch from "node-fetch";
import { existsSync } from "fs-extra";
import { ArgumentsCamelCase } from "yargs";
import { error, log } from "../util/log";
Expand Down Expand Up @@ -69,8 +70,8 @@ export const handler = buildHandler(
await generateEmailsDirectory(options);

if (argv.scaffoldOnly) {
return
}
return;
}

if (!argv.quiet) {
const emailResponse = await prompts({
Expand All @@ -83,7 +84,9 @@ export const handler = buildHandler(
if (email?.length > 0) {
log("great, talk soon");
try {
fetch(`${getMailingAPIBaseURL()}/api/users`, {
const url = `${getMailingAPIBaseURL()}/api/newsletterSubscribers`;

fetch(url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ email }),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,15 @@ function mockReadJSONSyncToThrowErrno() {
}

describe("setup", () => {
const MM_DEV_OG = process.env.MM_DEV;

beforeEach(() => {
jest.restoreAllMocks();
delete process.env.MM_DEV;
});

afterEach(() => {
process.env.MM_DEV = MM_DEV_OG;
});

describe("packageJsonVersionsMatch", () => {
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/src/commands/preview/server/setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ import { getNodeModulesDirsFrom } from "../../util/getNodeModulesDirsFrom";

export const COMPONENT_FILE_REGEXP = /^[^\s-]+\.[tj]sx$/; // no spaces, .jsx or .tsx
export const DOT_MAILING_IGNORE_REGEXP =
/__test__|generator_templates|yarn-error.log|src\/index\.ts$|src\/dev\.js$|\.mailing$|\.next|node_modules$|^cypress/;
/__test__|__integration__|generator_templates|yarn-error.log|src\/index\.ts$|src\/dev\.js$|\.mailing$|\.next|node_modules$|^cypress/;

export type PreviewServerOptions = {
emailsDir: string;
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/src/commands/preview/server/start.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ export default async function startPreviewServer() {
.listen(port, async () => {
clearTimeout(loadLag);
log(`running preview at ${currentUrl}`);
if (!quiet) await open(currentUrl);
if (!quiet && !process.env.MAILING_CI) await open(currentUrl);
})
.on("error", async function onServerError(e: NodeJS.ErrnoException) {
if (e.code === "EADDRINUSE") {
Expand Down
Loading

2 comments on commit 2ed3d46

@vercel
Copy link

@vercel vercel bot commented on 2ed3d46 Oct 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vercel
Copy link

@vercel vercel bot commented on 2ed3d46 Oct 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Successfully deployed to the following URLs:

web – ./packages/web

web-sofn.vercel.app
web-git-main-sofn.vercel.app
mailing.run
web-rho-puce.vercel.app
www.mailing.run

Please sign in to comment.