Skip to content

Latest commit

 

History

History
601 lines (413 loc) · 17.6 KB

notes.md

File metadata and controls

601 lines (413 loc) · 17.6 KB
Raw

Notes

Designed for developing TypeScript NPM libraries. The layout can be applied to other web development projects.

Table of Contents

Package Layout

  • .github
    • GitHub specific files
  • .vscode
    • Visual Studio Code configuration files
  • config
    • configuration files for tools
  • dist
    • compiled package content (this is never checked in)
  • docs
    • manually written internal documentation for development
  • documentation
    • autogenerated public documentation for API exposed by the package
  • scripts
    • manually run scripts
  • src
    • package source, should have a single index.ts file that exposes all public APIs.
  • test
    • package tests

docs folder use

The docs folder contains documentation needed for development of the package.

Why place all documents in a single folder? (Instead of spreading them throughout the project)

  • Single place to look for all documents
  • Allow link references to other markdown documents without having to worry about code structure
  • GitHub uses the docs folder to publish a wiki which is a convenient way to view information about the project

package.json

npm package.json field documentation

Convention

For packages that are not published the following field should be present "private:":true and the package version should be set to 0.0.0 to help identify that this is a NPM based project and NOT a published package.

Additional Fields

Commands

build

Standard development command.

[clean, style, spell-check, lint, compile]

test

run tests

test-coverage

get test coverage

separate command because test-coverage can take longer to run than simply running tests

style

Apply consistent formatting in {src, test, scripts}.

spell-check

Check for spelling errors in {src, test, scripts}.

lint

Detect common issues in {src, test, scripts}.

clean

Remove contents of dist and temp folders.

Wipe out the contents of dist and temp before building to avoid persistance of removed files.

compile

Run the TypeScript compiler and generate the dist.

doc

Generates the doc folder

Uses compiled dist/index.d.ts

Checks for changes in the packages publish signature

prepack

npm built in command automatically run before publish during npm publish

should be used to build everything and test the package.

Standard Technologies

Tools

note: the template will need to be updated as technology changes.

Libraries

Libraries for solving common problems

Visual Studio Code Extensions

  • ESLint
    • id: dbaeumer.vscode-eslint
    • config: config\eslint.json
    • requires command line option --config
    • requires .vscode\settings.json "eslint.options": { "configFile": "config/eslint.json", "resolvePluginsRelativeTo": "${workspaceFolder}" },
  • Prettier
    • id: esbenp.prettier-vscode
    • config: config\prettier.json
    • requires command line option --config
    • requires .vscode\settings.json "prettier.configPath": "config/prettier.json",
  • markdownlint
    • id: DavidAnson.vscode-markdownlint
    • config: config\.markdownlint.json
    • requires .vscode\settings.json "markdownlint.config": { "extends": "./config/.markdownlint.json" },
  • Code Spell Checker
    • id: streetsidesoftware.code-spell-checker
    • config: .vscode\settings.json add additional words cSpell.words
  • Playwright
    • id: ms-playwright.playwright
    • config: config\playwright.config.ts

Tool Notes

NVM

NVM setup

NVM is used to deal with node versions.

nvm install lts

nvm use lts

Node

Node Commands

End all node processes:

taskkill /f /im node.exe

NPM

NPM Commands

Show all published versions of a package.

npm show @wandyezj/package@* version

Prettier

prettier disables

prettier ignore code

// disables prettier for the next node
// prettier-ignore

prettier settings

  • "printWidth":100
    • Prettier's built in setting of printWidth:80 wraps more than needed for readability.
  • "tabWidth": 4
    • Spaces have consistent spacing across editors compare to tabs. Four spaces of indentation highlights potentially too deep nesting.
  • "endOfLine":"lf"
    • Unix line ending are preferred.

cspell

Ignore a specific word in a file.

// cspell:ignore wordToIgnore

cspell Inline Document Settings

Playwright

Jest

jest commands

run a specific jest file

npm run test -- file

jest coverage disables

When Jest test coverage is enforced, the following will consider the next node covered.

ignore next node

/* istanbul ignore next */

ignore file

/* istanbul ignore file */

Jest run single or group of tests

--testNamePattern

npm run test -t name*

eslint

eslint disables

eslint disabling rules

// disables for the next line
// eslint-disable-next-line no-use-before-define

// eslint-disable-line @typescript-eslint/triple-slash-reference

eslint rules

  • eqeqeq
    • == double equals leads to bugs, double equals is usually a mistake.

TS Doc

@packageDocumentation - Document package

GitHub

Unit Test Debugging in VS Code

  1. Open in VS Code
  2. Select unit test file to debug
  3. Place a breakpoint on the test to debug
  4. Click Run and Debug (Ctrl + Shift + D) in VS Code
  5. Select the Jest Config
  6. Click green arrow Start Debugging (F5)

Features

  • Visual Studio Code F5 Debugging for jest tests.

Node Version

This package is intended to work on the latest node version and included NPM version.

NPM Version Issues

[email protected] does not allow script execution with unix style paths as used in the package.json. To support this behavior Downgrade npm install -g [email protected] or Upgrade to at least [email protected] to support this behavior.

The npm example highlights the importance of versioning tools together.

NPM Script Search Path

npm looks first in the node_modules directory for the tool. Then it looks in the global installs before falling back to the path.

In order to require that only the tools specified as part of the package be run it's important to use the specific path ./node_modules/.bin See GitHub issue

Benefits:

  • A fixed tool version allows everyone to use the same version of the tool.
  • Automation workflows only node needs to be installed (comes with npm) and only npm ci needs to be run before the npm run <script> commands ave available to use.
  • Avoids cluttering the global namespace with tools.
  • Ensures that the same tool version is run for everyone across environments.

However, for convenience, to work on Windows and Linux with the latest tool versions, the ./node_modules/.bin is left off.

Windows cmd search path

Windows cmd does not need the extension on the tool to work. cmd.exe has a precedence list for executing items in the path without the extension's presence by looking for the command without the extension (i.e. hello will call hello.exe, hello.bat, hello.cmd, etc.. in that order as it searches the path).

Technology and Design Considerations

Technologies were chosen based on ubiquity and commitment to long term support.

Widely deployed technologies with long term support are more likely to remain stable platforms in the future.

Adopting every new technology that has some small benefit comes with a cost. It's preferable to pick good standards that will remain relevant into the future to enable developers to focus on building new things instead of selecting and configuring tools. Standard widely adopted and well supported technologies are more likely to: have good support for common scenarios, have significant documentation, work together, and evolve together.

  • Strong types, Strong contracts, implicitly difficult in a dynamically typed language
  • Minimize Dependencies, document any reasons for dependencies, why they are required and how they are used
  • Reduce Attack Surface Area, everyone is responsible to security, do not invent or implement custom hashing or cryptographic algorithms leave these things to experts, do not use duplicate tools when one will do, this increases the attack surface are unnecessarily.
  • Privacy, do not send any data, do not cache or store any data
  • Semantic Versioning, avoid unless practicing it strictly, every change is a potential breaking change, perhaps even unexpectedly, semantic versioning is a way to signal, but it needs to adhere to a specified contract about what is considered breaking, but it is still inaccurate in terms of how dependencies are taken, execution changes verses compilation changes

Move all possible configs under the config folder:

  • have clarity where configs exist
  • have clarity on what calls the configs
  • remove clutter in the root folder

Technology Choice

GitHub Actions

Make sure the operating system and the node version used in GitHub Actions matches those used on the development machine.

Source Control

Git is used as the source control of choice. There are many source control systems available, however the default and most ubiquitous as of 2022-03-01 is Git.

Editor

Visual Studio Code is cross platform and is ubiquitous editor.

TypeScript

Strong Types aid development.

An alternative to TypeScript is JSDOC tags.

Operating System

Ubuntu

Node currently (2022-03-01) has significantly better performance on Linux than on Windows. This results in faster job execution time, and thus reduced cost, and wait time for GitHub Action pipelines.

Ubuntu is a ubiquitous distribution and available as a build pool, and in WSL.

Commands use linux paths. On Windows WSL can be used to execute commands.

Windows Subsystem for Linux

wls can be installed and used to run and install node.

wsl -- <command> executes a command in the default linux environment.

This can be used to run linux commands on Windows.

Useful Packages

Comments on Common Packages

Generally, it's preferable to limit the number of dependencies.

Each dependency is another security vulnerability especially if using them in build scripts.

express

For simple servers express can easily be replaced with node 'http'.

Example express alternative using pure node.

import http from "http";
import path from "path";
import * as fs from "fs";

// all valid resources stored in a resources folder.

// don't bother with express just write a simple node server.
export function startServer(port: number, serveResources: string[]) {
    const server = http.createServer((req, res) => {
        const {url} =  req;

        const found = serveResources.filter((value) => `/${value}` === url );

        if (found.length === 1) {
            const fileName = found[0];

            const filePath = path.join(__dirname + `/resources/${fileName}`);
            const fileData = fs.readFileSync(filePath)

            res.write(fileData);
        } else {
            res.write(`Not Found\n\nValid Resources:\n\n${serveResources.join("\n")}`);
        }

        res.end();
    });

    server.listen(port);

    return server;
}

fs-extra

Unneeded, use built in node 'fs' functions.

// just use node built in
fs.rmSync(path, {recursive: true});

fs.cpSync(from, to);

shelljs

Unneeded, use built in node 'child_process' functions.

// const {execSync} = require("child_process");
import {execSync} from "child_process";

try {
  execSync(commands, { encoding: 'utf-8', stdio: [0, 1, 2] });
} catch (e) {
  process.exit(e.status);
}

fetch

import https from "https";

// mirrors what is used from fetch in node-fetch
// https://nodejs.org/dist/latest-v16.x/docs/api/https.html
async function getUrl(url: string) {
    return new Promise<{ status: number; buffer: () => Promise<string> }>(
        (resolve, reject) => {
            https.get(url, (response) => {
                const statusCode = response.statusCode;

                response.on("error", (err) => {
                    reject(err);
                });

                const dataBuffer = new Promise<string>((resolveBuffer) => {
                    let data = "";
                    response.on("data", (chunk) => {
                        data += chunk;
                    });

                    response.on("end", () => {
                        resolveBuffer(data);
                    });
                });

                resolve({
                    status: statusCode || -1,
                    buffer: () => dataBuffer,
                });
            });
        }
    );
}

/**
 * fetches data at url or throws
 */
async function fetchFileDataAtUrl(url: string): Promise<string> {
    const {status, buffer} = await getUrl(url);

    if (status === 200) {
        return (await buffer()).toString();
    }

    throw new Error(`Error status ${status} when retrieving url ${url}`);
}