Skip to content

Commit

Permalink
Linguist - Removed Deprecated Code (#1804)
Browse files Browse the repository at this point in the history
* Linguist - Removed Deprecated Code

Signed-off-by: Andre Wanlin <[email protected]>

* Fixed typo

Signed-off-by: Andre Wanlin <[email protected]>

* Removed export

Signed-off-by: Andre Wanlin <[email protected]>

---------

Signed-off-by: Andre Wanlin <[email protected]>
  • Loading branch information
awanlin authored Nov 15, 2024
1 parent 9f019c3 commit 9602620
Show file tree
Hide file tree
Showing 15 changed files with 104 additions and 509 deletions.
5 changes: 5 additions & 0 deletions workspaces/linguist/.changeset/dry-teachers-tap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@backstage-community/plugin-linguist-backend': minor
---

**BREAKING** Removed support for what is known as the legacy backend, please use the New Backend System.
2 changes: 0 additions & 2 deletions workspaces/linguist/packages/backend/knip-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@
| @backstage/plugin-permission-common | package.json | error |
| @backstage/plugin-permission-node | package.json | error |
| @backstage/plugin-auth-node | package.json | error |
| @backstage/backend-common | package.json | error |
| @backstage/backend-tasks | package.json | error |
| @backstage/config | package.json | error |
| better-sqlite3 | package.json | error |
| dockerode | package.json | error |
Expand Down
1 change: 0 additions & 1 deletion workspaces/linguist/packages/backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
"@backstage-community/plugin-catalog-backend-module-linguist-tags-processor": "workspace:^",
"@backstage-community/plugin-linguist-backend": "workspace:^",
"@backstage/backend-defaults": "^0.5.2",
"@backstage/backend-tasks": "^0.6.1",
"@backstage/config": "^1.2.0",
"@backstage/plugin-app-backend": "^0.3.76",
"@backstage/plugin-auth-backend": "^0.23.1",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,3 @@
| Name | Location | Severity |
| :------------------------------------------ | :----------- | :------- |
| @backstage-community/plugin-linguist-common | package.json | error |
| @backstage/backend-common | package.json | error |

## Unused devDependencies (1)

| Name | Location | Severity |
| :----------------------- | :----------- | :------- |
| @backstage/backend-tasks | package.json | error |
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
"node-fetch": "^2.6.7"
},
"devDependencies": {
"@backstage/backend-tasks": "^0.6.1",
"@backstage/backend-test-utils": "^1.0.2",
"@backstage/cli": "^0.28.0",
"js-yaml": "^4.1.0",
Expand Down
155 changes: 64 additions & 91 deletions workspaces/linguist/plugins/linguist-backend/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Linguist Backend

Welcome to the Linguist backend plugin! This plugin provides data for the Linguist frontend features. Additionally, it provides an optional entity processor which will automate adding language tags to your entities.
Welcome to the Linguist backend plugin! This plugin provides data for the Linguist frontend features.

## Setup

Expand All @@ -17,136 +17,109 @@ Here's how to get the backend up and running:
yarn --cwd packages/backend add @backstage-community/plugin-linguist-backend
```

2. Then we will create a new file named `packages/backend/src/plugins/linguist.ts`, and add the
following to it:

```ts
import { TaskScheduleDefinition } from '@backstage/backend-tasks';
import { createRouter } from '@backstage-community/plugin-linguist-backend';
import { Router } from 'express';
import type { PluginEnvironment } from '../types';
export default async function createPlugin(
env: PluginEnvironment,
): Promise<Router> {
const schedule: TaskScheduleDefinition = {
frequency: { minutes: 2 },
timeout: { minutes: 15 },
initialDelay: { seconds: 90 },
};
return createRouter({ schedule: schedule }, { ...env });
}
```
2. Then in your `packages/backend/src/index.ts` make the following changes:

3. Next we wire this into the overall backend router, edit `packages/backend/src/index.ts`:
```ts
import linguist from './plugins/linguist';
// ...
async function main() {
// ...
// Add this line under the other lines that follow the useHotMemoize pattern
const linguistEnv = useHotMemoize(module, () => createEnv('linguist'));
// ...
// Insert this line under the other lines that add their routers to apiRouter in the same way
apiRouter.use('/linguist', await linguist(linguistEnv));
```
```diff
import { createBackend } from '@backstage/backend-defaults';
4. Now run `yarn start-backend` from the repo root
5. Finally open `http://localhost:7007/api/linguist/health` in a browser and it should return `{"status":"ok"}`
const backend = createBackend();
#### New Backend System
// ... other feature additions
The Linguist backend plugin has support for the [new backend system](https://backstage.io/docs/backend-system/), here's how you can set that up:
+ backend.add(import('@backstage-community/plugin-linguist-backend'));
In your `packages/backend/src/index.ts` make the following changes:
backend.start();
```
```diff
import { createBackend } from '@backstage/backend-defaults';
3. Then update your `app-config.yaml` with your desired options, more details in the [Plugin Options](#plugin-options) section below, here's a basic example to get started:
```yaml
# ... other config
linguist:
schedule:
frequency:
minutes: 2
timeout:
minutes: 2
initialDelay:
seconds: 15
age:
days: 30
batchSize: 2
useSourceLocation: false
# ... other config
```
const backend = createBackend();
4. Now run `yarn start-backend` from the repo root
5. Finally open `http://localhost:7007/api/linguist/health` in a browser and it should return `{"status":"ok"}`
// ... other feature additions
## Plugin Options
+ backend.add(import('@backstage-community/plugin-linguist-backend'));
The Linguist backend has various plugin options that you can provide in the `app-config.yaml` file that will allow you to configure various aspects of how it works. The following sections go into the details of these options
backend.start();
```
### Batch Size
The plugin options can be set through the `app-config.yaml`:
The Linguist backend is setup to process entities by acting as a queue where it will pull down all the applicable entities from the Catalog and add them to it's database (saving just the `entityRef`). Then it will grab the `n` oldest entities that have not been processed to determine their languages and process them. To control the batch size simply provide that to the `app-config.yaml` file like this:
```yaml
// ...
# ... other config
linguist:
schedule:
frequency:
minutes: 2
timeout:
minutes: 2
initialDelay:
seconds: 15
age:
days: 30
batchSize: 2
useSourceLocation: false
// ...
```
## Plugin Option
The Linguist backend has various plugin options that you can provide to the `createRouter` function in your `packages/backend/src/plugins/linguist.ts` file that will allow you to configure various aspects of how it works. The following sections go into the details of these options
### Batch Size
The Linguist backend is setup to process entities by acting as a queue where it will pull down all the applicable entities from the Catalog and add them to it's database (saving just the `entityRef`). Then it will grab the `n` oldest entities that have not been processed to determine their languages and process them. To control the batch size simply provide that to the `createRouter` function in your `packages/backend/src/plugins/linguist.ts` like this:
```ts
return createRouter({ schedule: schedule, batchSize: 40 }, { ...env });
# ... other config
```
**Note:** The default batch size is 20
### Kind
The default setup only processes entities of kind `['API', 'Component', 'Template']`. To control the `kind` that are processed provide that to the `createRouter` function in your `packages/backend/src/plugins/linguist.ts` like this:
The default setup only processes entities of kind `['API', 'Component', 'Template']`. To control the `kind` that are processed add that in your `app-config.yaml` file like this:
```yaml
# ... other config
```ts
return createRouter({ schedule: schedule, kind: ['Component'] }, { ...env });
linguist:
kind: ['Component']
# ... other config
```
### Refresh
The default setup will only generate the language breakdown for entities with the linguist annotation that have not been generated yet. If you want this process to also refresh the data you can do so by adding the `age` (as a `HumanDuration`) in your `packages/backend/src/plugins/linguist.ts` when you call `createRouter`:
The default setup will only generate the language breakdown for entities with the linguist annotation that have not been generated yet. If you want this process to also refresh the data you can do so by adding the `age` (as a `HumanDuration`) in your `app-config.yaml` file like this:
```yaml
# ... other config
```ts
return createRouter({ schedule: schedule, age: { days: 30 } }, { ...env });
linguist:
age: { days: 30 }
# ... other config
```
With the `age` setup like this if the language breakdown is older than 15 days it will get regenerated. It's recommended that if you choose to use this configuration to set it to a large value - 30, 90, or 180 - as this data generally does not change drastically.
### Linguist JS options
The default setup will use the default [linguist-js](https://www.npmjs.com/package/linguist-js) options, a full list of the available options can be found [here](https://www.npmjs.com/package/linguist-js#API).
The default setup will use the default [linguist-js](https://www.npmjs.com/package/linguist-js) options, a full list of the available options can be found [here](https://www.npmjs.com/package/linguist-js#API). Here's a simple example of using this in your `app-config.yaml` file:
```yaml
# ... other config
```ts
return createRouter(
{ schedule: schedule, linguistJsOptions: { offline: true } },
{ ...env },
);
linguist:
linguistJsOptions: { offline: true }
# ... other config
```
### Use Source Location
You may wish to use the `backstage.io/source-location` annotation over using the `backstage.io/linguist` as you may not be able to quickly add that annotation to your Entities. To do this you'll just need to set the `useSourceLocation` boolean to `true` in your `packages/backend/src/plugins/linguist.ts` when you call `createRouter`:
You may wish to use the `backstage.io/source-location` annotation over using the `backstage.io/linguist` as you may not be able to quickly add that annotation to your Entities. To do this you'll just need to set the `useSourceLocation` boolean to `true` in your `app-config.yaml` file:
```ts
return createRouter(
{ schedule: schedule, useSourceLocation: true },
{ ...env },
);
```yaml
# ... other config
linguist:
useSourceLocation: true
# ... other config
```
**Note:** This has the potential to cause a lot of processing, be very thoughtful about this before hand
Expand Down
34 changes: 2 additions & 32 deletions workspaces/linguist/plugins/linguist-backend/config.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
* limitations under the License.
*/

import { TaskScheduleDefinition } from '@backstage/backend-tasks';
import { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api';
import { HumanDuration } from '@backstage/types';
import { Options as LinguistJsOptions } from 'linguist-js/dist/types';

export interface Config {
/** Configuration options for the linguist plugin */
linguist?: {
schedule?: TaskScheduleDefinition;
schedule?: SchedulerServiceTaskScheduleDefinition;
/**
* @default 20
*/
Expand All @@ -42,35 +42,5 @@ export interface Config {
* [linguist-js](https://www.npmjs.com/package/linguist-js) options
*/
linguistJsOptions?: LinguistJsOptions;

/** Options for the tags processor */
tagsProcessor?: {
/**
* Determines how many bytes of a language should be in a repo
* for it to be added as an entity tag. Defaults to 0.
*/
bytesThreshold?: number;
/**
* The types of linguist languages that should be processed. Can be
* any of "programming", "data", "markup", "prose". Defaults to ["programming"].
*/
languageTypes?: string[];
/**
* A custom mapping of linguist languages to how they should be rendered as entity tags.
* If a language is mapped to '' it will not be included as a tag.
*/
languageMap?: {
[language: string]: string | undefined;
};
/**
* How long to cache entity languages for in memory. Used to avoid constant db hits during
* processing. Defaults to 30 minutes.
*/
cacheTTL?: HumanDuration;
/**
* An optional prefix to apply to all created tags from linguist
*/
tagPrefix?: string;
};
};
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,4 @@
| Name | Location | Severity |
| :------------------------------------------ | :----------- | :------- |
| @backstage-community/plugin-linguist-common | package.json | error |
| @backstage/backend-tasks | package.json | error |
| yn | package.json | error |
2 changes: 0 additions & 2 deletions workspaces/linguist/plugins/linguist-backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,8 @@
},
"dependencies": {
"@backstage-community/plugin-linguist-common": "workspace:^",
"@backstage/backend-common": "^0.25.0",
"@backstage/backend-defaults": "^0.5.2",
"@backstage/backend-plugin-api": "^1.0.1",
"@backstage/backend-tasks": "^0.6.1",
"@backstage/catalog-client": "^1.7.1",
"@backstage/catalog-model": "^1.7.0",
"@backstage/config": "^1.2.0",
Expand Down
60 changes: 0 additions & 60 deletions workspaces/linguist/plugins/linguist-backend/report.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,30 +3,8 @@
> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/).
```ts
import { AuthService } from '@backstage/backend-plugin-api';
import { BackendFeature } from '@backstage/backend-plugin-api';
import { Config } from '@backstage/config';
import { DatabaseService } from '@backstage/backend-plugin-api';
import { DiscoveryService } from '@backstage/backend-plugin-api';
import express from 'express';
import { HttpAuthService } from '@backstage/backend-plugin-api';
import { HumanDuration } from '@backstage/types';
import { Languages } from '@backstage-community/plugin-linguist-common';
import { LoggerService } from '@backstage/backend-plugin-api';
import { SchedulerService } from '@backstage/backend-plugin-api';
import { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api';
import { UrlReaderService } from '@backstage/backend-plugin-api';

// @public @deprecated (undocumented)
export function createRouter(
pluginOptions: PluginOptions,
routerOptions: RouterOptions,
): Promise<express.Router>;

// @public @deprecated (undocumented)
export function createRouterFromConfig(
routerOptions: RouterOptions,
): Promise<express.Router>;

// @public (undocumented)
export interface LinguistBackendApi {
Expand All @@ -39,42 +17,4 @@ export interface LinguistBackendApi {
// @public
const linguistPlugin: BackendFeature;
export default linguistPlugin;

// @public @deprecated (undocumented)
export interface PluginOptions {
// (undocumented)
age?: HumanDuration;
// (undocumented)
batchSize?: number;
// (undocumented)
kind?: string[];
// (undocumented)
linguistJsOptions?: Record<string, unknown>;
// (undocumented)
schedule?: SchedulerServiceTaskScheduleDefinition;
// (undocumented)
useSourceLocation?: boolean;
}

// @public @deprecated (undocumented)
export interface RouterOptions {
// (undocumented)
auth?: AuthService;
// (undocumented)
config: Config;
// (undocumented)
database: DatabaseService;
// (undocumented)
discovery: DiscoveryService;
// (undocumented)
httpAuth?: HttpAuthService;
// (undocumented)
linguistBackendApi?: LinguistBackendApi;
// (undocumented)
logger: LoggerService;
// (undocumented)
reader: UrlReaderService;
// (undocumented)
scheduler?: SchedulerService;
}
```
Loading

0 comments on commit 9602620

Please sign in to comment.