Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Reporting]: Move router + license checks to new platform #66331

Merged
merged 50 commits into from
May 29, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
004dd70
WIP: Move routes to new API, license and other checks inbound
May 8, 2020
938e14a
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 11, 2020
500b933
Move license checks over to np licensing observable
May 12, 2020
b9cc20c
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 12, 2020
4feb02f
Fix license checks + remove older modules
May 14, 2020
02394d5
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 14, 2020
2b18fe2
Fixing check_license tests, move to TS/Jest
May 19, 2020
dfd62f3
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 19, 2020
ac361a8
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 19, 2020
7fab2c6
Fix licensing setup for mocks
May 20, 2020
48c88f8
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 20, 2020
15ece89
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 20, 2020
d58fc14
Move job.test.ts over to np
May 21, 2020
6e1667e
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 21, 2020
007dd57
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 26, 2020
ed1ac8c
WIP: move user checks to higher-order func
May 26, 2020
c08ea7b
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 26, 2020
756551f
Move more handler logic over to Response factory vs Boom
May 26, 2020
7e6efa2
Major refactor to consolidate types, remove facades, and udpate helpers
May 27, 2020
82a1767
Fix validation for dates in immediate exports
May 27, 2020
bb0e7ee
Linter fix on check license test
May 27, 2020
ec94204
Fix job generation tests
May 27, 2020
39a1313
Move deps => setupDeps
May 27, 2020
c7cba25
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 27, 2020
b4e54e1
fix api test
tsullivan May 26, 2020
ee7db9a
fix jobs test
tsullivan May 27, 2020
e7262f4
authorized_user_pre_routing and tests
tsullivan May 27, 2020
d69be5f
Fixing duplicate identifiers
May 28, 2020
107eca0
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 28, 2020
ca58c89
Merge pull request #1 from tsullivan/chore/np-route-migration
joelgriffith May 28, 2020
a8f4bcd
Fix licensing implementation changes
May 28, 2020
54863d0
Merge branch 'chore/np-route-migration' of https://github.com/joelgri…
May 28, 2020
86920d3
WIP: Moving license over to async/observables
May 28, 2020
7532d55
Fix disabled-security case
May 28, 2020
dc57a6f
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 28, 2020
f764bf8
finish auth_user_pre_routing cleanup - no more license check
tsullivan May 28, 2020
01fc733
Merge pull request #2 from tsullivan/chore/np-route-migration
May 28, 2020
874222f
WIP: Fixing final api tests
May 28, 2020
dce4083
Trying to get schema differences in alignment
May 28, 2020
76904b7
Merge branch 'chore/np-route-migration' of https://github.com/joelgri…
May 28, 2020
2a5f9a0
Reverting back to previous generation handler
May 28, 2020
f7d1134
Fix final API tests
May 28, 2020
0aca67c
Final API test fixes, few more hardening tests and better error messages
May 29, 2020
7554127
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
May 29, 2020
27040a9
Simplify lower-level module implementation (core only interface) + te…
May 29, 2020
8ad57ff
Push some core logic into plugin
May 29, 2020
4f12c8a
Move some core logic up to plugin
May 29, 2020
3ab8095
Marking private setupDeps + downstream fixes
May 29, 2020
1cf1546
revert logger as a param
tsullivan May 29, 2020
fcf9f58
Merge pull request #3 from tsullivan/chore/np-route-migration
May 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions x-pack/legacy/plugins/reporting/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ export const WHITELISTED_JOB_CONTENT_TYPES = [
'application/pdf',
CONTENT_TYPE_CSV,
'image/png',
'text/plain',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed now so we can pass long certain job-status messages (they were incorrectly being sent as application/json, and I think hapi was fixing that under-the-hood)

];

// See:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,27 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { ReportingCore } from '../../../server';
import { cryptoFactory } from '../../../server/lib';
import {
ConditionalHeaders,
CreateJobFactory,
ESQueueCreateJobFn,
RequestFacade,
} from '../../../server/types';
import { CreateJobFactory, ESQueueCreateJobFn } from '../../../server/types';
import { JobParamsDiscoverCsv } from '../types';

export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
JobParamsDiscoverCsv
>> = function createJobFactoryFn(reporting: ReportingCore) {
const config = reporting.getConfig();
const crypto = cryptoFactory(config.get('encryptionKey'));
const setupDeps = reporting.getPluginSetupDeps();

return async function createJob(
jobParams: JobParamsDiscoverCsv,
headers: ConditionalHeaders['headers'],
request: RequestFacade
context: RequestHandlerContext,
request: KibanaRequest
) {
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const serializedEncryptedHeaders = await crypto.encrypt(request.headers);

const savedObjectsClient = request.getSavedObjectsClient();
const savedObjectsClient = context.core.savedObjects.client;
const indexPatternSavedObject = await savedObjectsClient.get(
'index-pattern',
jobParams.indexPatternId!
Expand All @@ -36,7 +33,7 @@ export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
return {
headers: serializedEncryptedHeaders,
indexPatternSavedObject,
basePath: request.getBasePath(),
basePath: setupDeps.basePath(request),
...jobParams,
};
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@

import { notFound, notImplemented } from 'boom';
import { get } from 'lodash';
import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { CSV_FROM_SAVEDOBJECT_JOB_TYPE } from '../../../../common/constants';
import { ReportingCore } from '../../../../server';
import { cryptoFactory, LevelLogger } from '../../../../server/lib';
import { CreateJobFactory, RequestFacade, TimeRangeParams } from '../../../../server/types';
import { CreateJobFactory, TimeRangeParams } from '../../../../server/types';
import {
JobDocPayloadPanelCsv,
JobParamsPanelCsv,
Expand All @@ -23,8 +24,9 @@ import { createJobSearch } from './create_job_search';

export type ImmediateCreateJobFn<JobParamsType> = (
jobParams: JobParamsType,
headers: Record<string, string>,
req: RequestFacade
headers: KibanaRequest['headers'],
context: RequestHandlerContext,
req: KibanaRequest
) => Promise<{
type: string | null;
title: string;
Expand All @@ -46,21 +48,21 @@ export const createJobFactory: CreateJobFactory<ImmediateCreateJobFn<

return async function createJob(
jobParams: JobParamsPanelCsv,
headers: any,
req: RequestFacade
headers: KibanaRequest['headers'],
context: RequestHandlerContext,
req: KibanaRequest
): Promise<JobDocPayloadPanelCsv> {
const { savedObjectType, savedObjectId } = jobParams;
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const client = req.getSavedObjectsClient();

const { panel, title, visType }: VisData = await Promise.resolve()
.then(() => client.get(savedObjectType, savedObjectId))
.then(() => context.core.savedObjects.client.get(savedObjectType, savedObjectId))
.then(async (savedObject: SavedObject) => {
const { attributes, references } = savedObject;
const {
kibanaSavedObjectMeta: kibanaSavedObjectMetaJSON,
} = attributes as SavedSearchObjectAttributesJSON;
const { timerange } = req.payload as { timerange: TimeRangeParams };
const { timerange } = req.body as { timerange: TimeRangeParams };

if (!kibanaSavedObjectMetaJSON) {
throw new Error('Could not parse saved object data!');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,11 @@
*/

import { i18n } from '@kbn/i18n';
import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { CONTENT_TYPE_CSV, CSV_FROM_SAVEDOBJECT_JOB_TYPE } from '../../../common/constants';
import { ReportingCore } from '../../../server';
import { cryptoFactory, LevelLogger } from '../../../server/lib';
import {
ExecuteJobFactory,
JobDocOutput,
JobDocPayload,
RequestFacade,
} from '../../../server/types';
import { ExecuteJobFactory, JobDocOutput, JobDocPayload } from '../../../server/types';
import { CsvResultFromSearch } from '../../csv/types';
import { FakeRequest, JobDocPayloadPanelCsv, JobParamsPanelCsv, SearchPanel } from '../types';
import { createGenerateCsv } from './lib';
Expand All @@ -25,7 +21,8 @@ import { createGenerateCsv } from './lib';
export type ImmediateExecuteFn<JobParamsType> = (
jobId: null,
job: JobDocPayload<JobParamsType>,
request: RequestFacade
context: RequestHandlerContext,
req: KibanaRequest
) => Promise<JobDocOutput>;

export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<
Expand All @@ -39,7 +36,8 @@ export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<
return async function executeJob(
jobId: string | null,
job: JobDocPayloadPanelCsv,
realRequest?: RequestFacade
context,
req
): Promise<JobDocOutput> {
// There will not be a jobID for "immediate" generation.
// jobID is only for "queued" jobs
Expand All @@ -58,10 +56,11 @@ export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<

jobLogger.debug(`Execute job generating [${visType}] csv`);

let requestObject: RequestFacade | FakeRequest;
if (isImmediate && realRequest) {
let requestObject: KibanaRequest | FakeRequest;

if (isImmediate && req) {
jobLogger.info(`Executing job from Immediate API using request context`);
requestObject = realRequest;
requestObject = req;
} else {
jobLogger.info(`Executing job async using encrypted headers`);
let decryptedHeaders: Record<string, unknown>;
Expand Down Expand Up @@ -103,6 +102,7 @@ export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<
let size = 0;
try {
const generateResults: CsvResultFromSearch = await generateCsv(
context,
requestObject,
visType as string,
panel,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@
*/

import { badRequest } from 'boom';
import { ReportingCore } from '../../../../server';
import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { LevelLogger } from '../../../../server/lib';
import { RequestFacade } from '../../../../server/types';
import { ReportingCore } from '../../../../server';
import { FakeRequest, JobParamsPanelCsv, SearchPanel, VisPanel } from '../../types';
import { generateCsvSearch } from './generate_csv_search';

export function createGenerateCsv(reporting: ReportingCore, logger: LevelLogger) {
return async function generateCsv(
request: RequestFacade | FakeRequest,
context: RequestHandlerContext,
request: KibanaRequest | FakeRequest,
visType: string,
panel: VisPanel | SearchPanel,
jobParams: JobParamsPanelCsv
Expand All @@ -26,11 +27,12 @@ export function createGenerateCsv(reporting: ReportingCore, logger: LevelLogger)
switch (visType) {
case 'search':
return await generateCsvSearch(
request as RequestFacade,
reporting,
logger,
context,
request as KibanaRequest,
panel as SearchPanel,
jobParams
jobParams,
logger
);
default:
throw badRequest(`Unsupported or unrecognized saved object type: ${visType}`);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { IUiSettingsClient, KibanaRequest } from '../../../../../../../../src/core/server';
import {
IUiSettingsClient,
KibanaRequest,
RequestHandlerContext,
} from '../../../../../../../../src/core/server';
import {
esQuery,
EsQueryConfig,
Expand All @@ -13,23 +17,16 @@ import {
Query,
} from '../../../../../../../../src/plugins/data/server';
import { CancellationToken } from '../../../../../../../plugins/reporting/common';
import { ReportingCore } from '../../../../server';
import { LevelLogger } from '../../../../server/lib';
import { RequestFacade } from '../../../../server/types';
import { ReportingCore } from '../../../../server';
import { createGenerateCsv } from '../../../csv/server/lib/generate_csv';
import {
CsvResultFromSearch,
GenerateCsvParams,
JobParamsDiscoverCsv,
SearchRequest,
} from '../../../csv/types';
import {
IndexPatternField,
QueryFilter,
SavedSearchObjectAttributes,
SearchPanel,
SearchSource,
} from '../../types';
import { IndexPatternField, QueryFilter, SearchPanel, SearchSource } from '../../types';
import { getDataSource } from './get_data_source';
import { getFilters } from './get_filters';

Expand All @@ -54,17 +51,16 @@ const getUiSettings = async (config: IUiSettingsClient) => {
};

export async function generateCsvSearch(
req: RequestFacade,
reporting: ReportingCore,
logger: LevelLogger,
context: RequestHandlerContext,
req: KibanaRequest,
searchPanel: SearchPanel,
jobParams: JobParamsDiscoverCsv
jobParams: JobParamsDiscoverCsv,
logger: LevelLogger
): Promise<CsvResultFromSearch> {
const savedObjectsClient = await reporting.getSavedObjectsClient(
KibanaRequest.from(req.getRawRequest())
);
const savedObjectsClient = context.core.savedObjects.client;
const { indexPatternSavedObjectId, timerange } = searchPanel;
const savedSearchObjectAttr = searchPanel.attributes as SavedSearchObjectAttributes;
const savedSearchObjectAttr = searchPanel.attributes;
const { indexPatternSavedObject } = await getDataSource(
savedObjectsClient,
indexPatternSavedObjectId
Expand Down Expand Up @@ -153,9 +149,7 @@ export async function generateCsvSearch(

const config = reporting.getConfig();
const elasticsearch = await reporting.getElasticsearchService();
const { callAsCurrentUser } = elasticsearch.dataClient.asScoped(
KibanaRequest.from(req.getRawRequest())
);
const { callAsCurrentUser } = elasticsearch.dataClient.asScoped(req);
const callCluster = (...params: [string, object]) => callAsCurrentUser(...params);
const uiSettings = await getUiSettings(uiConfig);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ export function getFilters(
let timezone: string | null;

if (indexPatternTimeField) {
if (!timerange) {
if (!timerange || !timerange.min || !timerange.max) {
throw badRequest(
`Time range params are required for index pattern [${indexPatternId}], using time field [${indexPatternTimeField}]`
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,19 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { RequestFacade } from '../../../../server/types';
import { KibanaRequest } from 'src/core/server';
import { JobParamsPanelCsv, JobParamsPostPayloadPanelCsv } from '../../types';

export function getJobParamsFromRequest(
request: RequestFacade,
request: KibanaRequest,
opts: { isImmediate: boolean }
): JobParamsPanelCsv {
const { savedObjectType, savedObjectId } = request.params;
const { timerange, state } = request.payload as JobParamsPostPayloadPanelCsv;
const { savedObjectType, savedObjectId } = request.params as {
savedObjectType: string;
savedObjectId: string;
};
const { timerange, state } = request.body as JobParamsPostPayloadPanelCsv;

const post = timerange || state ? { timerange, state } : undefined;

return {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,28 +5,23 @@
*/

import { validateUrls } from '../../../../common/validate_urls';
import { ReportingCore } from '../../../../server';
import { cryptoFactory } from '../../../../server/lib';
import {
ConditionalHeaders,
CreateJobFactory,
ESQueueCreateJobFn,
RequestFacade,
} from '../../../../server/types';
import { CreateJobFactory, ESQueueCreateJobFn } from '../../../../server/types';
import { JobParamsPNG } from '../../types';

export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
JobParamsPNG
>> = function createJobFactoryFn(reporting: ReportingCore) {
>> = function createJobFactoryFn(reporting) {
const config = reporting.getConfig();
const setupDeps = reporting.getPluginSetupDeps();
const crypto = cryptoFactory(config.get('encryptionKey'));

return async function createJob(
{ objectType, title, relativeUrl, browserTimezone, layout }: JobParamsPNG,
headers: ConditionalHeaders['headers'],
request: RequestFacade
{ objectType, title, relativeUrl, browserTimezone, layout },
context,
req
) {
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const serializedEncryptedHeaders = await crypto.encrypt(req.headers);

validateUrls([relativeUrl]);

Expand All @@ -37,7 +32,7 @@ export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
headers: serializedEncryptedHeaders,
browserTimezone,
layout,
basePath: request.getBasePath(),
basePath: setupDeps.basePath(req),
forceNow: new Date().toISOString(),
};
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,28 @@
*/

import { validateUrls } from '../../../../common/validate_urls';
import { ReportingCore } from '../../../../server';
import { cryptoFactory } from '../../../../server/lib';
import {
ConditionalHeaders,
CreateJobFactory,
ESQueueCreateJobFn,
RequestFacade,
} from '../../../../server/types';
import { CreateJobFactory, ESQueueCreateJobFn } from '../../../../server/types';
import { JobParamsPDF } from '../../types';

export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
JobParamsPDF
>> = function createJobFactoryFn(reporting: ReportingCore) {
>> = function createJobFactoryFn(reporting) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for cleaning up this inferrable types!

const config = reporting.getConfig();
const setupDeps = reporting.getPluginSetupDeps();
const crypto = cryptoFactory(config.get('encryptionKey'));

return async function createJobFn(
{ title, relativeUrls, browserTimezone, layout, objectType }: JobParamsPDF,
headers: ConditionalHeaders['headers'],
request: RequestFacade
context,
req
) {
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const serializedEncryptedHeaders = await crypto.encrypt(req.headers);

validateUrls(relativeUrls);

return {
basePath: request.getBasePath(),
basePath: setupDeps.basePath(req),
browserTimezone,
forceNow: new Date().toISOString(),
headers: serializedEncryptedHeaders,
Expand Down
Loading