Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Endpoint] Ensure that DS indices for response actions are created prior to sending action to Endpoint #196953

Merged
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
771f729
Added `ensureDatastreamIndicesExist()` to base response actions clien…
paul-tavares Oct 18, 2024
525481c
If creating the DS fails, don't stop the response action - only log e…
paul-tavares Oct 21, 2024
97d513b
Add new callback to the fleet extension point for post package policy…
paul-tavares Oct 21, 2024
46b4b52
Added method to Fleet services: `getPolicyNamespace()`
paul-tavares Oct 22, 2024
c8058b6
Added method `getPolicyNamespace()` to endpoint app context services …
paul-tavares Oct 22, 2024
b1c6e80
Added processing logic for creating indexes for Package Policies
paul-tavares Oct 22, 2024
0920384
spelling corrections
paul-tavares Oct 22, 2024
b4e0f7a
Merge remote-tracking branch 'upstream/main' into task/olm-10840-fix-…
paul-tavares Oct 22, 2024
c81f1d6
Fleet: add new server-side extension for package policy post update
paul-tavares Oct 22, 2024
da6383d
Add integration post-update callback to process changes to namespace
paul-tavares Oct 22, 2024
d85c1c0
Some refactoring to simplify implementation
paul-tavares Oct 22, 2024
0544c3e
Fleet: added `agentPolicyPostUpdate` server-side extension
paul-tavares Oct 23, 2024
a2f207f
Add `agentPolicyPostUpdate` extension callback to create indexes
paul-tavares Oct 23, 2024
809f20c
Fix fleet type errors
paul-tavares Oct 23, 2024
5e96696
Fix type error in endpoint app context services
paul-tavares Oct 23, 2024
277c989
Remove index creation logic from response actions base class
paul-tavares Oct 23, 2024
94e466c
Fleet: Tests for agent policy service calling 'agentPolicyPostUpdate'…
paul-tavares Oct 23, 2024
2616996
Fleet: Tests for package policy service calling 'packagePolicyPostUpd…
paul-tavares Oct 23, 2024
96a475a
Fleet: Update some mocks
paul-tavares Oct 24, 2024
d9929be
Unit tests for Endpoint Fleet Factory
paul-tavares Oct 24, 2024
0cee3b3
Unit tests for fleet integrations callback builder
paul-tavares Oct 24, 2024
57b5e81
Merge remote-tracking branch 'refs/remotes/upstream/main' into task/o…
paul-tavares Oct 24, 2024
0f5b56a
Add `deleteAll()` to SimpleMemCache + move it to top-level endpoint/lib
paul-tavares Oct 24, 2024
746e127
Unit tests for `createPolicyDatastreams()`
paul-tavares Oct 24, 2024
88c9827
Fix type errors
paul-tavares Oct 24, 2024
06c74ca
Fix type errors again
paul-tavares Oct 24, 2024
89b33fc
Adjustment due to code review feedback
paul-tavares Oct 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions x-pack/plugins/fleet/server/mocks/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ export const createPackagePolicyServiceMock = (): jest.Mocked<PackagePolicyClien
inspect: jest.fn(),
delete: jest.fn(),
get: jest.fn(),
getByIDs: jest.fn(),
getByIDs: jest.fn().mockResolvedValue(Promise.resolve([])),
list: jest.fn(),
listIds: jest.fn(),
update: jest.fn(),
Expand Down Expand Up @@ -224,13 +224,13 @@ export const createPackagePolicyServiceMock = (): jest.Mocked<PackagePolicyClien
*/
export const createMockAgentPolicyService = (): jest.Mocked<AgentPolicyServiceInterface> => {
return {
get: jest.fn(),
list: jest.fn(),
getFullAgentPolicy: jest.fn(),
getByIds: jest.fn(),
turnOffAgentTamperProtections: jest.fn(),
fetchAllAgentPolicies: jest.fn(),
fetchAllAgentPolicyIds: jest.fn(),
get: jest.fn().mockReturnValue(Promise.resolve()),
list: jest.fn().mockReturnValue(Promise.resolve()),
getFullAgentPolicy: jest.fn().mockReturnValue(Promise.resolve()),
getByIds: jest.fn().mockReturnValue(Promise.resolve()),
turnOffAgentTamperProtections: jest.fn().mockReturnValue(Promise.resolve()),
fetchAllAgentPolicies: jest.fn().mockReturnValue(Promise.resolve()),
fetchAllAgentPolicyIds: jest.fn().mockReturnValue(Promise.resolve()),
};
};

Expand Down
32 changes: 32 additions & 0 deletions x-pack/plugins/fleet/server/services/agent_policy.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1285,6 +1285,38 @@ describe('Agent policy', () => {
})
).resolves.not.toThrow();
});

it('should run external "agentPolicyPostUpdate" callbacks when update is successful', async () => {
const soClient = getAgentPolicyCreateMock();
const esClient = elasticsearchServiceMock.createClusterClient().asInternalUser;

const postUpdateCallback = jest.fn(async (policy) => policy);
mockedAppContextService.getExternalCallbacks.mockImplementation((type) => {
if (type === 'agentPolicyPostUpdate') {
return new Set([postUpdateCallback]);
}
});

soClient.get.mockResolvedValue({
attributes: {},
id: 'test-id',
type: 'mocked',
references: [],
});

await expect(
agentPolicyService.update(soClient, esClient, 'test-id', {
name: 'test',
namespace: 'default',
})
).resolves.not.toThrow();

expect(mockedAppContextService.getExternalCallbacks).toHaveBeenCalledWith(
'agentPolicyPostUpdate'
);

expect(postUpdateCallback).toHaveBeenCalled();
});
});

describe('deployPolicy', () => {
Expand Down
16 changes: 14 additions & 2 deletions x-pack/plugins/fleet/server/services/agent_policy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ import type {
PostAgentPolicyUpdateCallback,
PreconfiguredAgentPolicy,
OutputsForAgentPolicy,
PostAgentPolicyPostUpdateCallback,
} from '../types';
import {
AGENT_POLICY_INDEX,
Expand Down Expand Up @@ -309,8 +310,8 @@ class AgentPolicyService {

public async runExternalCallbacks(
externalCallbackType: ExternalCallback[0],
agentPolicy: NewAgentPolicy | Partial<AgentPolicy>
): Promise<NewAgentPolicy | Partial<AgentPolicy>> {
agentPolicy: NewAgentPolicy | Partial<AgentPolicy> | AgentPolicy
): Promise<NewAgentPolicy | Partial<AgentPolicy> | AgentPolicy> {
const logger = appContextService.getLogger();
logger.debug(`Running external callbacks for ${externalCallbackType}`);
try {
Expand All @@ -333,6 +334,12 @@ class AgentPolicyService {
);
updatedNewAgentPolicy = result;
}
if (externalCallbackType === 'agentPolicyPostUpdate') {
result = await (callback as PostAgentPolicyPostUpdateCallback)(
newAgentPolicy as AgentPolicy
);
updatedNewAgentPolicy = result;
}
}
newAgentPolicy = updatedNewAgentPolicy;
}
Expand Down Expand Up @@ -741,6 +748,11 @@ class AgentPolicyService {
bumpRevision: true,
removeProtection: false,
skipValidation: options?.skipValidation ?? false,
}).then((updatedAgentPolicy) => {
return this.runExternalCallbacks(
'agentPolicyPostUpdate',
updatedAgentPolicy
) as unknown as AgentPolicy;
});
}

Expand Down
35 changes: 35 additions & 0 deletions x-pack/plugins/fleet/server/services/package_policy.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1718,6 +1718,41 @@ describe('Package policy service', () => {
});
});

it('should run "packagePolicyPostUpdate" external callbacks', async () => {
const soClient = savedObjectsClientMock.create();
const esClient = elasticsearchServiceMock.createClusterClient().asInternalUser;
const mockPackagePolicy = createPackagePolicyMock();
const attributes = {
...mockPackagePolicy,
inputs: [],
};

jest.spyOn(appContextService, 'getExternalCallbacks');

soClient.get.mockResolvedValue({
id: 'test-package-policy',
type: LEGACY_PACKAGE_POLICY_SAVED_OBJECT_TYPE,
references: [],
attributes,
});

soClient.update.mockResolvedValue({
id: 'test-package-policy',
type: LEGACY_PACKAGE_POLICY_SAVED_OBJECT_TYPE,
references: [],
attributes,
});

await packagePolicyService.update(soClient, esClient, 'test-package-policy', {
...mockPackagePolicy,
inputs: [],
});

expect(appContextService.getExternalCallbacks).toHaveBeenCalledWith(
'packagePolicyPostUpdate'
);
});

describe('remove protections', () => {
beforeEach(() => {
mockAgentPolicyService.bumpRevision.mockReset();
Expand Down
90 changes: 45 additions & 45 deletions x-pack/plugins/fleet/server/services/package_policy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ import type {
DryRunPackagePolicy,
PostPackagePolicyCreateCallback,
PostPackagePolicyPostCreateCallback,
PutPackagePolicyPostUpdateCallback,
} from '../types';
import type { ExternalCallback } from '..';

Expand Down Expand Up @@ -127,6 +128,8 @@ import type {
PackagePolicyClient,
PackagePolicyClientFetchAllItemsOptions,
PackagePolicyService,
RunExternalCallbacksPackagePolicyArgument,
RunExternalCallbacksPackagePolicyResponse,
} from './package_policy_service';
import { installAssetsForInputPackagePolicy } from './epm/packages/install';
import { auditLoggingService } from './audit_logging';
Expand Down Expand Up @@ -1091,9 +1094,16 @@ class PackagePolicyClientImpl implements PackagePolicyClient {
await Promise.all([...bumpPromises, assetRemovePromise, deleteSecretsPromise]);

sendUpdatePackagePolicyTelemetryEvent(soClient, [packagePolicyUpdate], [oldPackagePolicy]);

logger.debug(`Package policy ${id} update completed`);

return newPolicy;
// Run external post-update callbacks and return
return packagePolicyService.runExternalCallbacks(
'packagePolicyPostUpdate',
newPolicy,
soClient,
esClient
);
}

public async bulkUpdate(
Expand Down Expand Up @@ -1930,56 +1940,29 @@ class PackagePolicyClientImpl implements PackagePolicyClient {

public async runExternalCallbacks<A extends ExternalCallback[0]>(
externalCallbackType: A,
packagePolicy: A extends 'packagePolicyDelete'
? DeletePackagePoliciesResponse
: A extends 'packagePolicyPostDelete'
? PostDeletePackagePoliciesResponse
: A extends 'packagePolicyPostCreate'
? PackagePolicy
: A extends 'packagePolicyCreate'
? NewPackagePolicy
: never,
packagePolicy: RunExternalCallbacksPackagePolicyArgument<A>,
soClient: SavedObjectsClientContract,
esClient: ElasticsearchClient,
context?: RequestHandlerContext,
request?: KibanaRequest
): Promise<
A extends 'packagePolicyDelete'
? void
: A extends 'packagePolicyPostDelete'
? void
: A extends 'packagePolicyPostCreate'
? PackagePolicy
: A extends 'packagePolicyCreate'
? NewPackagePolicy
: never
>;
public async runExternalCallbacks(
externalCallbackType: ExternalCallback[0],
packagePolicy:
| PackagePolicy
| NewPackagePolicy
| PostDeletePackagePoliciesResponse
| DeletePackagePoliciesResponse,
soClient: SavedObjectsClientContract,
esClient: ElasticsearchClient,
context?: RequestHandlerContext,
request?: KibanaRequest
): Promise<PackagePolicy | NewPackagePolicy | void> {
): Promise<RunExternalCallbacksPackagePolicyResponse<A>> {
const logger = appContextService.getLogger();
const numberOfCallbacks = appContextService.getExternalCallbacks(externalCallbackType)?.size;
let runResult: any;

logger.debug(`Running ${numberOfCallbacks} external callbacks for ${externalCallbackType}`);

try {
if (externalCallbackType === 'packagePolicyPostDelete') {
return await this.runPostDeleteExternalCallbacks(
runResult = await this.runPostDeleteExternalCallbacks(
packagePolicy as PostDeletePackagePoliciesResponse,
soClient,
esClient,
context,
request
);
} else if (externalCallbackType === 'packagePolicyDelete') {
return await this.runDeleteExternalCallbacks(
runResult = await this.runDeleteExternalCallbacks(
packagePolicy as DeletePackagePoliciesResponse,
soClient,
esClient
Expand All @@ -1988,21 +1971,33 @@ class PackagePolicyClientImpl implements PackagePolicyClient {
if (!Array.isArray(packagePolicy)) {
let newData = packagePolicy;
const externalCallbacks = appContextService.getExternalCallbacks(externalCallbackType);

if (externalCallbacks && externalCallbacks.size > 0) {
let updatedNewData = newData;
let updatedNewData: any = newData;

for (const callback of externalCallbacks) {
let result;
let thisCallbackResponse;

if (externalCallbackType === 'packagePolicyPostCreate') {
result = await (callback as PostPackagePolicyPostCreateCallback)(
thisCallbackResponse = await (callback as PostPackagePolicyPostCreateCallback)(
updatedNewData as PackagePolicy,
soClient,
esClient,
context,
request
);
updatedNewData = PackagePolicySchema.validate(result) as NewPackagePolicy;
updatedNewData = PackagePolicySchema.validate(thisCallbackResponse);
} else if (externalCallbackType === 'packagePolicyPostUpdate') {
thisCallbackResponse = await (callback as PutPackagePolicyPostUpdateCallback)(
updatedNewData as PackagePolicy,
soClient,
esClient,
context,
request
);
updatedNewData = PackagePolicySchema.validate(thisCallbackResponse);
} else {
result = await (callback as PostPackagePolicyCreateCallback)(
thisCallbackResponse = await (callback as PostPackagePolicyCreateCallback)(
updatedNewData as NewPackagePolicy,
soClient,
esClient,
Expand All @@ -2012,10 +2007,10 @@ class PackagePolicyClientImpl implements PackagePolicyClient {
}

if (externalCallbackType === 'packagePolicyCreate') {
updatedNewData = NewPackagePolicySchema.validate(result) as NewPackagePolicy;
updatedNewData = NewPackagePolicySchema.validate(thisCallbackResponse);
} else if (externalCallbackType === 'packagePolicyUpdate') {
const omitted = {
...omit(result, [
...omit(thisCallbackResponse, [
'id',
'spaceIds',
'version',
Expand All @@ -2026,23 +2021,28 @@ class PackagePolicyClientImpl implements PackagePolicyClient {
'created_by',
'elasticsearch',
]),
inputs: result.inputs.map((input) => omit(input, ['compiled_input'])),
inputs: thisCallbackResponse.inputs.map((input) =>
omit(input, ['compiled_input'])
),
};

updatedNewData = UpdatePackagePolicySchema.validate(omitted) as PackagePolicy;
updatedNewData = UpdatePackagePolicySchema.validate(omitted);
}
}

newData = updatedNewData;
}
return newData;

runResult = newData;
}
}
} catch (error) {
logger.error(`Error running external callbacks for ${externalCallbackType}:`);
logger.error(error);
throw error;
}

return runResult as unknown as RunExternalCallbacksPackagePolicyResponse<A>;
}

public async runPostDeleteExternalCallbacks(
Expand Down
Loading