Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] Support for remote ES output #169252

Merged
merged 25 commits into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
c98403e
remote es output
juliaElastic Oct 18, 2023
e49de06
[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…
kibanamachine Oct 18, 2023
22db8e2
generate service token from UI
juliaElastic Oct 19, 2023
97cf572
moved service token generation to settings
juliaElastic Oct 19, 2023
f153d18
removed unused translations
juliaElastic Oct 19, 2023
7cb9c22
hide remote es output from agent policy data output dropdown
juliaElastic Oct 23, 2023
1fa0426
include output_permissions and service_token for remote output
juliaElastic Oct 24, 2023
f0e8070
removed generate token from settings and added callout to flyout
juliaElastic Oct 25, 2023
9ef33e4
fix unit test
juliaElastic Oct 25, 2023
fd02cfd
fix test
juliaElastic Oct 25, 2023
f42f1c6
Merge remote-tracking branch 'upstream/main' into remote-es-output
juliaElastic Oct 25, 2023
11b8cc1
[CI] Auto-commit changed files from 'node scripts/jest_integration -u…
kibanamachine Oct 25, 2023
b818b90
fix type after merge conflict
juliaElastic Oct 25, 2023
9e0621b
turn feature flag off for now
juliaElastic Oct 25, 2023
2700ab6
Merge branch 'main' into remote-es-output
juliaElastic Oct 25, 2023
200a227
added tests
juliaElastic Oct 25, 2023
dc46e2f
Merge branch 'main' into remote-es-output
juliaElastic Oct 25, 2023
bc2e10b
updated callout
juliaElastic Oct 25, 2023
0315d01
added mappings_addition to SO config
juliaElastic Oct 26, 2023
43c864c
[CI] Auto-commit changed files from 'node scripts/jest_integration -u…
kibanamachine Oct 26, 2023
d9257fb
Merge branch 'main' into remote-es-output
juliaElastic Oct 26, 2023
d9eb33f
review comments
juliaElastic Oct 30, 2023
54c3900
Merge branch 'main' into remote-es-output
juliaElastic Oct 30, 2023
813846d
fix tests
juliaElastic Oct 30, 2023
fcaa3e8
fixing schema validation
juliaElastic Oct 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1680,6 +1680,10 @@
"type": "keyword",
"index": false
},
"service_token": {
"type": "keyword",
"index": false
},
"config": {
"type": "flattened"
},
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/fleet/common/constants/output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ export const outputType = {
Elasticsearch: 'elasticsearch',
Logstash: 'logstash',
Kafka: 'kafka',
RemoteElasticsearch: 'remote-elasticsearch',
juliaElastic marked this conversation as resolved.
Show resolved Hide resolved
} as const;

export const DEFAULT_OUTPUT_ID = 'fleet-default-output';
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/fleet/common/experimental_features.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ export const allowedExperimentalValues = Object.freeze<Record<string, boolean>>(
agentTamperProtectionEnabled: true,
secretsStorage: true,
kafkaOutput: true,
remoteESOutput: true,
});

type ExperimentalConfigKeys = Array<keyof ExperimentalFeatures>;
Expand Down
11 changes: 10 additions & 1 deletion x-pack/plugins/fleet/common/types/models/output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,20 @@ export interface NewElasticsearchOutput extends NewBaseOutput {
type: OutputType['Elasticsearch'];
}

export interface NewRemoteElasticsearchOutput extends NewBaseOutput {
type: OutputType['RemoteElasticsearch'];
service_token?: string;
}

export interface NewLogstashOutput extends NewBaseOutput {
type: OutputType['Logstash'];
}

export type NewOutput = NewElasticsearchOutput | NewLogstashOutput | KafkaOutput;
export type NewOutput =
| NewElasticsearchOutput
| NewRemoteElasticsearchOutput
| NewLogstashOutput
| KafkaOutput;

export type Output = NewOutput & {
id: string;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ export * from './use_quick_start_form';
// but exported here to support individual usage.
export * from './use_wait_for_fleet_server';
export * from './use_select_fleet_server_policy';
export * from './use_service_token';
export * from './use_fleet_server_host';
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@ import { useState } from 'react';

import type { DeploymentMode } from '../steps';

import { useServiceToken } from '../../../hooks/use_service_token';

import { useFleetServerHost } from './use_fleet_server_host';
import { useSelectFleetServerPolicy } from './use_select_fleet_server_policy';
import { useServiceToken } from './use_service_token';
import { useWaitForFleetServer } from './use_wait_for_fleet_server';

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@ import type { NewAgentPolicy } from '../../../types';

import type { FleetServerHost } from '../../../types';

import { useServiceToken } from '../../../hooks/use_service_token';

import { useSelectFleetServerPolicy } from './use_select_fleet_server_policy';
import { useServiceToken } from './use_service_token';
import { useFleetServerHost } from './use_fleet_server_host';

const QUICK_START_FLEET_SERVER_POLICY_FIELDS: NewAgentPolicy = {
Expand Down
juliaElastic marked this conversation as resolved.
Show resolved Hide resolved

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,38 +9,17 @@ import React from 'react';
import styled from 'styled-components';

import type { EuiStepProps } from '@elastic/eui';
import {
EuiButton,
EuiCallOut,
EuiCodeBlock,
EuiFlexGroup,
EuiFlexItem,
EuiSpacer,
EuiText,
} from '@elastic/eui';
import { EuiFlexGroup } from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n-react';

const FlexItemWithMinWidth = styled(EuiFlexItem)`
min-width: 0px;
max-width: 100%;
`;
import { GenerateServiceTokenComponent } from '../../generate_service_token';

export const ContentWrapper = styled(EuiFlexGroup)`
height: 100%;
margin: 0 auto;
max-width: 800px;
`;

// Otherwise the copy button is over the text
const CommandCode = styled.div.attrs(() => {
return {
className: 'eui-textBreakAll',
};
})`
margin-right: ${(props) => props.theme.eui.euiSizeM};
`;

export const getGenerateServiceTokenStep = ({
disabled = false,
serviceToken,
Expand All @@ -58,79 +37,11 @@ export const getGenerateServiceTokenStep = ({
}),
status: disabled ? 'disabled' : undefined,
children: !disabled && (
<ServiceTokenStepContent
<GenerateServiceTokenComponent
serviceToken={serviceToken}
generateServiceToken={generateServiceToken}
isLoadingServiceToken={isLoadingServiceToken}
/>
),
};
};

const ServiceTokenStepContent: React.FunctionComponent<{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved out and renamed this component to GenerateServiceTokenComponent to reuse in generating remote token

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the end I haven't used this component somewhere else, but I think the refactor can be left in.

serviceToken?: string;
generateServiceToken: () => void;
isLoadingServiceToken: boolean;
}> = ({ serviceToken, generateServiceToken, isLoadingServiceToken }) => {
return (
<>
<EuiText>
<FormattedMessage
id="xpack.fleet.fleetServerSetup.generateServiceTokenDescription"
defaultMessage="A service token grants Fleet Server permissions to write to Elasticsearch."
/>
</EuiText>
<EuiSpacer size="m" />
{!serviceToken ? (
<EuiFlexGroup>
<EuiFlexItem grow={false}>
<EuiButton
fill
isLoading={isLoadingServiceToken}
isDisabled={isLoadingServiceToken}
onClick={() => {
generateServiceToken();
}}
data-test-subj="fleetServerGenerateServiceTokenBtn"
>
<FormattedMessage
id="xpack.fleet.fleetServerSetup.generateServiceTokenButton"
defaultMessage="Generate service token"
/>
</EuiButton>
</EuiFlexItem>
</EuiFlexGroup>
) : (
<>
<EuiCallOut
iconType="check"
size="s"
color="success"
title={
<FormattedMessage
id="xpack.fleet.fleetServerSetup.saveServiceTokenDescription"
defaultMessage="Save your service token information. This will be shown only once."
/>
}
/>
<EuiSpacer size="m" />
<EuiFlexGroup direction="column" gutterSize="s">
<EuiFlexItem grow={false}>
<strong data-test-subject="serviceTokenSaveReminderHeader">
<FormattedMessage
id="xpack.fleet.fleetServerSetup.serviceTokenLabel"
defaultMessage="Service token"
/>
</strong>
</EuiFlexItem>
<FlexItemWithMinWidth>
<EuiCodeBlock paddingSize="m" isCopyable>
<CommandCode>{serviceToken}</CommandCode>
</EuiCodeBlock>
</FlexItemWithMinWidth>
</EuiFlexGroup>
</>
)}
</>
);
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import React from 'react';
import styled from 'styled-components';

import { EuiLink, EuiToolTip } from '@elastic/eui';
import {
EuiButton,
EuiCallOut,
EuiCodeBlock,
EuiFlexGroup,
EuiFlexItem,
EuiSpacer,
EuiText,
} from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n-react';

import { useStartServices } from '../../../hooks';
import { useCheckPermissions } from '../hooks';

const FlexItemWithMinWidth = styled(EuiFlexItem)`
min-width: 0px;
max-width: 100%;
`;

// Otherwise the copy button is over the text
const CommandCode = styled.div.attrs(() => {
return {
className: 'eui-textBreakAll',
};
})`
margin-right: ${(props) => props.theme.eui.euiSizeM};
`;

export const GenerateServiceTokenComponent: React.FunctionComponent<{
serviceToken?: string;
generateServiceToken: (remote?: boolean) => void;
isLoadingServiceToken: boolean;
isRemote?: boolean;
}> = ({ serviceToken, generateServiceToken, isLoadingServiceToken, isRemote = false }) => {
const { docLinks } = useStartServices();
const { permissionsError, isPermissionsLoading } = useCheckPermissions();

return (
<>
<EuiText>
{isRemote ? (
<FormattedMessage
id="xpack.fleet.settings.remoteServiceToken.generateServiceTokenDescription"
defaultMessage="A service token grants Fleet Server permissions to write to Elasticsearch, and can be used to configure this Elasticsearch cluster for remote output. For more information, see the {fleetUserGuide}."
values={{
fleetUserGuide: (
<EuiLink href={docLinks.links.fleet.guide} target="_blank">
{i18n.translate('xpack.fleet.settings.editOutputFlyout.fleetUserGuideLabel', {
defaultMessage: 'Fleet User Guide',
})}
</EuiLink>
),
}}
/>
) : (
<FormattedMessage
id="xpack.fleet.fleetServerSetup.generateServiceTokenDescription"
defaultMessage="A service token grants Fleet Server permissions to write to Elasticsearch."
/>
)}
</EuiText>
<EuiSpacer size="m" />
{!serviceToken ? (
<EuiFlexGroup>
<EuiFlexItem grow={false}>
<EuiToolTip
content={
isRemote && !!permissionsError ? (
<FormattedMessage
id="xpack.fleet.settings.remoteServiceToken.noPermissionTooltip"
defaultMessage="To generate service token, you must have the minimum required privileges. Contact your administrator."
/>
) : null
}
>
<EuiButton
fill
isLoading={isLoadingServiceToken}
isDisabled={isLoadingServiceToken || isPermissionsLoading || !!permissionsError}
onClick={() => {
generateServiceToken(isRemote);
}}
data-test-subj="fleetServerGenerateServiceTokenBtn"
>
<FormattedMessage
id="xpack.fleet.fleetServerSetup.generateServiceTokenButton"
defaultMessage="Generate service token"
/>
</EuiButton>
</EuiToolTip>
</EuiFlexItem>
</EuiFlexGroup>
) : (
<>
<EuiCallOut
iconType="check"
size="s"
color="success"
title={
<FormattedMessage
id="xpack.fleet.fleetServerSetup.saveServiceTokenDescription"
defaultMessage="Save your service token information. This will be shown only once."
/>
}
/>
<EuiSpacer size="m" />
<EuiFlexGroup direction="column" gutterSize="s">
<EuiFlexItem grow={false}>
<strong data-test-subject="serviceTokenSaveReminderHeader">
<FormattedMessage
id="xpack.fleet.fleetServerSetup.serviceTokenLabel"
defaultMessage="Service token"
/>
</strong>
</EuiFlexItem>
<FlexItemWithMinWidth>
<EuiCodeBlock paddingSize="m" isCopyable>
<CommandCode>{serviceToken}</CommandCode>
</EuiCodeBlock>
</FlexItemWithMinWidth>
</EuiFlexGroup>
</>
)}
</>
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ export * from '../../../components';

export * from './search_bar';
export * from './fleet_server_instructions';
export * from './generate_service_token';
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,5 @@ export * from '../../../hooks';

export * from './use_breadcrumbs';
export * from './use_apm_service_href';
export * from './use_service_token';
export * from './use_check_permissions';
Loading