Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.15](backport #4333) [apm] Update APM docs to reflect changes related to the Elasticsearch apm-data plugin #4425

Merged
merged 3 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The steps to follow are:

* copy the `session.id` from the relevant document.
* Open the Discover page.
* Select the appropriate data view (use `APM` to search all datastreams)
* Select the appropriate data view (use `APM` to search all data streams)
* set filter to the copied `session.id`

Here we can see the `session.id` guid in the metadata viewer in the error detail view:
Expand Down
26 changes: 1 addition & 25 deletions docs/en/observability/apm-ui/troubleshooting.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,32 +25,11 @@ don't forget to check our other troubleshooting guides or discussion forum:
[[troubleshooting-apm-app]]
== Troubleshoot common APM UI problems

* <<no-apm-data-found>>
* <<troubleshooting-too-many-transactions>>
* <<troubleshooting-unknown-route>>
* <<troubleshooting-fields-unsearchable>>
* <<service-map-rum-connections>>

[float]
[[no-apm-data-found]]
=== No APM data found

This section can help with any of the following:

* Data isn't displaying in the APM UI
* Data isn't displaying in the APM UI after an upgrade
* You see a message like "No Services Found",
* You see errors like "Fielddata is disabled on text fields by default..."

These problems are likely to be caused by a missing index template or ingest pipeline.
By default, {fleet} sets up these and other APM assets when the APM integration is installed.
Try reinstalling the APM integration by navigating to
**Integrations** > **Elastic APM** > **Manage in Fleet** > **Settings** > **Reinstall Elastic APM**.

Because assets cannot be applied to indices retroactively,
after reinstalling the APM integration you must either wait for the index to rollover or force a rollover.
To force a rollover, use the {ref}/indices-rollover-index.html[rollover API] to target the relevant {apm-guide-ref}/apm-data-streams.html[APM data streams].

[float]
[[troubleshooting-too-many-transactions]]
=== Too many unique transaction names
Expand Down Expand Up @@ -142,7 +121,7 @@ You can also use the agent's public API to manually set a name for the transacti
=== Fields are not searchable

In Elasticsearch, index templates are used to define settings and mappings that determine how fields should be analyzed.
The recommended index templates for APM are installed by {fleet} when the Elastic APM integration is installed.
The recommended index templates for APM come from the built-in {es} apm-data plugin.
These templates, by default, enable and disable indexing on certain fields.

As an example, some APM agents store cookie values in `http.request.cookies`.
Expand All @@ -158,9 +137,6 @@ If you don't, the data view doesn't exist.
To fix this, navigate to the APM UI in {kib} and select *Add data*.
In the APM tutorial, click *Load Kibana objects* to create the APM data view.

If creating an APM data view doesn't solve the problem,
see <<no-apm-data-found>> for further troubleshooting.

*Ensure a field is searchable*
There are two things you can do to if you'd like to ensure a field is searchable:

Expand Down
51 changes: 0 additions & 51 deletions docs/en/observability/apm/common-problems.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,57 +18,6 @@ If no data shows up in {es}, first make sure that your APM components are proper

include::{observability-docs-root}/docs/en/observability/apm/tab-widgets/no-data-indexed-widget.asciidoc[]

[[apm-data-indexed-no-apm]]
[float]
=== Data is indexed but doesn't appear in the APM UI

The APM UI relies on index mappings to query and display data.
If your APM data isn't showing up in the APM UI, but is elsewhere in {kib}, like the Discover app,
you may have a missing index mapping.

You can determine if a field was mapped correctly with the `_mapping` API.
For example, run the following command in the {kib} {kibana-ref}/console-kibana.html[console].
This will display the field data type of the `service.name` field.

[source,curl]
----
GET *apm*/_mapping/field/service.name
----

If the `mapping.name.type` is `"text"`, your APM indices were not set up correctly.

[source,yml]
----
".ds-metrics-apm.transaction.1m-default-2023.04.12-000038": {
"mappings": {
"service.name": {
"full_name": "service.name",
"mapping": {
"name": {
"type": "text" <1>
}
}
}
}
}
----
<1> The `service.name` `mapping.name.type` would be `"keyword"` if this field had been set up correctly.

To fix this problem, install the APM integration by following these steps:

--
include::{observability-docs-root}/docs/en/observability/apm/getting-started-apm-server.asciidoc[tag=install-apm-integration]
--

This will reinstall the APM index templates and trigger a data stream index rollover.

You can verify the correct index templates were installed by running the following command in the {kib} console:

[source,curl]
----
GET /_index_template/traces-apm
----

[float]
[[apm-common-ssl-problems]]
=== Common SSL-related problems
Expand Down
2 changes: 1 addition & 1 deletion docs/en/observability/apm/configure/general.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ Defaults to `debug/vars`.
== Data stream namespace

Change the default namespace.
This setting changes the name of the integration's data stream.
This setting changes the name of the data stream.

For {fleet}-managed users, the namespace is inherited from the selected {agent} policy.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,6 @@ output.console:
pretty: true
------------------------------------------------------------------------------

ifdef::apm-server[]
[float]
=== {kib} configuration

include::../../shared-kibana-endpoint.asciidoc[tag=shared-kibana-config]
endif::[]

[float]
=== Configuration options

Expand Down
5 changes: 0 additions & 5 deletions docs/en/observability/apm/configure/outputs/kafka.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,6 @@ output.kafka:

NOTE: Events bigger than <<apm-kafka-max_message_bytes,`max_message_bytes`>> will be dropped. To avoid this problem, make sure APM Server does not generate events bigger than <<apm-kafka-max_message_bytes,`max_message_bytes`>>.

[float]
=== {kib} configuration

include::../../shared-kibana-endpoint.asciidoc[tag=shared-kibana-config]

[float]
[[apm-kafka-compatibility]]
=== Compatibility
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ protocol, which runs over TCP.
To send events to {ls}, you must:

. <<apm-ls-output-config,Enable the {ls} output in APM Server>>
. <<apm-ls-kib-config,Enable the {kib} endpoint in APM Server>>
. <<apm-ls-config-pipeline,Create a {ls} configuration pipeline in {ls}>>

[float]
Expand All @@ -43,12 +42,6 @@ output.logstash:
<1> The `hosts` option specifies the {ls} server and the port (`5044`) where {ls} is configured to listen for incoming
APM Server connections.

[float]
[[apm-ls-kib-config]]
=== {kib} endpoint configuration

include::../../shared-kibana-endpoint.asciidoc[tag=shared-kibana-config]

[float]
[[apm-ls-config-pipeline]]
=== {ls} configuration pipeline
Expand Down
5 changes: 0 additions & 5 deletions docs/en/observability/apm/configure/outputs/redis.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,6 @@ output.redis:
timeout: 5
------------------------------------------------------------------------------

[float]
=== {kib} configuration

include::../../shared-kibana-endpoint.asciidoc[tag=shared-kibana-config]

[float]
=== Compatibility

Expand Down
2 changes: 1 addition & 1 deletion docs/en/observability/apm/data-streams.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ See the {fleet-guide}/data-streams.html[{fleet} and {agent} Guide] to learn more

// tag::data-streams[]
APM data follows the `<type>-<dataset>-<namespace>` naming scheme.
The `type` and `dataset` are predefined by the APM integration,
The `type` and `dataset` are predefined by the {es} apm-data plugin,
but the `namespace` is your opportunity to customize how different types of data are stored in {es}.
There is no recommendation for what to use as your namespace--it is intentionally flexible.
For example, you might create namespaces for each of your environments,
Expand Down
71 changes: 3 additions & 68 deletions docs/en/observability/apm/getting-started-apm-server.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,8 @@ TIP: The easiest way to get started with Elastic APM is by using our
The {es} Service is available on AWS, GCP, and Azure.
See <<traces-get-started,getting started documentation>> to get started in minutes.


// TODO: MOVE THIS
IMPORTANT: Starting in version 8.0.0, {fleet} uses the APM integration to set up and manage APM index templates,
{ilm-init} policies, and ingest pipelines. APM Server will only send data to {es} _after_ the APM integration has been installed.
IMPORTANT: Starting in version 8.15.0, the {es} apm-data plugin manages APM index templates,
lifecycle policies, and ingest pipelines.

The APM Server receives performance data from your APM agents,
validates and processes it, and then transforms the data into {es} documents.
Expand Down Expand Up @@ -306,69 +304,6 @@ See <<apm-running-on-docker, Running on Docker>> for deploying Docker containers
[[apm-server-configuration]]
==== Step 2: Set up and configure

// This content is reused in the upgrading guide
// tag::why-apm-integration[]
Starting in version 8.0.0, {fleet} uses the APM integration to set up and manage APM index templates,
{ilm-init} policies, and ingest pipelines. APM Server will only send data to {es} _after_ the APM integration has been installed.
// end::why-apm-integration[]

[float]
===== Install the APM integration

// This content is reused in the upgrading guide
// tag::install-apm-integration[]
[%collapsible%open]
.**If you have an internet connection**
====
An internet connection is required to install the APM integration via the {fleet} UI in {kib}.

// lint ignore elastic-agent
. Open {kib} and select **Add integrations** > **Elastic APM**.
. Click **APM integration**.
. Click **Add Elastic APM**.
. Click **Save and continue**.
. Click **Add Elastic Agent later**. You do not need to run an {agent} to complete the setup.
====

// tag::install-apm-integration-no-internet[]
[%collapsible]
.**If you don't have an internet connection**
====
If your environment has network traffic restrictions, there are other ways to install the APM integration.
See {fleet-guide}/air-gapped.html[Air-gapped environments] for more information.

Option 1: Update `kibana.yml`::
+
Update `kibana.yml` to include the following, then restart {kib}.
+
[source,yaml]
----
xpack.fleet.packages:
- name: apm
version: latest
----
+
See {kibana-ref}/settings.html[Configure Kibana] to learn more about how to edit the Kibana configuration file.

Option 2: Use the {fleet} API::
+
Use the {fleet} API to install the APM integration. To be successful, this needs to be run against the {kib}
API, not the {es} API.
+
["source","yaml",subs="attributes"]
----
POST kbn:/api/fleet/epm/packages/apm/{apm_server_version}
{ "force": true }
----
+
See {kibana-ref}/api.html[Kibana API] to learn more about how to use the Kibana APIs.
====
// end::install-apm-integration-no-internet[]
// end::install-apm-integration[]

[float]
===== Configure APM

Configure APM by editing the `apm-server.yml` configuration file.
The location of this file varies by platform--see the <<apm-directory-layout>> for help locating it.

Expand Down Expand Up @@ -506,7 +441,7 @@ include::{observability-docs-root}/docs/en/observability/tab-widgets/add-apm-int
An internet connection is required to install the APM integration via the Fleet UI in Kibana.

--
include::{observability-docs-root}/docs/en/observability/apm/getting-started-apm-server.asciidoc[tag=install-apm-integration-no-internet]
include::{observability-docs-root}/docs/en/observability/apm/shared/install-apm-integration-no-internet.asciidoc[]
--
****

Expand Down
Loading