How to diagnose and troubleshoot problems on Kaspersky Managed Detection and Response EPP agents
Show applications and versions that this article concerns
- Kaspersky Managed Detection and Response
- Kaspersky Endpoint Security 12 for Windows
- Kaspersky Endpoint Security 11 for Windows
- Kaspersky Endpoint Security 12 for Linux
- Kaspersky Endpoint Security 11 for Linux
- Kaspersky Endpoint Security 12 for Mac
- Kaspersky Endpoint Security 11 for Mac
- Kaspersky Endpoint Agent
- Kaspersky Security for Virtualization Light Agent
- Kaspersky Anti Targeted Attack Platform
This article will help you analyze trace files of ЕРР applications that send data to Kaspersky Managed Detection and Response (hereinafter MDR) to identify and resolve possible problems.
What files you need for diagnostics
- Trace files for Kaspersky Endpoint Security for Windows, the KES*SRV*.log file is most often required.
- Trace files for Kaspersky Endpoint Security for Linux, the kesl*.log file is most often required.
- The Kaspersky Managed Detection and Response configuration file.
How to diagnose and troubleshoot telemetry data loss
Diagnostics
Open trace files in any text editor and look for lines that contain "Errcount":
- If there is no data loss, you will see the value 0. Example:
15:34:21.177 231473 INF ksnclnt SetRouteStatus for service FR succeeded: address = dc1-file.ksn.kaspersky-labs.com Errcount: 0
21:24:09.344 2053788 INF ksnclnt SetRouteStatus for service FR succeeded: address = dc1-file.ksn.kaspersky-labs.com Errcount: 0 - If there is partial data loss, some messages will show 0 and others will show the number of errors. Example:
12:11:22.180 0x1af4 INF ksnclnt SetRouteStatus for service P2P succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 0
12:11:35.196 0x18cc INF ksnclnt SetRouteStatus for service S succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 4
12:11:43.482 0x1c88 INF ksnclnt SetRouteStatus for service FR succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 0
12:12:06.456 0x1c88 INF ksnclnt SetRouteStatus for service S succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 6 - If no data is received, you will see the number of errors in all messages. Example:
12:24:54.801 0x1540 INF ksnclnt SetRouteStatus for service FR succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 4
12:24:55.943 0x1af4 INF ksnclnt SetRouteStatus for service FR succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 6
12:24:56.853 0x1418 INF ksnclnt SetRouteStatus for service U succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 7
12:25:08.039 0xde4 INF ksnclnt SetRouteStatus for service I succeeded: address = hqkscrootsrv1.avp.ru; used ip = 10.73.19.11 Errcount: 4
Solution
- Make sure that the MDR service is enabled on the devices:
- Open the folder where the Kaspersky Endpoint Security for Windows trace files are located and run the command:
grep -i "allowed services" KES*SRV*.log | tail -1If the MDR service is enabled, you will see the following:
kesl.2946448.2023-08-24T173006.log:2023.08.24 14:30:10.097 2946539 INF ksnclnt Ping: Allowed Services = {V, U, P2P, FR, S, MDR} - Open the folder where the Kaspersky Endpoint Security for Linux trace files are located and run the command:
grep -ai 'allowed services' kesl* | tail -1If the MDR service is enabled, you will see the following:
KES.21.16.6.467.SRV_2024-05-13T150959.0001956_0001.log:12:24:30.569 0x1540 INF ksnclnt Ping: Allowed Services = {U, S, P2P, CERTINFO, F, FR, I, MDR, V}
- Open the folder where the Kaspersky Endpoint Security for Windows trace files are located and run the command:
- Use the instructions below to check that all mandatory components are enabled on the devices. You can optionally enable the recommended components to enrich the telemetry data:
- Kaspersky Endpoint Security for Windows
- Kaspersky Endpoint Security for Linux
- Kaspersky Endpoint Security for Mac
- Kaspersky Security for Virtualization
If telemetry losses from Kaspersky Endpoint Agent or Kaspersky Anti Targeted Attack Platform occur, make sure that integration with MDR is configured. - Make sure that the Managed Detection and Response policy is applied to the devices:
- The Managed detection and response check box is selected in the policy settings and editing of the setting is blocked (the lock is closed).
- The BLOB file is uploaded and the Remove button is active. See the instructions for Kaspersky Endpoint Security for Windows and Kaspersky Endpoint Security for Linux.
- The Kaspersky Private Security Network configuration file is uploaded to Kaspersky Security Center.
- Allow outgoing unencrypted network traffic to Kaspersky servers for ports 443 and 1443 on the devices and disable traffic scan.
- Follow the recommendations to configure the devices for stable telemetry data transfer.
How to diagnose and troubleshoot licensing problems
Troubleshooting
- For KESW:
- Open the folder where the Kaspersky Endpoint Security for Windows trace files are located.
- Check the license expiration date using the command:
grep -i "expiration date" KES*SRV*.log | head -1Example:KES.21.16.6.467.SRV_2024-05-13T150959.0001956_0001.log:12:10:11.665 0xe64 INF aveng klavsys: #41900993 expiration date: 2026-08-06T00-00-00Z (0x1DD25368624C000)
- Check the license expiration date in the BLOB file using the command:
grep -i "MdrBlobKeeper::UpdateBlob" KES*SRV*.logExample:KES.21.16.6.467.SRV_2024-05-13T150959.0001956_0001.log:12:10:01.531 0x53c INF bl [MdrBlobKeeper] mdr product::component::mdr::MdrBlobKeeper::UpdateBlob: new blob size: 2313, content: status: 0, version: 1, expirationDate: 2026-08-06T00:00:00.000Z, ignoreKpsn: true
- For KESL:
- Open the folder where the Kaspersky Endpoint Security for Linux trace files are located.
- Check the license expiration date using the command:
grep -ai 'expiration date' kesl* | tail -n 1Example:kesl.2946448.2023-08-24T173006.log:2023.08.24 14:30:10.578 2946458 INF aveng klavsys: #41900993 expiration date: 2026-08-06T00-00-00Z (0x1DD25368624C000)
- Make sure that the status of the BLOB file is active using the command:
grep -ai 'Blob status' kesl* | tail -1Example:kesl.2946448.2023-08-24T173006.log:2023.08.24 14:30:10.578 2946458 DBG mdr Blob status: Valid, version: 1, signerpk size: 550, payload size: 48
Solution
- Renew your license if it has expired.
- Update the BLOB file if it is invalid or has an incorrect license expiration date. To do so:
- Download the archive with the MDR configuration file again.
- Unzip the archive to extract the BLOB file.
- Upload the BLOB file to the policy settings. See the instructions for Kaspersky Endpoint Security for Windows and Kaspersky Endpoint Security for Linux.
- Distribute the policy to the devices.
How to diagnose and fix duplicated device IDs
Issue
Two devices appear and disappear in the MDR console one-by-one.
Cause
The problem is often caused by the same ID on two devices, such as cloned virtual machines.
Troubleshooting
To make sure that the reason is in duplicated IDs:
- Collect the trace files on both devices.
- Open the folder where the trace files are located.
- Compare the ID values on the devices:
- In the Kaspersky Endpoint Security for Windows traces, find the Machine id or base machine id value using the command:
grep -i 'machine id' KES*SRV*.logExample:KES.21.16.6.467.SRV_2024-05-13T150959.0001956_0001.log:12:10:11.665 0xe64 INF ksvla ksvla::virtual_machine::CurrentDevice::MachineId: Machine id: 06A3481D-D433-47F8-91AE-571B1734912B
KES.21.16.6.467.SRV_2024-05-13T150959.0001956_0001.log:12:10:11.665 0xe64 INF bl product::VirtualMachineInfoProvider::TryLoadBaseMachineIdFromPersistentCacheOrSaveDefult: base machine id (from persistent storage): 06A3481D-D433-47F8-91AE-571B1734912B - In the Kaspersky Endpoint Security for Linux traces, find the Machine ID salt loaded from KVS value using the command:
grep -ai 'machine id' kesl*Example:kesl.2053651.2023-08-12T002406.log:2023.08.11 21:24:09.442 2053671 INF aveng apsmdr: #51926414 Machine ID salt loaded from KVS: 62472cf2-4ac8-11de-2f18-5b186731a4d0
- In the Kaspersky Endpoint Security for Windows traces, find the Machine id or base machine id value using the command:
Solution
Change the ID on one of the devices:
- Contact Kaspersky Technical Support to get a patch for Kaspersky Endpoint Security for Windows 12.5, 12.4, 12.3 or 12.2.
- Reinstall Kaspersky Endpoint Security for Linux.
What to do if the issue persists
If the issue persists, collect diagnostic data and submit a request to Kaspersky Technical Support via Kaspersky CompanyAccount.