Logit.io Response to Apache Log4j 2 CVE-2021-44228
Incident Report for Logit.io
Resolved
All stacks have been patched and are operational. We have removed the org/apache/logging/log4j/core/lookup/JndiLookup.class file from all installations of log4j
Posted Dec 14, 2021 - 00:23 UTC
Update
All Elasticsearch clusters in the UK and US regions have been patched and updated. We continue in parallel to update the EU region
Posted Dec 13, 2021 - 20:44 UTC
Update
We can confirm we have applied the patches to all Logstash Inputs for Stacks in the EU Regions.
Posted Dec 13, 2021 - 20:05 UTC
Update
Update: We can confirm that we have now also applied the patches to Logstash for Filter nodes in the EU region.
Posted Dec 13, 2021 - 16:59 UTC
Update
Update: We can confirm that we have now also applied the patches to Logstash for Filter nodes in the EU region.
Posted Dec 13, 2021 - 16:58 UTC
Update
Update: Our engineers are in the process of rolling out a patched version of Logstash for both Input and Filter nodes for all active versions of ELK. We can confirm we have applied the patches to all Stacks in the UK and US Regions, and we are awaiting the completion of the EU Region.
Posted Dec 13, 2021 - 14:10 UTC
Update
Update: Our engineers are in the process of rolling out a patched version of Logstash for both Input and Filter nodes for all active versions of ELK.
We can confirm we have applied the patches to all Stacks in the UK and US Regions, and we are awaiting the completion of the EU Region.
Posted Dec 13, 2021 - 14:07 UTC
Identified
Follow updates here on how our teams are currently responding with the highest priority to vulnerability CVE-2021-44228 that is impacting multiple versions of the Apache Log4j 2 which was disclosed publicly via Github on December 9th 2021.

Logit.io engineers and security incident teams continue to actively analyse, identify and where necessary patch all affected log4j versions across all Logit.io data centre instances.

Logit.io are initially updating and following the current remediation advice as a first priority as detailed here https://github.com/advisories/GHSA-jfh8-c2jp-5v3q

"In previous releases (>=2.10) this behaviour can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class)"

This remediation will be applied to all versions of Logit.io instances and we will post further updates here as this is rolled out.

In addition the Logit.io security team has analysed its logs and updated its monitoring of all internal services to include active alerting of any attempts to issue remote code execution via the JndiLookup class.

The information leak in Log4j does not permit access to data within an Elasticsearch cluster but allows an attacker to extract certain environmental data via DNS. The data leaked is limited to that available via Log4j lookups.

If you have any questions or concerns, please reach out to us via the normal support channels or email support@logit.io. You can also choose to subscribe to updates at the top of this page to receive all future notifications.
Posted Dec 13, 2021 - 10:29 UTC
This incident affected: UK Region (Elasticsearch Hosts, Logstash Ingestion Hosts, Logstash Filtering Hosts), EU Region (Elasticsearch Hosts, Logstash Ingestion Hosts, Logstash Filtering Hosts), and US Region (Elasticsearch Hosts, Logstash Ingestion Hosts, Logstash Filtering Hosts).