Logs

Log management and observability are no longer a matter of “storing text and searching it quickly.” In 2026, the real challenge is economic and architectural: ingesting growing volumes, retaining data for months, correlating logs with metrics and traces, and responding during an incident without costs spiraling out of control.

For years, Elasticsearch was the dominant option. Then came OpenSearch, Loki, Quickwit and, with significant momentum, ClickHouse as an analytical backend. All of them can solve part of the problem, but they do not optimize for the same things. The senior decision should not be “which one is faster” in the abstract, but which questions the team will ask, what retention it needs and how much complexity it can operate.

The Historical Standard: Elasticsearch and OpenSearch

OpenSearch began as an open source fork of Elasticsearch and keeps the mental model most familiar to many teams: indices, shards, replicas, mappings, Query DSL, aggregations and a mature full-text experience. If your organization already comes from Elasticsearch, it is the most direct transition.

Architecturally, it is built on Apache Lucene. Each index is divided into shards; each shard maintains segments and inverted indices; the cluster coordinates writes, replicas, distributed searches and aggregations. This provides a great deal of power, but also a clear operational footprint: JVM, heap sizing, merges, hot/warm/cold tiers and mapping tuning.

For logs, OpenSearch shines when you need expressive text search, rich filters, aggregations, mature dashboards, existing tooling or use cases close to SIEM. The cost appears as volume grows: indexing a lot of content makes it fast to search, but consumes storage and memory.

In 2026, the 3.x branch also matters. OpenSearch 3.0 introduced relevant changes such as Lucene 10 and JVM 21, along with performance improvements and vector search capabilities. That signals project vitality, but it also means upgrades need more careful planning than in a purely stateless stack.

Loki: Observability Centered on Efficiency

Grafana Loki takes a different philosophy: do not index the full content of logs, only the labels. In practice, it is closer to “Prometheus for logs” than to Elasticsearch: queries by labels and subsequent filtering over lines with LogQL.

Its storage is based on compressed chunks and a small index. Chunks usually live in object storage such as S3, GCS, Azure Blob or MinIO, while the index points to streams defined by label combinations. That decision reduces cost, but shifts responsibility to label design.

The practical rule is simple: labels must have controlled cardinality. cluster, namespace, app, environment or service usually work well. request_id, user_id or trace_id as labels can break the model because they multiply streams, hurt compaction and grow the index. For those values, it is better to filter log content or rely on traces.

Loki shines in Kubernetes because the label model fits pods, namespaces and services. Its Grafana integration is also excellent: log exploration, jumps from metrics, alerts with LogQL and correlation with Tempo or Prometheus.

Its weakness appears when you want to treat logs as an arbitrary search corpus. A query like “find this fragment across all logs from the last six months” can be much more expensive than in a system with a full-text index. Loki is designed for queries bounded by labels and time, not to replace a general-purpose search engine.

In 2026, Loki remains highly active. The official documentation lists recent 3.x versions, including the 3.7 branch, and Grafana continues to position it as a horizontally scalable, highly available and cost-efficient log aggregation system. Grafana’s acquisition of Logline further reinforces the focus on difficult searches within logs, although it does not change the central principle: label design remains critical.

When Loki Shines

Loki is the pragmatic choice for Kubernetes teams that already live in Grafana, need to retain volume at low cost and query by service, namespace, cluster, environment and time window. It is less convincing for global full-text search or deep forensic analysis.

Quickwit: Cloud-Native Search on Object Storage

Quickwit is a distributed search platform written in Rust, designed for append-only workloads such as logs and traces. Its thesis is attractive: keep powerful search, but decouple compute and storage by using object storage as the main layer.

Instead of depending on hot local disks, Quickwit writes indices and splits to S3-compatible storage, Azure Blob, Google Cloud Storage or MinIO. Indexers process data, searchers query segments and the metastore coordinates state. The model uses inverted indices optimized for immutable data and remote storage: closer to OpenSearch in search than Loki, but with cloud-native economics.

Quickwit is especially interesting for teams that want to replace part of an expensive Elasticsearch/OpenSearch cluster, already have logs in S3 or can move them there, and need text search with a better operational cost profile. It also has Grafana integration and APIs that are partially compatible with Elasticsearch/OpenSearch, which helps with some existing workflows.

The critical nuance for 2026 is its project status. Datadog acquired Quickwit in 2024. The acquisition did not mean the open source repository disappeared: the quickwit-oss/quickwit repository remains public and shows recent activity, including 2026 tags and discussions. The Quickwit Grafana plugin also publishes recent compatibility with Grafana 12.1+ and 13. That said, the product risk is different from Loki or OpenSearch: strategic direction is influenced by Datadog, and part of the effort may be oriented toward internal or commercial use cases.

The honest reading is this: Quickwit should not be dismissed as “dead,” because according to the reviewed sources it is not archived. But it should be evaluated with more diligence: release cadence, critical issues, roadmap, real API compatibility, operational ease and dependency on the maintainer team. For a core observability platform, that evaluation matters as much as the benchmark.

ClickHouse: The Analytical Engine of New Observability

ClickHouse was born as a columnar analytical database, not as a log platform. That is exactly why it has become relevant to observability. Many incidents are not solved by searching for an exact string, but by grouping by service, calculating percentiles, exploring cardinalities, joining logs with traces and querying billions of events over a time range.

ClickHouse’s model is columnar. Data is stored by columns, compresses extremely well and is queried with vectorized execution, ordered primary indices, data skipping indexes and engines such as MergeTree. For observability, this makes it possible to store wide events and read only the columns needed.

The tradeoff is that ClickHouse is not Elasticsearch. Full-text search exists and has improved, but its strength remains structured and semi-structured analysis. If your queries are “give me all 500 errors by version, region and endpoint in the last 15 minutes,” ClickHouse is a very good fit. If your main query is “find any line containing a rare phrase across six months of unstructured text,” you probably want another component or a hybrid strategy.

ClickHouse has moved decisively toward observability. The acquisition of HyperDX and the launch of ClickStack change its positioning: it is no longer only “use ClickHouse as the backend and build the UI yourself,” but a stack with OpenTelemetry, HyperDX as the interface and ClickHouse as the engine. It shines in high-volume platforms, structured logs, traces, derived metrics and low-latency aggregate queries, although it requires understanding modeling, TTLs, compression and ingestion costs.

Quick Comparison

Which Should You Choose in 2026?

The answer depends less on technology fashion and more on the team’s profile.

If You Are a Platform Team with Elasticsearch Heritage

OpenSearch is the conservative option if you already have automation, alerts and operational knowledge around Elasticsearch/OpenSearch. Do not migrate “as is” expecting costs to drop: review mappings, retention, ILM/ISM, shards, indexed fields and the separation between hot data and archive.

If You Are a Kubernetes Team Centered on Grafana

Loki is usually the first option. It is efficient, integrates very well with Grafana and reduces the cost of operational logs. The key is to govern labels from day one: allowed taxonomy, cardinality limits and context-driven queries before text searches.

If You Want to Reduce Cost Versus OpenSearch Without Losing Search

Quickwit deserves a serious PoC when Loki falls short on search and OpenSearch is expensive. The decision must include continuity: repository activity, releases, clients, community support, Grafana integration and Datadog’s position.

If You Want a Unified Observability Platform

ClickHouse is probably the strongest bet for consolidating logs, traces, derived metrics and events in a shared analytical database. ClickStack and HyperDX reduce the need to build the entire experience from scratch, but adoption depends on maturity in SQL, modeling and analytical database operations.

If You Have Security, Audit or SIEM Requirements

OpenSearch remains more natural for textual investigation and security tooling. ClickHouse can work for security analytics at scale, but it requires modeling. Loki is rarely the first choice for SIEM. Quickwit can be interesting, with a smaller ecosystem.

If Budget Is the Bottleneck

Do not choose only by storage cost per TB. Calculate total cost: ingestion, queries, retention, operations, backups, upgrades, training and incident time. Loki and ClickHouse usually win on raw cost; Quickwit can be highly competitive on object storage; OpenSearch can be reasonable if it is limited to data that is truly searchable and hot.

How to Frame a Decision PoC

A useful test is not installing four Helm charts and looking at a demo. Define real questions from past incidents and run them against representative data:

• The last 7 days of production logs, with real cardinality.
• A hot 2-hour window with high volume.
• A rare text search over a broad period.
• An aggregation by service, endpoint, version and region.
• A correlation flow from alert to logs and traces.

Measure sustained ingestion, storage cost, p50/p95 latency, operational complexity and team experience. Include an uncomfortable scenario: a failed node, slow object storage, unexpected cardinality or a badly written query.

Conclusion

The era when Elasticsearch was the only viable option for logs is over. In 2026, there are four clear paths.

OpenSearch is the mature choice for full-text search and compatibility. Loki is efficient for operational logs in Kubernetes. Quickwit offers modern search on object storage, with mandatory governance evaluation after the Datadog acquisition. ClickHouse is consolidating as an analytical engine for unified observability, especially with ClickStack and HyperDX.

The best platform is not the one that wins every benchmark. It is the one that answers your production questions with the lowest sustainable cost and the least operational friction for your team.

CTA: Decide with Data, Not Preferences

Before migrating or standardizing, run a two-week PoC with real data, real queries and an honest estimate of total cost. It should reveal which data belongs in search, which belongs in analytics and which belongs in cheap storage.

Sources

• OpenSearch 3.0: https://opensearch.org/blog/unveiling-opensearch-3-0/
• OpenSearch 3.0, Lucene 10 and JVM 21: https://opensearch.org/blog/opensearch-3-0-what-to-expect/
• Grafana Loki, documentation: https://grafana.com/docs/loki/latest/
• Grafana Loki, releases 3.x: https://grafana.com/docs/loki/latest/release-notes/
• Grafana Loki, labels and cardinality: https://grafana.com/blog/how-labels-in-loki-can-make-log-queries-faster-and-easier/
• Datadog acquires Quickwit: https://www.datadoghq.com/blog/datadog-acquires-quickwit/
• Quickwit OSS: https://github.com/quickwit-oss/quickwit
• Quickwit datasource for Grafana: https://github.com/quickwit-oss/quickwit-datasource
• ClickStack: https://clickhouse.com/clickstack
• ClickStack/HyperDX in ClickHouse Cloud: https://clickhouse.com/docs/use-cases/observability/clickstack/overview
• ClickHouse acquires HyperDX: https://clickhouse.com/blog/202504-newsletter
• ClickHouse and observability 2026: https://clickhouse.com/resources/engineering/what-is-observability