Observability In Kubernetes — Alexandre Vazquez

OpenSearch, Loki, Quickwit and ClickHouse: Which Platform Should You Choose for Logs and Observability in 2026?

2026-07-22 by

Log management and observability are no longer a matter of “storing text and searching it quickly.” In 2026, the real challenge is economic and architectural: ingesting growing volumes, retaining data for months, correlating logs with metrics and traces, and responding during an incident without costs spiraling out of control.

For years, Elasticsearch was the dominant option. Then came OpenSearch, Loki, Quickwit and, with significant momentum, ClickHouse as an analytical backend. All of them can solve part of the problem, but they do not optimize for the same things. The senior decision should not be “which one is faster” in the abstract, but which questions the team will ask, what retention it needs and how much complexity it can operate.

The Historical Standard: Elasticsearch and OpenSearch

OpenSearch began as an open source fork of Elasticsearch and keeps the mental model most familiar to many teams: indices, shards, replicas, mappings, Query DSL, aggregations and a mature full-text experience. If your organization already comes from Elasticsearch, it is the most direct transition.

Architecturally, it is built on Apache Lucene. Each index is divided into shards; each shard maintains segments and inverted indices; the cluster coordinates writes, replicas, distributed searches and aggregations. This provides a great deal of power, but also a clear operational footprint: JVM, heap sizing, merges, hot/warm/cold tiers and mapping tuning.

For logs, OpenSearch shines when you need expressive text search, rich filters, aggregations, mature dashboards, existing tooling or use cases close to SIEM. The cost appears as volume grows: indexing a lot of content makes it fast to search, but consumes storage and memory.

In 2026, the 3.x branch also matters. OpenSearch 3.0 introduced relevant changes such as Lucene 10 and JVM 21, along with performance improvements and vector search capabilities. That signals project vitality, but it also means upgrades need more careful planning than in a purely stateless stack.

Loki: Observability Centered on Efficiency

Grafana Loki takes a different philosophy: do not index the full content of logs, only the labels. In practice, it is closer to “Prometheus for logs” than to Elasticsearch: queries by labels and subsequent filtering over lines with LogQL.

Its storage is based on compressed chunks and a small index. Chunks usually live in object storage such as S3, GCS, Azure Blob or MinIO, while the index points to streams defined by label combinations. That decision reduces cost, but shifts responsibility to label design.

The practical rule is simple: labels must have controlled cardinality. cluster, namespace, app, environment or service usually work well. request_id, user_id or trace_id as labels can break the model because they multiply streams, hurt compaction and grow the index. For those values, it is better to filter log content or rely on traces.

Loki shines in Kubernetes because the label model fits pods, namespaces and services. Its Grafana integration is also excellent: log exploration, jumps from metrics, alerts with LogQL and correlation with Tempo or Prometheus.

Its weakness appears when you want to treat logs as an arbitrary search corpus. A query like “find this fragment across all logs from the last six months” can be much more expensive than in a system with a full-text index. Loki is designed for queries bounded by labels and time, not to replace a general-purpose search engine.

In 2026, Loki remains highly active. The official documentation lists recent 3.x versions, including the 3.7 branch, and Grafana continues to position it as a horizontally scalable, highly available and cost-efficient log aggregation system. Grafana’s acquisition of Logline further reinforces the focus on difficult searches within logs, although it does not change the central principle: label design remains critical.

When Loki Shines

Loki is the pragmatic choice for Kubernetes teams that already live in Grafana, need to retain volume at low cost and query by service, namespace, cluster, environment and time window. It is less convincing for global full-text search or deep forensic analysis.

Quickwit: Cloud-Native Search on Object Storage

Quickwit is a distributed search platform written in Rust, designed for append-only workloads such as logs and traces. Its thesis is attractive: keep powerful search, but decouple compute and storage by using object storage as the main layer.

Instead of depending on hot local disks, Quickwit writes indices and splits to S3-compatible storage, Azure Blob, Google Cloud Storage or MinIO. Indexers process data, searchers query segments and the metastore coordinates state. The model uses inverted indices optimized for immutable data and remote storage: closer to OpenSearch in search than Loki, but with cloud-native economics.

Quickwit is especially interesting for teams that want to replace part of an expensive Elasticsearch/OpenSearch cluster, already have logs in S3 or can move them there, and need text search with a better operational cost profile. It also has Grafana integration and APIs that are partially compatible with Elasticsearch/OpenSearch, which helps with some existing workflows.

The critical nuance for 2026 is its project status. Datadog acquired Quickwit in 2024. The acquisition did not mean the open source repository disappeared: the quickwit-oss/quickwit repository remains public and shows recent activity, including 2026 tags and discussions. The Quickwit Grafana plugin also publishes recent compatibility with Grafana 12.1+ and 13. That said, the product risk is different from Loki or OpenSearch: strategic direction is influenced by Datadog, and part of the effort may be oriented toward internal or commercial use cases.

The honest reading is this: Quickwit should not be dismissed as “dead,” because according to the reviewed sources it is not archived. But it should be evaluated with more diligence: release cadence, critical issues, roadmap, real API compatibility, operational ease and dependency on the maintainer team. For a core observability platform, that evaluation matters as much as the benchmark.

ClickHouse: The Analytical Engine of New Observability

ClickHouse was born as a columnar analytical database, not as a log platform. That is exactly why it has become relevant to observability. Many incidents are not solved by searching for an exact string, but by grouping by service, calculating percentiles, exploring cardinalities, joining logs with traces and querying billions of events over a time range.

ClickHouse’s model is columnar. Data is stored by columns, compresses extremely well and is queried with vectorized execution, ordered primary indices, data skipping indexes and engines such as MergeTree. For observability, this makes it possible to store wide events and read only the columns needed.

The tradeoff is that ClickHouse is not Elasticsearch. Full-text search exists and has improved, but its strength remains structured and semi-structured analysis. If your queries are “give me all 500 errors by version, region and endpoint in the last 15 minutes,” ClickHouse is a very good fit. If your main query is “find any line containing a rare phrase across six months of unstructured text,” you probably want another component or a hybrid strategy.

ClickHouse has moved decisively toward observability. The acquisition of HyperDX and the launch of ClickStack change its positioning: it is no longer only “use ClickHouse as the backend and build the UI yourself,” but a stack with OpenTelemetry, HyperDX as the interface and ClickHouse as the engine. It shines in high-volume platforms, structured logs, traces, derived metrics and low-latency aggregate queries, although it requires understanding modeling, TTLs, compression and ingestion costs.

Quick Comparison

Characteristic	OpenSearch	Loki	Quickwit	ClickHouse
Main model	Distributed search on Lucene	Logs by streams and labels	Cloud-native distributed search	Columnar analytical database
Storage model	Shards, Lucene segments, hot/warm/cold tiers	Compressed chunks + small index	Indices/splits in object storage	Compressed columns in MergeTree
Index	Inverted, very complete	Labels, not full content	Inverted, optimized for object storage	Primary index, data skipping, columns
Full-text search	Excellent	Limited and dependent on filters	Very good	Good, not its main strength
Relative cost	High	Low	Low-medium	Low-medium
Query latency	Low on well-sized hot indices	Low if bounded by labels; worse on broad scans	Low-medium depending on cache and object storage	Very low on well-modeled aggregations
Scaling	Mature, but operationally involved	Horizontal and economical	Decouples compute and storage	Excellent for analytics and ingestion
Kubernetes	Good	Excellent	Good	Good, better with ClickStack/OTel
Grafana	Yes	Native	Yes	Yes
Elasticsearch compatibility	High in OpenSearch	No	Partial	No
Best use case	Complex search and compatibility	Low-cost Kubernetes logs	Efficient search on S3	Unified analytical observability
Main risk	Cost and complexity	Label cardinality	Ecosystem and governance after acquisition	Modeling and SQL/operations learning curve

Which Should You Choose in 2026?

The answer depends less on technology fashion and more on the team’s profile.

If You Are a Platform Team with Elasticsearch Heritage

OpenSearch is the conservative option if you already have automation, alerts and operational knowledge around Elasticsearch/OpenSearch. Do not migrate “as is” expecting costs to drop: review mappings, retention, ILM/ISM, shards, indexed fields and the separation between hot data and archive.

If You Are a Kubernetes Team Centered on Grafana

Loki is usually the first option. It is efficient, integrates very well with Grafana and reduces the cost of operational logs. The key is to govern labels from day one: allowed taxonomy, cardinality limits and context-driven queries before text searches.

If You Want to Reduce Cost Versus OpenSearch Without Losing Search

Quickwit deserves a serious PoC when Loki falls short on search and OpenSearch is expensive. The decision must include continuity: repository activity, releases, clients, community support, Grafana integration and Datadog’s position.

If You Want a Unified Observability Platform

ClickHouse is probably the strongest bet for consolidating logs, traces, derived metrics and events in a shared analytical database. ClickStack and HyperDX reduce the need to build the entire experience from scratch, but adoption depends on maturity in SQL, modeling and analytical database operations.

If You Have Security, Audit or SIEM Requirements

OpenSearch remains more natural for textual investigation and security tooling. ClickHouse can work for security analytics at scale, but it requires modeling. Loki is rarely the first choice for SIEM. Quickwit can be interesting, with a smaller ecosystem.

If Budget Is the Bottleneck

Do not choose only by storage cost per TB. Calculate total cost: ingestion, queries, retention, operations, backups, upgrades, training and incident time. Loki and ClickHouse usually win on raw cost; Quickwit can be highly competitive on object storage; OpenSearch can be reasonable if it is limited to data that is truly searchable and hot.

How to Frame a Decision PoC

A useful test is not installing four Helm charts and looking at a demo. Define real questions from past incidents and run them against representative data:

The last 7 days of production logs, with real cardinality.
A hot 2-hour window with high volume.
A rare text search over a broad period.
An aggregation by service, endpoint, version and region.
A correlation flow from alert to logs and traces.

Measure sustained ingestion, storage cost, p50/p95 latency, operational complexity and team experience. Include an uncomfortable scenario: a failed node, slow object storage, unexpected cardinality or a badly written query.

Conclusion

The era when Elasticsearch was the only viable option for logs is over. In 2026, there are four clear paths.

OpenSearch is the mature choice for full-text search and compatibility. Loki is efficient for operational logs in Kubernetes. Quickwit offers modern search on object storage, with mandatory governance evaluation after the Datadog acquisition. ClickHouse is consolidating as an analytical engine for unified observability, especially with ClickStack and HyperDX.

The best platform is not the one that wins every benchmark. It is the one that answers your production questions with the lowest sustainable cost and the least operational friction for your team.

CTA: Decide with Data, Not Preferences

Before migrating or standardizing, run a two-week PoC with real data, real queries and an honest estimate of total cost. It should reveal which data belongs in search, which belongs in analytics and which belongs in cheap storage.

Sources

OpenSearch 3.0: https://opensearch.org/blog/unveiling-opensearch-3-0/
OpenSearch 3.0, Lucene 10 and JVM 21: https://opensearch.org/blog/opensearch-3-0-what-to-expect/
Grafana Loki, documentation: https://grafana.com/docs/loki/latest/
Grafana Loki, releases 3.x: https://grafana.com/docs/loki/latest/release-notes/
Grafana Loki, labels and cardinality: https://grafana.com/blog/how-labels-in-loki-can-make-log-queries-faster-and-easier/
Datadog acquires Quickwit: https://www.datadoghq.com/blog/datadog-acquires-quickwit/
Quickwit OSS: https://github.com/quickwit-oss/quickwit
Quickwit datasource for Grafana: https://github.com/quickwit-oss/quickwit-datasource
ClickStack: https://clickhouse.com/clickstack
ClickStack/HyperDX in ClickHouse Cloud: https://clickhouse.com/docs/use-cases/observability/clickstack/overview
ClickHouse acquires HyperDX: https://clickhouse.com/blog/202504-newsletter
ClickHouse and observability 2026: https://clickhouse.com/resources/engineering/what-is-observability

Prometheus Monitoring in TIBCO Cloud Integration

2022-05-072019-12-19 by Alexandre Vazquez

In previous posts, I’ve explained how to integrate TIBCO BusinessWorks 6.x / BusinessWorks Container Edition (BWCE) applications with Prometheus, one of the most popular monitoring systems for cloud layers. Prometheus is one of the most widely used solutions to monitor your microservices inside a Kubernetes cluster. In this post, I will explain steps to leverage Prometheus for integrating with applications running on TIBCO Cloud Integration (TCI).

TCI is TIBCO’s iPaaS and primarily hides the application management complexity of an app from users. You need your packaged application (a.k.a EAR) and manifest.json — both generated by the product to simply deploy the application.

Isn’t it magical? Yes, it is! As explained in my previous post related to Prometheus integration with BWCE, which allows you to customize your base images, TCI allows integration with Prometheus in a slightly different manner. Let’s walk through the steps.

TCI has its own embedded monitoring tools (shown below) to provide insights into Memory and CPU utilization, plus network throughput, which is very useful.

While the monitoring metrics provided out-of-the-box by TCI are sufficient for most scenarios, there are hybrid connectivity use-cases (application running on-prem and microservices running on your own cluster that could be on a private or public cloud) that might require a unified single-pane view of monitoring.

Step one is to import the Prometheus plugin from the current GitHub location into your BusinessStudio workspace. To do that, you just need to clone the GitHub Repository available here: https://github.com/TIBCOSoftware/bw-tooling OR https://github.com/alexandrev/bw-tooling

Import the Prometheus plugin by choosing Import → Plug-ins and Fragments option and specifying the directory downloaded from the above mentioned GitHub location. (shown below)

Step two involves adding the Prometheus module previously imported to the specific application as shown below:

Step three is just to build the EAR file along with manifest.json.

NOTE: If the EAR doesn’t get generated once you add the Prometheus plugin, please follow the below steps:

Export the project with the Prometheus module to a zip file.
Remove the Prometheus project from the workspace.
Import the project from the zip file generated before.

Before you deploy the BW application on TCI, we need to enable an additional port on TCI to scrape the Prometheus metrics.

Step four Updating manifest.json file.

By default, a TCI app using the manifest.json file only exposes one port to be consumed from outside (related to functional services) and the other to be used internally for health checks.

For Prometheus integration with TCI, we need an additional port listening on 9095, so Prometheus server can access the metrics endpoints to scrape the required metrics for our TCI application.

Note: This document does not cover the details on setting the Prometheus server (it is NOT needed for this PoC) but you can find the relevant information on https://prometheus.io/docs/prometheus/latest/installation/

We need to slightly modify the generated manifest.json file (of BW app) to expose an additional port, 9095 (shown below) .

Also, to tell TCI that we want to enable Prometheus endpoint we need to set a property in the manifest.json file. The property is TCI_BW_CONFIG_OVERRIDES and provide the following value: BW_PROMETHEUS_ENABLE=true, as shown below:

We also need to add an additional line (propertyPrefix) in the manifest.json file as shown below.

Now, we are ready to deploy the BW app on TCI and once it is deployed we can see there are two endpoints

If we expand the Endpoints options on the right (shown above), you can see that one of them is named “prometheus” and that’s our Prometheus metrics endpoint:

Just copy the prometheus URL and append it with /metrics (URL in the below snapshot) — this will display the Prometheus metrics for the specific BW app deployed on TCI.

Note: appending with /metrics is not compulsory, the as-is URL for Prometheus endpoint will also work.

In the list you will find the following kind of metrics to be able to create the most incredible dashboards and analysis based on that kind of information:

JVM metrics around memory used, GC performance and thread pools counts
CPU usage by the application
Process and Activity execution counts by Status (Started, Completed, Failed, Scheduled..)
Duration by Activity and Process.

With all this available the information you can create dashboards similar to the one shown below, in this case using Spotfire as the Dashboard tool:

But you can also integrate those metrics with Grafana or any other tool that could read data from Prometheus time-series database.

Kubernetes Service Discovery for Prometheus: Dynamic Scraping the Right Way

2026-01-162019-08-05 by Alexandre Vazquez

In previous posts, we described how to set up Prometheus to work with your TIBCO BusinessWorks Container Edition apps, and you can read more about it here.

Prometheus Monitoring for Microservices using TIBCO

We’re living a world with constant changes and this is even more true in the Enterprise Application world. I’ll not spend much time talking about things you already know, but just say that the microservices architecture approach and the PaaS solutions have been a game-changer for all enterprise integration technologies. This time I’d like to […]

In that post, we described that there were several ways to update Prometheus about the services that ready to monitor. And we choose the most simple at that moment that was the static_config configuration which means:

Don’t worry Prometheus, I’ll let you know the IP you need to monitor and you don’t need to worry about anything else.

And this is useful for a quick test in a local environment when you want to test quickly your Prometheus set up or you want to work in the Grafana part to design the best possible dashboard to handle your need.

But, this is not too useful for a real production environment, even more, when we’re talking about a Kubernetes cluster when services are going up & down continuously over time. So, to solve this situation Prometheus allows us to define a different kind of ways to perform this “service discovery” approach. In the official documentation for Prometheus, we can read a lot about the different service discovery techniques but at a high level these are the main service discovery techniques available:

Configuration | Prometheus

An open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.

azure_sd_configs: Azure Service Discovery
consul_sd_configs: Consul Service Discovery
dns_sd_configs: DNS Service Discovery
ec2_sd_configs: EC2 Service Discovery
openstack_sd_configs: OpenStack Service Discovery
file_sd_configs: File Service Discovery
gce_sd_configs: GCE Service Discovery
kubernetes_sd_configs: Kubernetes Service Discovery
marathon_sd_configs: Marathon Service Discovery
nerve_sd_configs: AirBnB’s Nerve Service Discovery
serverset_sd_configs: Zookeeper Serverset Service Discovery
triton_sd_configs: Triton Service Discovery
static_config: Static IP/DNS for the configuration. No Service Discovery.

And even, it all these options are not enough for you and need something more specific you have an API available to extend the Prometheus capabilities and create your own Service Discovery technique. You can find more info about it here:

Implementing Custom Service Discovery | Prometheus

An open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.

But this is not our case, for us, the Kubernetes Service Discovery is the right choice for our approach. So, we’re going to change the static configuration we had in the previous post:

- job_name: 'bwdockermonitoring'
  honor_labels: true
  static_configs:
    - targets: ['phenix-test-project-svc.default.svc.cluster.local:9095']
      labels:
        group: 'prod'

For this Kubernetes configuration

- job_name: 'bwce-metrics'
  scrape_interval: 5s
  metrics_path: /metrics/
  scheme: http
  kubernetes_sd_configs:
  - role: endpoints
    namespaces:
      names:
      - default
  relabel_configs:
  - source_labels: [__meta_kubernetes_service_label_app]
    separator: ;
    regex: (.*)
    replacement: $1
    action: keep
  - source_labels: [__meta_kubernetes_endpoint_port_name]
    separator: ;
    regex: prom
    replacement: $1
    action: keep
  - source_labels: [__meta_kubernetes_namespace]
    separator: ;
    regex: (.*)
    target_label: namespace
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_pod_name]
    separator: ;
    regex: (.*)
    target_label: pod
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: service
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: job
    replacement: 1
    action: replace
  - separator: ;
    regex: (.*)
    target_label: endpoint
    replacement: $1
    action: replace

As you can see this is quite more complex than the previous configuration but it is not as complex as you can think at first glance, let’s review it by different parts.

- role: endpoints
    namespaces:
      names:
      - default

It says that we’re going to use role for endpoints that are created under the default namespace and we’re going to specify the changes we need to do to find the metrics endpoints for Prometheus.

scrape_interval: 5s
 metrics_path: /metrics/
 scheme: http

This says that we’re going to execute the scrape process in a 5 seconds interval, using http on the path /metrics/

And then, we have a relabel_config section:

- source_labels: [__meta_kubernetes_service_label_app]
    separator: ;
    regex: (.*)
    replacement: $1
    action: keep
  - source_labels: [__meta_kubernetes_endpoint_port_name]
    separator: ;
    regex: prom
    replacement: $1
    action: keep

That means that we’d like to keep that label for prometheus:

- source_labels: [__meta_kubernetes_namespace]
    separator: ;
    regex: (.*)
    target_label: namespace
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_pod_name]
    separator: ;
    regex: (.*)
    target_label: pod
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: service
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: job
    replacement: 1
    action: replace
  - separator: ;
    regex: (.*)
    target_label: endpoint
    replacement: $1
    action: replace

That means that we want to do a replace of the label value and we can do several things:

Rename the label name using the target_label to set the name of the final label that we’re going to create based on the source_labels.
Replace the value using the regex parameter to define the regular expression for the original value and the replacement parameter that is going to express the changes that we want to do to this value.

So, now after applying this configuration when we deploy a new application in our Kubernetes cluster, like the project that we can see here:

Automatically we’re going to see an additional target on our job-name configuration “bwce-metrics”

📚 Want to dive deeper into Kubernetes? This article is part of our comprehensive Kubernetes Architecture Patterns guide, where you’ll find all fundamental and advanced concepts explained step by step.

Prometheus TIBCO Monitoring for Containers: Quick and Simple in 5 Minutes!

2026-01-152019-07-22 by Alexandre Vazquez

Prometheus is becoming the new standard for Kubernetes monitoring and today we are going to cover how we can do Prometheus TIBCO monitoring in Kubernetes.

This article is part of my comprehensive TIBCO Integration Platform Guide where you can find more patterns and best practices for TIBCO integration platforms.

We’re living in a world with constant changes and this is even more true in the Enterprise Application world. I’ll not spend much time talking about things you already know, but just say that the microservices architecture approach and the PaaS solutions have been a game-changer for all enterprise integration technologies.

This time I’d like to talk about monitoring and the integration capabilities we have of using Prometheus to monitor our microservices developed under TIBCO technology. I don’t like to spend too much time either talking about what Prometheus is, as you probably already know, but in a summary, this is an open-source distributed monitoring platform that has been the second project released by the Cloud Native Computing Foundation (after Kubernetes itself) and that has been established as a de-facto industry standard for monitoring K8S clusters (alongside with other options in the market like InfluxDB and so on).

Prometheus has a lot of great features, but one of them is that it has connectors for almost everything and that’s very important today because it is so complicated/unwanted/unusual to define a platform with a single product for the PaaS layer. So today, I want to show you how to monitor your TIBCO BusinessWorks Container Edition applications using Prometheus.

Most of the info I’m going to share is available in the bw-tooling GitHub repo, so you can get to there if you need to validate any specific statement.

bw-tooling/prometheus-integration at master · TIBCOSoftware/bw-tooling

Collection of tools designed to simplify deployment and management of TIBCO BusinessWorks applications – bw-tooling/prometheus-integration at master · TIBCOSoftware/bw-tooling

Ok, are we ready? Let’s start!!

I’m going to assume that we already have a Kubernetes cluster in place and Prometheus installed as well. So, the first step is to enhance the BusinessWorks Container Edition base image to include the Prometheus capabilities integration. To do that we need to go to the GitHub repo page and follow these instructions:

Download & unzip the prometheus-integration.zip folder.

Open TIBCO BusinessWorks Studio and point it to a new workspace.
Right-click in Project Explorer → Import… → select Plug-ins and Fragments → select Import from the directory radio button

Browse it to prometheus-integration folder (unzipped in step 1)

Now click Next → Select Prometheus plugin → click Add button → click Finish. This will import the plugin in the studio.

Now, to create JAR of this plugin so first, we need to make sure to update com.tibco.bw.prometheus.monitor with ‘.’ (dot) in Bundle-Classpath field as given below in META-INF/MANIFEST.MF file.

Right-click on Plugin → Export → Export…

Select type as JAR file click Next

Now Click Next → Next → select radio button to use existing MANIFEST.MF file and browse the manifest file

Click Finish. This will generate prometheus-integration.jar

Now, with the JAR already created what we need to do is include it in your own base image. To do that we place the JAR file in the <TIBCO_HOME>/bwce/2.4/docker/resources/addons/jar

And we launch the building image command again from the <TIBCO_HOME>/bwce/2.4/docker folder to update the image using the following command (use the version you’re using at the moment)

docker build -t bwce_base:2.4.4 .

So, now we have an image with Prometheus support! Great! We’re close to the finish, we just create an image for our Container Application, in my case, this is going to be a very simple echo service that you can see here.

And we only need to keep these things in particular when we deploy to our Kubernetes cluster:

We should set an environment variable with the BW_PROMETHEUS_ENABLE to “TRUE”
We should expose the port 9095 from the container to be used by Prometheus to integrate.

Now, we only need to provide this endpoint to the Prometheus scrapper system. There are several ways to do that, but we’re going to focus on the simple one.

We need to change the prometheus.yml to add the following job data:

- job_name: 'bwdockermonitoring'
  honor_labels: true
  static_configs:
    - targets: ['phenix-test-project-svc.default.svc.cluster.local:9095']
      labels:
        group: 'prod'

And after restarting Prometheus we have all the data indexed in the Prometheus database to be used for any dashboard system.

In this case, I’m going to use Grafana to do quick dashboard.

Each of these graph components is configured based on the metrics that are being scraped by Prometheus TIBCO exporter.

OpenTracing in TIBCO BusinessWorks Container Edition: Tracing with Jaeger Explained

2026-01-162019-06-10 by Alexandre Vazquez

The past month during the KubeCon 2019 Europe in Barcelona OpenTracing announces its merge with OpenCensus project to create a new standard named OpenTelemetry that is going to be live in September 2019.

OpenTracing, OpenCensus Merge into a Single New Project, OpenTelemetry - The New Stack

Two open source projects that have been instrumental in providing metrics for cloud native operations have merged into a single project. The fusion of Google’s OpenCensus and the Cloud Native Computing Foundation’s OpenTracing will be known as OpenTelemetry, and will be managed by the CNCF. The idea…

So, I think that would be awesome to take a look at the capabilities regarding OpenTracing we have available in TIBCO BusinessWorks Container Edition

Today’s world is too complex in terms of how our architectures are defined and managed. New concepts in the last years like containers, microservices, service mesh, give us the option to reach a new level of flexibility, performance, and productivity but also comes with a cost of management we need to deal with.

Years ago architectures were simpler, service was a concept that was starting out, but even then a few issues begin to arise regarding monitoring, tracing, logging and so on. So, in those days everything was solved with a Development Framework that all our services were going to include because all of our services were developed by the same team, same technology, and in that framework, we can make sure things were handled properly.

Now, we rely on standards to do this kind of things, and for example, for Tracing, we rely on OpenTracing. I don’t want to spend time talking about what OpenTracing is where they have a full medium account talking themselves much better than I could ever do, so please take some minutes to read about it.

Distributed Tracing in 10 Minutes

With the intrinsic concurrency and asynchrony of modern software applications, distributed tracing has become part of the table stakes for effective monitoring. That said, instrumenting a system for…

The only statement I want to do here is the following one:

Tracing is not Logging, and please be sure you understand that.

Tracing is about sampling, it’s like how flows are performing and if everything is worked but it is not about a specific request has been done well for customer ID whatever… that’s logging, no tracing.

So OpenTracing and its different implementations like Jaeger or Zipkin are the way we can implement tracing today in a really easy way, and this is not something that you could only do in your code-based development language, you can do it with our zero-code tools to develop cloud-native applications like TIBCO BusinessWorks Container Edition and that’s what I’d like to show you today. So, let the match, begin…

First thing I’d like to do is to show you the scenario we’re going to implement, and this is going to be the one shown in the image below:

You are going to have two REST service that is going to call one to each other, and we’re going to export all the traces to Jaeger external component and later we can use its UI to analyze the flow in a graphical and easy way.

So, the first thing we need to do is to develop the services that as you can see in the pictures below are going to be quite easy because this is not the main purpose of our scenario.

Once, we have our docker images based on those applications we can start, but before we launch our applications, we need to launch our Jaeger system you can read all info about how to do it in the link below:

Getting started

Get up and running with Jaeger in your local environment

But at the end we only to run the following command:

docker run -d --name jaeger -e COLLECTOR_ZIPKIN_HTTP_PORT=9411  -p 5775:5775/udp  -p 6831:6831/udp  -p 6832:6832/udp  -p 5778:5778  -p 16686:16686  -p 14268:14268  -p 9411:9411  jaegertracing/all-in-one:1.8

And now, we’re ready to launch our applications and the only things we need to do in our developments because as you could see we didn’t do anything strange in our development and it was quite straightforward is to add the following environment variables when we launch our container

BW_JAVA_OPTS=”-Dbw.engine.opentracing.enable=true” -e JAEGER_AGENT_HOST=jaeger -e JAEGER_AGENT_PORT=6831 -e JAEGER_SAMPLER_MANAGER_HOST_PORT=jaeger:5778

And… that’s it, we launch our containers with the following commands and wait until applications are up & running

docker run -ti -p 5000:5000 — name provider -e BW_PROFILE=Docker -e PROVIDER_PORT=5000 -e BW_LOGLEVEL=ERROR — link jaeger -e BW_JAVA_OPTS=”-Dbw.engine.opentracing.enable=true” -e JAEGER_AGENT_HOST=jaeger -e JAEGER_AGENT_PORT=6831 -e JAEGER_SAMPLER_MANAGER_HOST_PORT=jaeger:5778 provider:1.0

docker run — name consumer -ti -p 6000:6000 -e BW_PROFILE=Docker — link jaeger — link provider -e BW_JAVA_OPTS=”-Dbw.engine.opentracing.enable=true” -e JAEGER_AGENT_HOST=jaeger -e JAEGER_AGENT_PORT=6831 -e JAEGER_SAMPLER_MANAGER_HOST_PORT=jaeger:5778 -e CONSUMER_PORT=6000 -e PROVIDER_HOST=provider consumer:1.0

Once they’re running, let’s generate some requests! To do that I’m going to use a SOAPUI project to generate some stable load for 60 secs, as you can see in the image below:

And now we’re going to go to the following URL to see the Jaeger UI and we can see the following thing as soon as you click in the Search button

And then if we zoom in some specific trace:

That’s pretty amazing but that’s not all, because you can see if you search in the UI about the data of this traces, you can see technical data from your BusinessWorks Container Edition flows as you can see in the picture below:

But… what if you want to add your custom tags to those traces? You can do it as well!! Let me explain to you how.

Since BusinessWorks Container Edition 2.4.4 you are going to find a new tab in all your activities named “Tags” where you can add the custom tags that you want this activity to include, for example, a custom id that is going to be propagated through the whole process we can define it as you can see here.

And if you take a look at the data we have in the system, you can see all of these traces has this data:

You can take a look at the code in the following GitHub repository:

GitHub - alexandrev/bwce-opentracing-customtags

Contribute to alexandrev/bwce-opentracing-customtags development by creating an account on GitHub.