What's new in Apache Karaf Decanter 2.8.0?

Apache Karaf Decanter 2.8.0 has just been released. This release includes several fixes, improvements, and dependency updates.

I encourage all Decanter users to update to 2.8.0 release ;)

In this blog post, I will highlight some important changes and fixes we did in this release.

Prometheus appender

The Prometheus appender has been improved to expose more gauges.

As reminder, the Decanter Prometheus appender is basically a servlet that expose Prometheus compliant data, that Prometheus instances can poll to get the latest updated metrics.

Prometheus appender only looking for numeric data (coming from the Decanter collectors) to create and expose gauges.

Unfortunately, in previous Decanter releases, Prometheus appender only looking for numeric value for "first level" properties. It means if a collected data property value was a Map, no inner data was considered by Prometheus appender, even if inner values were numeric.

That's the first improvement we did in Prometheus appender: if the property value is a Map, the appender goes in the map values, looking for numeric type, and creating corresponding gauge.

Related to that, to avoid "confusion", the gauge name is now built (in the case of map) with collected data property key followed by the map entry key.

Another improvement we did on the Prometheus appender is to allow you to "select" the collected data you want to "expose" in the Prometheus servlet. By default, any numeric collected data are "rendered" by the Promethus appender. If you want to specify the properties you want to include in the Prometheus appender, you can add the selected properties in etc/org.apache.karaf.decanter.appender.prometheus.cfg by prefixing the selected properties with prometheus.key. and using true as value, like for instance: 


alias=/decanter/prometheus

prometheus.key.foo=true
prometheus.key.bar=true

InfluxDB appender fix

The InfluxDB appender contained a mistake in the OSGi heeaders definition, causing a ClassNotFoundException at startup.

It's now fixed on Decanter 2.8.0 and you can install the InfluxDB appender without problem.

Warning message on the socket collector bounded stream

In order to avoid huge memory consumption or DoS attack, the socket collector uses a bounded input stream, limiting the sent request size.

The bounded input stream limit can be configured via the max.request.size property in etc/org.apache.karaf.decanter.collector.socket.cfg, for instance: 


max.request.size=100000

You can use -1 for unbounded input stream.

Before Decanter 2.8.0, if the received request data was larger then the socket collector max request size, then the collector "truncated" the data without any warning messages. Some users were surprised to have some missing properties due to that without anything in the log.

In Decanter 2.8.0, the socket collector now log a warning message if the received request is larger. In that case, you could see this kind of log message: 


[WARN] Reach socket read input stream limit

Topic name definition in all collectors

You were already able to define the Decanter dispatcher topics name where the appenders are listening, thanks to the event.topics property in the appender configuration file. For instance, you can specify the topic name where the elasticsearch appender is listening using this in etc/org.apache.karaf.decanter.appender.elasticsearch.cfg configuration file: 


...
event.topics=decanter/collect/*
...

Thanks to that, the appenders are able to listen only some kind of events, or events coming from some collectors.

However, not all collectors allowed you to specify the topics where the collector is sending collected data. Decanter 2.8.0 now fixes that. You can also use event.topics property in any collector configuration file to specify the topic where the collector will send the collected data. For instance, for the JMX collector, you can now add: 


...
event.topics=decanter/collect/my-jmx
...

NB: event.topics is optional, meaning that the collector send to the same default topic as before.

Dependency updates

As usual, as in any Karaf release, Decanter 2.8.0 brings a bunch of dependency updates, especially:

  • velocity 2.3
  • commons-io 2.11
  • johnzon 1.2.14
  • dropwizard 1.2.3
  • jetty 9.4.43.v20210629
  • oshi 5.8.2
  • redisson 3.16.2
  • CXF 3.4.4
  • Camel 3.11.1
  • hadoop-client 3.3.1
  • kafka 2.8.0
  • orientdb-client 3.2.0
  • mongodb java driver 3.12.10
  • prometheus 0.11.0
  • lucene 8.9.0
  • aws-java-sdk 1.12.62

Comments

Post a Comment

Popular posts from this blog

Getting started with Apache Karaf Minho

Image
Apache Karaf Minho Apache Karaf Minho is a new runtime, completely created from scratch. The objectives are: super easy to use and run ligtning fast, including GraalVM support Kubernetes compliant to be easily used on the cloud helping to optimize cloud costs by supporting applications colocation providing turnkey services allowing you to create Minho applications in a minute Minho is composed by: minho-boot is the core of Minho. It contains the runtime launcher and the core services (service registry, lifecycle service, configuration service, ...). minho-* services provided additional functionnalities in the runtime. You can use these services to create your own services, it's also where you will find the services managing sepcific kind of applications, allowing colocation. minho-tooling optionally provide convenient tools to create your own runtime. It includes minho-maven-plugin for instance. In this blog post, we will take a quick tour on A
Read more

Exposing Apache Karaf configurations with Apache Arrow Flight

Image
This blog post shows how to use Apache Arrow and Arrow Flight in Apache Karaf runtime. The code base is available here: https://github.com/jbonofre/karaf-arrow . Apache Arrow ? Apache Arrow is composed by: The Arrow format (specification). Format implementations with different languages (C, Java, Go, Rust, ...). Extra subprojects like Flight, Flight SQL, ADBC, nanoarrow, ... Apache Arrow format is a in-memory columnar format, super efficient for scanning and iterating a large chunks of data. In traditional memory buffer, we focus on "row". Apache Arrow format "groups" data per column, which is much more efficient on modern CPUs/GPUs. The Apache Arrow format is basically a specification. We have several implementations for this specification: C C++ C# Go Java Python Ruby Rust As the purpose of this blog post is to show Apache Arrow in Apache Karaf, we will use Arrow Java. In addition, Apache Arrow also provides several
Read more

Using Apache Karaf with Kubernetes

Image
In a previous blog post ( http://blog.nanthrax.net/?p=893 ), I introduced the “new” docker support in Karaf and the use of Karaf static distribution. This post follows the previous one by illustrating how to use Karaf docker image with a Kubernetes cluster. Preparing Karaf static docker image As in the previous blog post, we are preparing a Karaf static distribution based docker image. For this blog post, I’m using the example provided in Karaf: https://github.com/apache/karaf/tree/master/examples/karaf-docker-example/karaf-docker-example-static-dist . I’m just building the distribution as usual: karaf-docker-example-static-dist$ mvn clean install Preparing kubernetes testing cluster For this blog, I’m using a minikube installed locally. The installation is pretty simple: Download minikube from https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 Rename minikube-linux-amd64 as minikube Copy minikube in your path, for instance /usr/local/bin Now that minikube
Read more