The Kieker monitoring framework and toolset comprises different monitoring probes utilizing different probe introduction mechanisms for Java and other languages, a wide variety of analysis stages to receive, read and analyze monitoring data, and tools to support handling data and creating analyses based on monitoring data. In recent years we started to use the new pipe-and-filter framework TeeTime for the analysis and tool part which reduces the amount of configuration and implementation code necessary to create your own combination of analyses.

In the iObserve project, we heavily relied on TeeTime and the new Kieker stages (filter) implemented and ported to work with TeeTime. In this process, we created new tools and ported older Kieker tools to TeeTime. Furthermore, we updated the build process to create ready to use packages for these tools. Therefore, users of Kieker tooling do no longer need to download the complete Kieker package and extract the tools they need.

In this article, we will introduce the new stages and tools which originated in iObserve or where first used by iObserve, as the key user for the addition. As these additions are numerous, we will present only a brief overview in this article and add new posts to provide more details to certain new additions.

In case you are in a rush, all the new features can be found in the current Kieker snapshot and will be part of Kieker 1.15.

New Tool Packages

In the past Kieker produced one large distribution package containing all tools, libraries, probes, and setups. This had the advantage to present especially students with an easy way to get all what they need in one download. However, nowadays Kieker is used in more research and commercial projects and people want to have packages for specific tasks. To support this need, we now produce specific packages for tools which can be found here. As our efforts will progress, more tools will be available separately. Documentation to the tools is available on our wiki.

• Trace Analysis Tool allows to analyze call traces through applications
• Convert Logging Timestamps converts timestamps into local time
• Log Replayer allows to replay logs to feed them into analysis services, like iObserve and ExplorViz.
• Collector – Kieker Data Bridge allows to receive monitoring data from different source including TCP/binary, HTTP/JSON, files etc. and stores them in a joint log file or repeats them to another node.
• Resource Monitor
• Trace Analysis — GUI a graphical UI to control the Trace Analysis Tool
• Dot-Pic File Converter Helper script to process output files from the Trace Analysis Tool

Kieker Tools Library

The tools in Kieker are highly configurable. Especially when it comes to reading, receiving or writing monitoring logs or other information. This requires that tools can adapt their pipe-and-filter configuration at startup time. As this can be cumbersome to maintain this adaptive feature in an ever growing number of tools, we collected such functionality in the Kieker tool library, reducing the implementation cost. Therefore, most tools can receive data from any reader implemented in Kieker, e.g., binary TCP (compressed or uncompressed), via REST API and from files in different formats.

Ongoing Changes to the Build System

The Kieker build system was based on an growing ant file in the past. We switched to gradle in 2015. However, many special constructs from the ant file had to be ported to gradle and do not fit typical modern Java gradle project layouts and workflows. Therefore, we slowly migrate to a more canonical layout for the Kieker project. One effect of this migration are the aforementioned tool packages. Furthermore, these changes will enable us to create releases faster and provide better support for releases in the long run.

Documentation

Kieker comes with a tutorial and usage guide in PDF. While this was a practical approach in the past, it is not fitting for today. Therefore, we are moving the Kieker documentation to our wiki pages. In the process, we also upgrade the documentation to reflect the current state of Kieker. The Kieker wiki can be found here.

Further Plans

Our future plans aim to improve build quality, release cycle and delivery of packages. We will also collect new stages (filters) and tools from the iObserve project. Furthermore, we will expand and document existing features of Kieker to support C, Fortran, Python and Javascript with monitoring probes capable to communicate with the collector and all tools using the same adaptive reader stages.

We already support C and at least two ways to instrument C and C++ applications one utilizing Aspect C++ and one utilizing features of the GNU Compiler Collection which supports instrumentation for all gcc supported languages. We have also an old version for Perl and other platforms and languages, which will be made available eventually.