Engineering Chronon

All the technical and design details on Chronon - The DVR for Java

Inside Chronon 3 Recorder: How we achieved the magical speed boost

We released Chronon 3 last week with an order of magnitude performance improvement. Let’s take a look under the hood to see what makes it all tick!

The Recipe

Look at the source code of any computer program. You can pretty much predict what it is going to do, given you had some extra info like what branch will be taken and/or what the return value of a particular method call is.

In the code below, if you knew the value of 'a', you could predict whether 'even' or 'odd' will be printed.

Chronon 3's recorder follows a similar principal. It instruments the bytecode of you program as its loaded and does static analysis on it to come up with an initial set of 'predictions' as to how your program will behave.

Image

Once your program is running, it keeps monitoring it against the initial set of predictions. If the execution at any time differs from what was 'predicted':

1. The prediction data structure is modified to take into account the new information.

Image_1

 

2. Recording data is generated to select the right 'prediction' from the structure. This usually takes only 1 to 4 bits.

3. If the prediction matches accurately though then no data is generated at all!

Image_2

See it in Action

For example, consider the code below:

Assume there are 2 threads that call the method process().

The first thread has an object of type Foo and calls Foo.bar(), which matches the prediction made by the recorder exactly, so no data is generated at that point for that thread.

The second thread calls has an object of FooSub instead, and thus ends up calling FooSub.bar().

Now here is where things get interesting. For this thread only, the prediction structure is adjusted to account for this new method call. Then a single bit is stored to select the right method call from the prediction structure. The previous thread, on the other hand, which keeps Foo.bar() keeps generating no data as its prediction is accurately matched.

Results

As you can see that the new prediction mechanism helps in saving a ton of data that would otherwise have been generated and even when data is indeed generated, it’s only very minimal. Apart from the example given above, we have a whole bunch of predictions inside the Chronon 3 Recorder. We now even have a pluggable architecture where we can keep plugging in more of these predictions and constantly increasing the speed of the recorder, without needing to wait an entire year.  

So far the current architecture has worked extremely well and almost all our previous users running into OutOfMemory issues are now able to record easily!

Filed under  //     

See how Nokia uses Chronon

We sat down with Harikishore Tadigotla of Nokia to see how Nokia uses a combination of Chronon Recording Server and Chronon Time Travelling Debugger to diagnose their SMS processing issues.

>> Read full Nokia Case Study


Some tidbits from the case study:

Challenges:

“Before Chronon, we used to take a huge amount of time in reproducing production bugs. Some bugs were simply impossible to reproduce since our local environments were never exact replicas of production environment.”

Solution

“Reproducing the bugs in our local environments was usually a very huge and cumbersome process as it involved a long and tedious process to redeploy server builds onto local environments before we got to verify them. Verifying local environments involves running simulations of SMS requests, which was not very accurate. Using Chronon we could switch to the payload of accurate SMS requests, which increased the accuracy of tests run in local environments.”

Results

“The primary benefit is the ease with which bugs can be reproduced ‘offline’. Another major benefit we found with using Chronon is it offers an understanding the existing codebase in a large team. This was found to be particularly helpful in our team since we had a huge codebase and only few developers had familiarity to major portions of our codebase.”

 

To get the full details, read the entire Nokia Case Study.

 

Bugfix update 3.0.8

For those seeing NullPointerExceptions in the Debugger with the Chronon 3 udpate, this should fix that.

Also since due to 'per thread time', we run each thread truly independently, if you view the value of a variable that was created in a different thread, chronon will try to guess its value. >95% of the time the value will be correct, but even then the debugger marks such values as <<fuzzy>>.

Fuzzy_values

Filed under  //     

The Amazing Chronon 3

Its been a long time coming, but Chronon 3 is finally here! 

In the next few posts, we will dive deep into what took us so long and all the technical magic that makes Chronon 3 tick. For now lets see what Chronon 3 brings us:

Features

  • Enormous performance upgrade
    C3-memusage

The number one issues of Chronon users has been running into OutOfMemoryErrors and all of Chronon 3 is dedicated to eradicating that. The all new, rewritten from scratch Recorder is built to conserve memory. It uses off heap memory as much as possible and even then limits the amount of total data collected by the Recorder to be many times lower than that of previous versions. This graph gives you an idea of the improvement.

  • 5x faster unpacking

The Unpacker is much faster too! Apart from consuming much, much less resources, it also runs 5x faster!

  • Per Thread Time

With Chronon 3, we are introducing Per Thread Time, which gives each thread its own timeline, facilitating in must easier and faster debugging. Now the timeline view shows you the time and progess of the current thread instead of the whole system, giving you much more focused data. This is carried on in other views too, like the Stack view shows the current thread only instead of cluttering the view with all the threads in the system. The result of this is much more focused data and it also allows us to make the recorder more concurrent and faster.

  • On Demand Split

In case you missed it, a few months back we also introduced On Demand Split in the Recording Server, allowing you to split recordings anytime, without having to wait for a split interval.

  • Support for JBoss

Previously, Jboss 7 was having issues with Chronon due to OSGI classloader issues. With Chronon 3, this should be solved and you should be able to fully record your JBoss apps.

Future Updates

Now that we are done with this huge release, we will be doing much, much faster releases to keep bringing you new features. Due to time constraints for this release, we have decided not to include the 'Recorded Console' view. We think with Post Execution Logging you can achieve the same goal without cluttering your code with println() statements. However, if there is demand for it, we will bring the view back in the next update.

Renewed Trial

If you don't own a Chronon subscription and your trial has expired, dont worry! You'll get a fully renewed trial now when you download Chronon 3. Same goes for students. If your student license has expired, request your student license again and you will get a renewed one too.

Download Now

Stop waiting and go ahead and give the Chronon 3 Debugger and Recording Server  a try now!

Filed under  //         

Embedded Chronon documentation publicly available

We have made the documentation for Embedded Chronon publicly available now, including full Javadoc for the Embedded Chronon APIs.

Previously this was available only after you requested a trial.

http://chronon.onconfluence.com/display/DOC/Embedded+Chronon

This way, if you want to see for yourself how easy it is to use Embedded Chronon before you request a trial for it, now you can!

Filed under  //   

Chronon 2.5.4 released

This is our final feature update for Chronon 2 before the Chronon 3 release. The update is mainly to the Chronon Debugger and contains some bugfixes and polish of existing features:

Keyboard shortcuts in Stack View

Keyboard-shortcuts
The Stack View now shows the keyboard shortcut corresponding to each action when you hover over any of the buttons. No need to memorize the shortcuts anymore!

Chronon 3 recording warning

C3warning

If you try to open a Chronon 3 recording in Chronon 2, instead of just a random crash, an appropriate error message is displayed.

More polished Select Value dialog in Method History View

Selectvaluedialog

When you select to filter by 'Local' expressions in Method History view, the 'Select Value' dialog now enables/disables controls as you select them and does validation on each field to make sure you enter a correct value.

Next up... Chronon 3

As mentioned previously, next post onwards we will start talking exclusively about Chronon 3. Why we got delayed? What have we been upto? How does Chronon 3 get its insane speed?

We know you have many questions. Take the red pill and come with us down the rabbit hole. The answers are coming!

Filed under  //     

'Per Thread Time' and updated Timeline View in Chronon 3

While we get the final Chronon 3 build ready, lets talk about some of the UI stuff in Chronon 3. Although this release is mainly focused on insane performance improvements, we have taken some time to improve the Chronon Debugger too.

Per Thread Time

Up till Chronon 2, we only had a single global timeline to represent all events.

With Chronon 3, we are introducing 'Per Thread Time', which gives each thread its own timeline, facilitating in must easier and faster debugging.

Timeline View

The Timeline View has always been one of the most critical views in the Chronon Debugger UI. It does the fundamental job of telling you whether you just jumped forward/backward in the execution of your program. Without it, you would be 'lost in time'. The Timeline view also allows to save Time Bookmarks.

Time Progress Bars

Newtimelineview

The biggest UI change you will notice is that instead of having a single bar showing progress through time, we have 2 bars.

One shows 'global time', which tells you how far you are down the execution of the entire program. This is conceptually similar to the single bar that used to be present in Chronon 2.

The bar below that shows you how far you are down the execution of the current thread.

Time Bookmarks

Time Bookmarks now save time as 'current thread time' instead of 'global time'

Conclusion

Per Thread Time is a feature that many users have wanted and is extremely useful.

A lot of times, in a large recording, you can have many, many threads that were created and destroyed throughout the course of the program, and you really want to know whether the code you are debugging in the current thread, are you close to the end of that thread or in the middle. A good example is when debugging uncaught exceptions, since when you are near an uncaught exception, the 'current thread' bar will be almost full.

With that said, we are just packaging things up for the final release. Stay tuned to our twitter feed to hear about the release as soon as it is available! 

Filed under  //   

Morning with Control Flow Graphs

Please dont write methods that result in Control Flow Graphs (CFGs) like this (takes 5 seconds to load full image on windows image viewer on my 12 core machine with the latest SSD):

Execute

 

Filed under  //     

Chronon and Jubula join forces

Chronon Systems and BREDEX (the company behind Jubula), have joined forces to bring you the best of both testing and debugging worlds. Official press release here.

If you dont already know about Jubula, it is a wildly popular test automation tool and part of the Eclipse project. It forms the basis for 'Eclipse for Testers'.

Their is also a commercial version of Jubula called GUIdancer with some extra features. 

Both Jubula and GUIdancer now integrate with Chronon and Embedded Chronon

(Note: From this point on, instead of mentioning 'Jubula and GUIdancer' everywhere, I will just mention 'Jubula' to refer to both.)

How does the Chronon integration with Jubula work?

Tests that are created and executed in Jubula can now be recorded with the Chronon recorder, using a simple gui configuration.

The Standalone version of Jubula now ships with the Chronon recorder by default, enabling this integration.

This means you can record all your tests within Jubula and if any one of them fails, you can take the Chronon recording of it and debug it using the Chronon Time Travelling Debugger.

Can the Jubula-Chronon integration be used outside of the Jubula GUI

Note that although you can technically run your Jubula tests with Chronon enabled, outside the Jubula GUI, that use case is not recommended or supported by either Chronon Systems or BREDEX.

To record outside of the GUI (Jubula test environment), please look at our Chronon Recording Server and Embedded Chronon offerings, which are explicitly designed for and support such use cases.

Is the Chronon and Embedded integration available only on standalone version of Jubula?

Yes.

The 'Eclipse for Testers' release does not contain the Chronon recorder due to the rules of Eclipse foundation not allowing any non-EPL code in an Eclipse download.

Do I need to download the Chronon Time Travelling Debugger separately?

Yes.

Since the standalone version of Jubula doesnt contain Eclipse JDT, a dependency of the Chronon Debugger, you need to download eclipse separately and install the Chronon Debugger plugin in it. If you already have an eclipse installation, just add the Chronon Debugger plugin to it.

Also since the Time Travelling Debugger is a commercial product, after the 30 day trial, you can either:

How does the Embedded Chronon integration with Jubula work?

Jubula is one of the first open source projects to ship with Embedded Chronon.
This means you can record Jubula itself.

Embedded-chronon

Thus if you run into a bug with Jubula:

Instead of send a detailed bug report to the Jubula bug tracker with instruction on how Jubula developers can reproduce the bug on your box, you can just record the bug and attach the Chronon recording in a bug report at the Jubula bug tracker.

This also means Jubula devs don't need to go through all kinds of hoops to reproduce your bug on their side and can diagnose and fix the bug quickly!

Note that in the current version of Jubula, you have to modify the jubula.ini file to enable the Embedded Chronon before you can record Jubula with it. This is not something that is required by Embedded Chronon, but is due to some technicalities on the Jubula side which will be changed as a part of the the next Jubula release. For the moment, you just have to remember to comment two lines into the Jubula.ini to let you start the recording from the Jubula ITE.

So, what are you waiting for, go ahead and download Jubula with Chronon and speed up your development and testing processes!

Chronon 2.5 released

While we get Chronon 3 ready, we thought of making an interim release with some goodies our users have been asking for:

Chronon Recording Server

Recording Server is the highlight of this release, and contains the most voted feature from our users:

On Demand Split

Crs-splitrecording

Now when you encounter a bug while testing/QA, no longer do you have to wait for an 'automatic split interval' or do a 'Stop Recording' which results in a lengthy 'deinstrumentation' phase and then 'Start Recording' again (which will result in another lengthy 'instrumentation' phase).

Just click the new 'Split Recording' button and a recording is created instantly on disk and the recorder keeps running!

Here is a demo video demonstrating this feature:

Make sure you get the latest version of Recording Server now!

Chronon Time Travelling Debugger

Some enhancements based on user feedback:

Timeline View

Timelineview

  • An Id column has been added to allow easily referring to each time bookmark by an id, instead of a complex, lengthy 'time' value.
  • All columns are now sortable.

Exceptions View

Exceptionsview

  • An Id column has been added here too. 

Dont forget to update your debugger!

    Embedded Chronon and Chronon Recorder

    Plenty of bugfixes and minor performance enhancements in the Chronon Recorder for users of Embedded Chronon and Chronon Recording Server.

    If you were getting a corrupt recording when using either Embedded Chronon or Chronon Recording Server, these bugfixes should now prevent that.

    Also for the first time we have updated the native agents too, from version 1.0 to 1.1.0.

    Filed under  //