Loading Testing with Citrix for Non-Standard Protocols

Load testing is necessary when deploying any new system or redistributing data streams among existing systems. I think everybody knows this or at least would guess this is the case.
Everything seems simple: grab your load testing tool, record some traffic, get some scripts, run your tests, and enjoy the process. At least that’s the theory…

Approaches to Loading

Actually, in most cases engineers are faced with testing information systems that have a classic three-tiered architecture:

The standard approach to performance testing is to emulate user traffic from clients to the back end. This works, but only if the end client applications use standard protocols such as HTTP/HTTPS, SOAP, ODBC/JDBC, SAP GUI, and a few others.

But what do you do if the exchange between client and server uses a closed protocol or, even worse, a proprietary protocol, and the traffic cannot be recorded using standard tools?

You can try to figure out the protocol, decompile the application, reverse engineer it, and develop a plug-in or sampler based on the data received. But the likelihood of success is extremely small. And if you also consider the time required and translate that to money, then the entire undertaking is very difficult to justify.

In other words, the task is far from trivial and not always solvable. In fact, this is akin to hacking. It’s like a TV show where the players being told the story before being asked what’s in the black box. But you may not even have access to the story (read: source code for the application’s client end).

Of course, you can use various traffic analyzers (Wireshark, TcpDump, and others), but the volume of traffic may be so large that you’ll need months to analyze it!

In our experience, it is not uncommon situation when a customer starts to think about load testing only one week before a scheduled release.

Of course, you still have to write a wrapper so it works correctly with the chosen loading tool.
Well, we can only salute those who choose this path and succeed. Your results will be truly grand if you can overcome all of the “stumbling blocks”, but we in general we do not recommend this approach.
What can be done if you aren’t willing to expend so many resources on such lengthy investigations?
At this point, an experienced load tester will, of course, say, “Hey, guys, try CITRIX!”

Using Load Testing Citrix Virtualization Technologies

One way to handle this task is to use Citrix virtualization products, which may it possible to gain remote access to an application on almost any device. What’s at the heart of Citrix performance testing?

The client application that end users interact with is published on a dedicated Citrix server (farm).
Unlike the first approach where the loading tool is “inserted” into the client-server link and parametrized traffic is used, here our tool sends commands to copies of the client application using the ICA protocol.

To tell the truth, we actually manage the applications through a terminal server.

This lets us emulate not the behavior of the client application through replaying traffic, but rather the behavior of a large number of users of the client application. It’s no secret that Citrix has been used in performance testing protocols long enough to emulate user behavior on a client application. But by no means can every loading tool work with this protocol.

Loading Tools

The most popular Citrix load testing tool for working is LoadRunner. It’s popularity is a result of the fact that there is no other choice. Looking at JMeter, we see nothing, empty space, anguish, pain, and somebody’s forgotten broken crutches on an overgrown path through a forest that leads who knows where.

Let’s returning to use Citrix protocol in Load Runner. This product recommends itself as massive and functional. However, for all of its pluses there are minuses.

The most obvious is the product’s cost and complexity when expanding functionality.

Anyway, we’ve chosen a tool, so everything ought to work out, but now we count up the cost of the licenses. In current market conditions, when converting dollar prices to rubles, the cost of licenses for testing may exceed the value of the solution being tested.

What’s more, the closest analog, Apache JMeter, is entirely free and easy to extend, thanks to open source code and a simple, understandable API for writing plugins.So what am I driving at? Yes, Load Runner is good, but our customers’ desire for a more budget-friendly tool prompted us to develop our own JMeter plugin to link with CITRIX, OCR, and synchronization.

Our plugin work produced a highly versatile and easy-to-use tool for working with Citrix protocol, which almost entirely covers LoadRunner’s functionality when working with this protocol, and in some instances surpasses it. But the most important advantage for our customers is the fact that it is free.

Differences from LoadRunner-based Solutions

If you’ve ever written terminal scripts for LR, then you are familiar with the agony of debugging, which requires the synchronization of every little thing, every image, and where being off by a couple of pixels can break everything. Tedious, slow, unreliable, devilishly torturous – these are the epithets you want to cry out during coding and debugging. The pain and suffering literally permeate you when you have to support this model.

Knowing this, when developing the plugin to work with Citrix we placed great emphasis on achieving the following benefits:

• flexibility and easy of use similar to LoadRunner;
• automation mechanisms to save time when implementing certain difficult algorithms directly in load scenarios;
• integrated optical character recognition (OCR), which gives the virtual user the ability to respond to any text that appears on the user’s screen and, as a result, makes it possible to handle more complex test scenarios.

Features

• Record user actions (keyboard input, mouse clicks and movement) with the ability to choose the required input sources.
• Playback recorded scripts.
• Start sessions using *.ica files.
• Start sessions with selected parameters (size of the Citrix workspace, color depth).
• Specify whether to save screenshots made during recording and playback.
• Parametrize any command/parameter in samplers.
• Specify a multiplier for all ThinkTimes.
• Start sessions without blocking user input.
• Configure extensive synchronization settings.
• Specify a (theoretically) unlimited number of checksums for synchronization.
• Recognize text with the integrated OCR (Optical Character Recognition) engine.
• Configure whether the tool starts a new session with each iteration.
• Enjoy Remote Support.

And now we’ll briefly explain how it all works.
Anybody not directly associated with load testing can skip this section in order to avoid brain trauma.

Preparing JMeter

What you need to get started

You’ll need the following in order for the plugin to work properly:

• JMeter for the plugin to run;
• The com4j library for working with COM (required by the plugin; you can download it here);
• JRE x86 v7+ for COM4J and the plugin to work properly (compiling with JRE v6 is also possible);
• Citrix Receiver, used for recording and playback.

Installing the plugin

I won’t describe how to install JMeter and JRE for the Citrix load testing. I assume everybody is already familiar with these procedures. Installing Citrix Receiver is also straightforward, so we’ll get right to installation of the plugin. To install the plugin itself, simply copy the plugin’s .jar files and com4j library files to the lib folder in JMeter’s root directory. But to be able to record and playback you also need to add a registry entry (the path given is for Windows x64; for 32-bit versions, remove the Wow6432Node subdirectory from the path):

This text can be saved in a text file with the .reg extension and then added to the registry by executing the file.

To record scripts, you must add the Citrix ICA Recorder element (located in the Non-Test Elements category) to WorkBench and there indicate the correct connection parameters:

The recording parameters must also be specified:

Recording a session

Once all of the settings have been entered, you can connect to the a session and start recording. To start/stop recording, use the Recording controls button group in the recorder.

The Connect button simply connects to the session. This does not start recording. You can start recording later.

The Record button connects and immediately starts recording the selected actions.

The Stop button stops recording and disconnects from the session. The session itself (Citrix Receiver window) is not closed. Moreover, when the session is exited (for example, by closing the application) recording is stopped automatically based on the LogOut event.

The Step controls button group is for managing steps. It becomes available once recording has begun.

The Add Step button saves the current step and adds it to the selected controller. Additionally, the recording begins to be received by a new sampler.

The Restart Step button simply clears all actions from the current step (helpful if you make a mistake). The Start Tag button inserts a tag so that all commands recorded until the End Tag is pressed will be ignored and instead the text entered by the user will be inserted. Accordingly, the End Tag button inserts a tag that terminates the effect of the Start Tag command.

The Synchronize button creates a screenshot of the Citrix session’s entire workspace and brings up a dialog with synchronization settings, which I would like to discuss in more detail.

Synchronization

There are two 2 synchronization options available at present: screenshot-based synchronization and text-based synchronization (using OCR).

Screenshot-based synchronization

To synchronize using screenshots, select the synchronization area (red semitransparent rectangle) and specify the right parameters.

This type of synchronization involves comparing checksums of the specified screenshot areas during recording and playback. As a result, changing the color of a single pixel during playback will change the screenshot’s checksum, which will cause the comparison to fail and the area will be considered missing.

Text-based synchronization

For text-based synchronization, you also must choose the area of the screen in which text will be recognized. You also need to specify two parameters: OCR config handle – A string indicated in the OCR Config test element in the OCR Handle field (more about this later), used to select the customized OCR engine to be used for text recognition; and OCR result variable – The name of a variable used to record the result. If nothing is recognized, the variable will be created and initialized with an empty string.

Preparing for playback

Once the script has been recorded, you should have a test plan with a controller and Citrix ICA Samplers inside. At least one Citrix ICA Config element is required for playback.

Configuring playback

A Citrix ICA config looks very similar to a Citrix ICA Recorder. It has exactly the same connection parameters, so there’s no point in describing them here. I’ll only mention that all fields in the connection configuration can be parametrized, and these values will be used when initializing the session, i.e. when executing the first Citrix ICA Sampler encountered. For example, don’t be afraid to use CSV Data Set Config to parametrize fields. The Citrix Config Handle field must be unique for all such configs. The samplers use it to determine which connection settings to use, and to connect to the session to continue the test.

The settings to playback sessions differ from those in the recorder. I’ll point out the differences:

Configuring OCR

If text recognition actions were recorded during recording, then an OCR Config element must also be added. It is quite simple and has only two parameters:

The training images are images with sequentially printed symbols (in a line or in a table). The range of codes for the specified symbols must be uninterrupted and in increasing order. A single range should be indicated for each image.

Description of samplers 

In addition to the standard names and comments, the Citrix ICA Sampler has two fields. First,  Citrix Config Handle  indicates which settings to use to start a session or the pool from which to take an existing active session. The second field contains a list of all the commands it will execute. This field may be fully parametrized using JMeter variables.

Script playback 

To playback a recorded and parametrized script, simply run the test. After each sampler has run, you can view the result of each command in Response Data.

Tips

If an .ica file is used to initialize sessions, it’s best to parametrize the field that corresponds to it. Because the file is single-use, the second stream (iteration) cannot be started using the same file. To solve this problem, you can get these files immediately before starting a session, e.g. using a web protocol.
If a sampler is unable to gain control of a session within 120 seconds, it will attempt to logout and disconnect the session for another 60 seconds. Then another attempt is connect will be made automatically.

If in rare instances an error occurs during screenshot-based synchronization, don’t worry. Try adding sampler the checksum obtained during playback (you can view the checksum on the Response Data tab in the View Results Tree, if the error occurred during synchronization).

If one of several actions is required based on what is displayed on the screen, you can use synchronization without a timeout and conditional controllers in the cycle in order to execute the desired action.

When using Citrix Receiver v4.4+, you cannot start a session with “direct” connection parameters. Only through .ica files.

Advantages of Using Citrix during Performance Testing

Using Citrix technologies in performance testing has its advantages over traditional approaches.

Benefits of using Citrix:

During this type of testing, interaction with the system takes place through the client application. Data sent from the client to the server are not emulated, but rather the user’s actions on the client. As a result, there is no need to analyze the details of the client’s interaction with the system. And this increases information security, makes it possible to test systems with an extremely complex/closed protocol, and saves time on development of test scenarios. This approach also provides the ability to assess the client application’s perfomance (for example, the time require to render information, the time required to write data to disk, etc.). What’s more, you can test the scalability of the Citrix farm itself.

Deficiencies of performance testing using Citrix:

The main deficiency is that to achieve universality you have to pay for load bench capacity and a Citrix farm. This technology works well when you have hundreds or thousands of users, but when there are tens of thousands or hundreds of thousands — alas, you have to think of something on the protocol level during Citrix load testing software. (We’ll tell you about this another time.)

Summary

Choosing a performance testing tool is crucial when creating a testing strategy. The individual specifics of the system being tested, the protocols being used, and the load on the system are highly important.
If a testing project involves Citrix, then we can boldly state that we can perform the testing using our in-house tools just as well as LoadRunner, which can yield significant savings on testing costs.

This extension is a truly “fresh” solution, which was developed at the end of 2015 but has already proven itself well in performance testing at a major bank.

A comparison with other solutions revealed no material disadvantages, so if the difference is trivial, why pay more?