Go back to all articles

Loading Testing with Citrix for Non-Standard Protocols

Feb 20, 2020
14 min read

Load testing is necessary when deploying any new system or redistributing data streams among existing systems. I think everybody knows this or at least would guess this is the case.
Everything seems simple: grab your load testing tool, record some traffic, get some scripts, run your tests, and enjoy the process. At least that’s the theory…

Approaches to Loading

Actually, in most cases engineers are faced with testing information systems that have a classic three-tiered architecture:

1 three tiered architecture

The standard approach to performance testing is to emulate user traffic from clients to the back end. This works, but only if the end client applications use standard protocols such as HTTP/HTTPS, SOAP, ODBC/JDBC, SAP GUI, and a few others.

2 standart approach to performance testing

But what do you do if the exchange between client and server uses a closed protocol or, even worse, a proprietary protocol, and the traffic cannot be recorded using standard tools?

You can try to figure out the protocol, decompile the application, reverse engineer it, and develop a plug-in or sampler based on the data received. But the likelihood of success is extremely small. And if you also consider the time required and translate that to money, then the entire undertaking is very difficult to justify.

In other words, the task is far from trivial and not always solvable. In fact, this is akin to hacking. It’s like a TV show where the players being told the story before being asked what’s in the black box. But you may not even have access to the story (read: source code for the application’s client end).

Of course, you can use various traffic analyzers (Wireshark, TcpDump, and others), but the volume of traffic may be so large that you’ll need months to analyze it!

In our experience, it is not uncommon situation when a customer starts to think about load testing only one week before a scheduled release.

Of course, you still have to write a wrapper so it works correctly with the chosen loading tool.
Well, we can only salute those who choose this path and succeed. Your results will be truly grand if you can overcome all of the “stumbling blocks”, but we in general we do not recommend this approach.
What can be done if you aren’t willing to expend so many resources on such lengthy investigations?
At this point, an experienced load tester will, of course, say, “Hey, guys, try CITRIX!”

Using Load Testing Citrix Virtualization Technologies

One way to handle this task is to use Citrix virtualization products, which may it possible to gain remote access to an application on almost any device. What’s at the heart of Citrix performance testing?

3 load testing with

The client application that end users interact with is published on a dedicated Citrix server (farm).
Unlike the first approach where the loading tool is “inserted” into the client-server link and parametrized traffic is used, here our tool sends commands to copies of the client application using the ICA protocol.

To tell the truth, we actually manage the applications through a terminal server.

This lets us emulate not the behavior of the client application through replaying traffic, but rather the behavior of a large number of users of the client application. It’s no secret that Citrix has been used in performance testing protocols long enough to emulate user behavior on a client application. But by no means can every loading tool work with this protocol.

Loading Tools

The most popular Citrix load testing tool for working is LoadRunner. It’s popularity is a result of the fact that there is no other choice. Looking at JMeter, we see nothing, empty space, anguish, pain, and somebody’s forgotten broken crutches on an overgrown path through a forest that leads who knows where.

Let’s returning to use Citrix protocol in Load Runner. This product recommends itself as massive and functional. However, for all of its pluses there are minuses.

The most obvious is the product’s cost and complexity when expanding functionality.

Anyway, we’ve chosen a tool, so everything ought to work out, but now we count up the cost of the licenses. In current market conditions, when converting dollar prices to rubles, the cost of licenses for testing may exceed the value of the solution being tested.

What’s more, the closest analog, Apache JMeter, is entirely free and easy to extend, thanks to open source code and a simple, understandable API for writing plugins.So what am I driving at? Yes, Load Runner is good, but our customers’ desire for a more budget-friendly tool prompted us to develop our own JMeter plugin to link with CITRIX, OCR, and synchronization.

Our plugin work produced a highly versatile and easy-to-use tool for working with Citrix protocol, which almost entirely covers LoadRunner’s functionality when working with this protocol, and in some instances surpasses it. But the most important advantage for our customers is the fact that it is free.

Differences from LoadRunner-based Solutions

If you’ve ever written terminal scripts for LR, then you are familiar with the agony of debugging, which requires the synchronization of every little thing, every image, and where being off by a couple of pixels can break everything. Tedious, slow, unreliable, devilishly torturous – these are the epithets you want to cry out during coding and debugging. The pain and suffering literally permeate you when you have to support this model.

Knowing this, when developing the plugin to work with Citrix we placed great emphasis on achieving the following benefits:

  • flexibility and easy of use similar to LoadRunner;
  • automation mechanisms to save time when implementing certain difficult algorithms directly in load scenarios;
  • integrated optical character recognition (OCR), which gives the virtual user the ability to respond to any text that appears on the user’s screen and, as a result, makes it possible to handle more complex test scenarios.
And finally
The most important difference is that it is free of charge for our customers. Using JMeter instead of LoadRunner makes considerable savings possible, especially during lengthy testing, because using LoadRunner like a citrix testing tool requires the purchase of licenses whose cost is based on the number of virtual users and the duration of testing.

Features

  • Record user actions (keyboard input, mouse clicks and movement) with the ability to choose the required input sources.
  • Playback recorded scripts.
  • Start sessions using *.ica files.
  • Start sessions with selected parameters (size of the Citrix workspace, color depth).
  • Specify whether to save screenshots made during recording and playback.
  • Parametrize any command/parameter in samplers.
  • Specify a multiplier for all ThinkTimes.
  • Start sessions without blocking user input.
  • Configure extensive synchronization settings.
  • Specify a (theoretically) unlimited number of checksums for synchronization.
  • Recognize text with the integrated OCR (Optical Character Recognition) engine.
  • Configure whether the tool starts a new session with each iteration.
  • Enjoy Remote Support.

And now we’ll briefly explain how it all works.
Anybody not directly associated with load testing can skip this section in order to avoid brain trauma.

Preparing JMeter

What you need to get started

You’ll need the following in order for the plugin to work properly:

  • JMeter for the plugin to run;
  • The com4j library for working with COM (required by the plugin; you can download it here);
  • JRE x86 v7+ for COM4J and the plugin to work properly (compiling with JRE v6 is also possible);
  • Citrix Receiver, used for recording and playback.

Installing the plugin

I won’t describe how to install JMeter and JRE for the Citrix load testing. I assume everybody is already familiar with these procedures. Installing Citrix Receiver is also straightforward, so we’ll get right to installation of the plugin. To install the plugin itself, simply copy the plugin’s .jar files and com4j library files to the lib folder in JMeter’s root directory. But to be able to record and playback you also need to add a registry entry (the path given is for Windows x64; for 32-bit versions, remove the Wow6432Node subdirectory from the path):

4 windows registry editor

This text can be saved in a text file with the .reg extension and then added to the registry by executing the file.

5 Citrix ICA recorder

To record scripts, you must add the Citrix ICA Recorder element (located in the Non-Test Elements category) to WorkBench and there indicate the correct connection parameters:

  • ICA file location
    The path to the .ica file for initializing the session. If an ICA file is specified, the remaining connection parameters will be ignored. You can use the GUI to choose the file by clicking Browse;
  • Host
    Address of the server;
  • Port
    Connection port. If no port is specified, the standard port will be used;
  • User Name
    The user name of a user authorized to access the published application;
  • User Password
    The user’s password;
  • Domain
    The user’s domain. Optional;
  • Application
    The name of the published application

The recording parameters must also be specified:

  • Save screenshots
    If enabled, then screenshots saved during recording will be saved;
  • Screenshot folder
    The path to the folder where the screenshots will be saved. The Browse lets you use the GUI to choose the path;
  • Resolution (WxH)
    Indicates the dimensions of the application’s Citrix workspace (maximum window size). Specified as x, separated by a lowercase Latin “x”;
  • Color 24 bit
    Indicates which color depth to use;
  • Record
    Indicates which events to record and which to not record. Possible values: Keyboard – Record keyboard events, Mouse Clicks – Record mouse clicks, Mouse Move – Record movements of the mouse cursor;
  • Default ICA Config
    Indicates which ICA Config Element to use by default in recorded samplers (more details about this later);
  • Select controller
    Indicates which controller to put the recorded samplers in

Recording a session

Once all of the settings have been entered, you can connect to the a session and start recording. To start/stop recording, use the Recording controls button group in the recorder.

The Connect button simply connects to the session. This does not start recording. You can start recording later.

The Record button connects and immediately starts recording the selected actions.

The Stop button stops recording and disconnects from the session. The session itself (Citrix Receiver window) is not closed. Moreover, when the session is exited (for example, by closing the application) recording is stopped automatically based on the LogOut event.

The Step controls button group is for managing steps. It becomes available once recording has begun.

The Add Step button saves the current step and adds it to the selected controller. Additionally, the recording begins to be received by a new sampler.

The Restart Step button simply clears all actions from the current step (helpful if you make a mistake). The Start Tag button inserts a tag so that all commands recorded until the End Tag is pressed will be ignored and instead the text entered by the user will be inserted. Accordingly, the End Tag button inserts a tag that terminates the effect of the Start Tag command.

The Synchronize button creates a screenshot of the Citrix session’s entire workspace and brings up a dialog with synchronization settings, which I would like to discuss in more detail.

Synchronization

There are two 2 synchronization options available at present: screenshot-based synchronization and text-based synchronization (using OCR).

Screenshot-based synchronization

To synchronize using screenshots, select the synchronization area (red semitransparent rectangle) and specify the right parameters.

  • Wait sync
    Indicates whether to wait until an area with the indicated hash is found and how long (at most) to wait. The second parameter , try every, indicates how often to check. The parameters are specified in milliseconds using whole numbers.
  • Delta x, y
    Indicates the maximum delta along the x and y axes, respectively, to search for the area. This is helpful if the element used for synchronization may shift slightly. But it is utterly useless if the element changes its dimensions, because the screenshot’s checksum will be different.
  • Set sampler error if failed
    Indicates whether to make a note of a sampler that fails to find the specified area within the specified time. If this is not enabled, then the sampler will considered always successful and the only way to discern whether the area appeared is to check the value of the variable in the following item.
  • Write result in variable named
    Indicates the name of a variable used to record whether synchronization is successful. The value of the variable will be true if the area is found. Otherwise, it will be false.

This type of synchronization involves comparing checksums of the specified screenshot areas during recording and playback. As a result, changing the color of a single pixel during playback will change the screenshot’s checksum, which will cause the comparison to fail and the area will be considered missing.

6 screenshot based sync

Text-based synchronization

For text-based synchronization, you also must choose the area of the screen in which text will be recognized. You also need to specify two parameters: OCR config handle – A string indicated in the OCR Config test element in the OCR Handle field (more about this later), used to select the customized OCR engine to be used for text recognition; and OCR result variable – The name of a variable used to record the result. If nothing is recognized, the variable will be created and initialized with an empty string.

7 text based sync with OCR

Preparing for playback

Once the script has been recorded, you should have a test plan with a controller and Citrix ICA Samplers inside. At least one Citrix ICA Config element is required for playback.

Configuring playback

8 Citrix ICA config

A Citrix ICA config looks very similar to a Citrix ICA Recorder. It has exactly the same connection parameters, so there’s no point in describing them here. I’ll only mention that all fields in the connection configuration can be parametrized, and these values will be used when initializing the session, i.e. when executing the first Citrix ICA Sampler encountered. For example, don’t be afraid to use CSV Data Set Config to parametrize fields. The Citrix Config Handle field must be unique for all such configs. The samplers use it to determine which connection settings to use, and to connect to the session to continue the test.

The settings to playback sessions differ from those in the recorder. I’ll point out the differences:

  • New session each iteration
    If a value is specified, a new session will be created for each iteration. The old session, if it is not closed, will remain active but will not do anything.
  • Replay mode
    If a value is indicated, then all input to the started sessions will be blocked. Otherwise, the user can use the mouse/keyboard to enter data into the open application. Most likely needed for debugging.
  • Output mode
    The window display mode. If Normal, then the window will be displayed on the screen. If Windowless, then the window will not be displayed, which saves resources on load benches (not working in the current version; currently the Receiver ignores this setting).
  • Use sleep times
    If a value is indicated, all recorded or manually entered delays will be replayed. You can also indicate a multiplier used to increase wait times (only for SleepTime commands).
  • Unlock all sessions and Lock all sessions
    These parameters respectively unblock and block input for all active sessions, i.e. changes the Replay mode parameter to “nope”. They only works with sessions started on an active JMeter. They do not work for sessions started on remote servers.

Configuring OCR

8 Citrix ICA config

If text recognition actions were recorded during recording, then an OCR Config element must also be added. It is quite simple and has only two parameters:

  • OCR handle
    This parameter differs from Citrix ICA Handle only in that it identifies an OCR Config rather than a Citrix ICA Config. It must be unique for all OCR Configs.
  • Table
    The left column indicates the path to an image used for training. The right column indicates the range of symbols in the image (specified as – without the angled brackets and separated by a hyphen).

The training images are images with sequentially printed symbols (in a line or in a table). The range of codes for the specified symbols must be uninterrupted and in increasing order. A single range should be indicated for each image.

Description of samplers 

8 Citrix ICA config

In addition to the standard names and comments, the Citrix ICA Sampler has two fields. First,  Citrix Config Handle  indicates which settings to use to start a session or the pool from which to take an existing active session. The second field contains a list of all the commands it will execute. This field may be fully parametrized using JMeter variables.

Script playback 

To playback a recorded and parametrized script, simply run the test. After each sampler has run, you can view the result of each command in Response Data.

Tips

If an .ica file is used to initialize sessions, it’s best to parametrize the field that corresponds to it. Because the file is single-use, the second stream (iteration) cannot be started using the same file. To solve this problem, you can get these files immediately before starting a session, e.g. using a web protocol.
If a sampler is unable to gain control of a session within 120 seconds, it will attempt to logout and disconnect the session for another 60 seconds. Then another attempt is connect will be made automatically.

If in rare instances an error occurs during screenshot-based synchronization, don’t worry. Try adding sampler the checksum obtained during playback (you can view the checksum on the Response Data tab in the View Results Tree, if the error occurred during synchronization).

If one of several actions is required based on what is displayed on the screen, you can use synchronization without a timeout and conditional controllers in the cycle in order to execute the desired action.

When using Citrix Receiver v4.4+, you cannot start a session with “direct” connection parameters. Only through .ica files.

Advantages of Using Citrix during Performance Testing

Using Citrix technologies in performance testing has its advantages over traditional approaches.

Benefits of using Citrix:

During this type of testing, interaction with the system takes place through the client application. Data sent from the client to the server are not emulated, but rather the user’s actions on the client. As a result, there is no need to analyze the details of the client’s interaction with the system. And this increases information security, makes it possible to test systems with an extremely complex/closed protocol, and saves time on development of test scenarios. This approach also provides the ability to assess the client application’s perfomance (for example, the time require to render information, the time required to write data to disk, etc.). What’s more, you can test the scalability of the Citrix farm itself.

Deficiencies of performance testing using Citrix:

The main deficiency is that to achieve universality you have to pay for load bench capacity and a Citrix farm. This technology works well when you have hundreds or thousands of users, but when there are tens of thousands or hundreds of thousands — alas, you have to think of something on the protocol level during Citrix load testing software. (We’ll tell you about this another time.)

Have a Project in Mind?​
We have been working on performance testing projects since 2008.
Drop us a line to find out what our team can do for you.
Get a quote You’ll hear back from our tech account manager in one day if not sooner

Summary

Choosing a performance testing tool is crucial when creating a testing strategy. The individual specifics of the system being tested, the protocols being used, and the load on the system are highly important.
If a testing project involves Citrix, then we can boldly state that we can perform the testing using our in-house tools just as well as LoadRunner, which can yield significant savings on testing costs.

This extension is a truly “fresh” solution, which was developed at the end of 2015 but has already proven itself well in performance testing at a major bank.

A comparison with other solutions revealed no material disadvantages, so if the difference is trivial, why pay more?

Table of contents

Related insights in blog articles

Explore what we’ve learned from these experiences
10 min read

JMeter vs LoadRunner vs PFLB: Which Tool to Choose?

jmeter loadRunner or PFLB choosing the optimal performance testing tool
Nov 28, 2024

Overview Performance testing is essential for evaluating the response time, scalability, reliability, speed, and resource usage of applications and web services under their expected workloads. The software market currently offers a variety of performance testing tools, but two names that automatically come to mind are Apache JMeter and LoadRunner. The JMeter and LoadRunner difference is […]

8 min read

10 Top Performance Testing Companies: Detailed Overview

best performance testing companies
Nov 26, 2024

Did you know the software testing market is already worth $51.8 billion, and it’s set to almost double by 2032, reaching $97.3 billion?  With so many platforms and companies springing up, how do you choose the best performance testing company for your business? That’s exactly what we’re here to help you with.  In this article, […]

14 min read

TOP 10 Best Online Load Testing Tools for 2024

best online load testing tools preview
Nov 7, 2024

In this article, we will go through our favourite features of each of these cloud-based load testing tools, while in the end you will find a parameterized comparison of all of them in one table.

10 min read

Essential Guide to ITSM Change Management: Processes, Benefits, and Tips

Essential Guide to ITSM Change Management
Oct 15, 2024

ITSM change management is essential for managing and implementing IT changes smoothly. It focuses on minimizing risks and aligning changes with business goals. In this guide, we’ll explore what ITSM change management entails, discuss its benefits, and provide practical tips for implementation. Key Takeaways What is ITSM Change Management? ITSM change management is a key […]

  • Be the first one to know

    We’ll send you a monthly e-mail with all the useful insights that we will have found and analyzed