Loading Testing with Citrix for Non-Standard Protocols
Load testing is necessary when deploying any new system or redistributing data streams among existing systems. I think everybody knows this or at least would guess this is the case.
Everything seems simple: grab your load testing tool, record some traffic, get some scripts, run your tests, and enjoy the process. At least that’s the theory…
Approaches to Loading
Actually, in most cases engineers are faced with testing information systems that have a classic three-tiered architecture:
The standard approach to performance testing is to emulate user traffic from clients to the back end. This works, but only if the end client applications use standard protocols such as HTTP/HTTPS, SOAP, ODBC/JDBC, SAP GUI, and a few others.
But what do you do if the exchange between client and server uses a closed protocol or, even worse, a proprietary protocol, and the traffic cannot be recorded using standard tools?
You can try to figure out the protocol, decompile the application, reverse engineer it, and develop a plug-in or sampler based on the data received. But the likelihood of success is extremely small. And if you also consider the time required and translate that to money, then the entire undertaking is very difficult to justify.
In other words, the task is far from trivial and not always solvable. In fact, this is akin to hacking. It’s like a TV show where the players being told the story before being asked what’s in the black box. But you may not even have access to the story (read: source code for the application’s client end).
Of course, you can use various traffic analyzers (Wireshark, TcpDump, and others), but the volume of traffic may be so large that you’ll need months to analyze it!
In our experience, it is not uncommon situation when a customer starts to think about load testing only one week before a scheduled release.
Of course, you still have to write a wrapper so it works correctly with the chosen loading tool.
Well, we can only salute those who choose this path and succeed. Your results will be truly grand if you can overcome all of the “stumbling blocks”, but we in general we do not recommend this approach.
What can be done if you aren’t willing to expend so many resources on such lengthy investigations?
At this point, an experienced load tester will, of course, say, “Hey, guys, try CITRIX!”
Using Load Testing Citrix Virtualization Technologies
One way to handle this task is to use Citrix virtualization products, which may it possible to gain remote access to an application on almost any device. What’s at the heart of Citrix performance testing?
The client application that end users interact with is published on a dedicated Citrix server (farm).
Unlike the first approach where the loading tool is “inserted” into the client-server link and parametrized traffic is used, here our tool sends commands to copies of the client application using the ICA protocol.
To tell the truth, we actually manage the applications through a terminal server.
This lets us emulate not the behavior of the client application through replaying traffic, but rather the behavior of a large number of users of the client application. It’s no secret that Citrix has been used in performance testing protocols long enough to emulate user behavior on a client application. But by no means can every loading tool work with this protocol.
Loading Tools
The most popular Citrix load testing tool for working is LoadRunner. It’s popularity is a result of the fact that there is no other choice. Looking at JMeter, we see nothing, empty space, anguish, pain, and somebody’s forgotten broken crutches on an overgrown path through a forest that leads who knows where.
Let’s returning to use Citrix protocol in Load Runner. This product recommends itself as massive and functional. However, for all of its pluses there are minuses.
The most obvious is the product’s cost and complexity when expanding functionality.
Anyway, we’ve chosen a tool, so everything ought to work out, but now we count up the cost of the licenses. In current market conditions, when converting dollar prices to rubles, the cost of licenses for testing may exceed the value of the solution being tested.
What’s more, the closest analog, Apache JMeter, is entirely free and easy to extend, thanks to open source code and a simple, understandable API for writing plugins.So what am I driving at? Yes, Load Runner is good, but our customers’ desire for a more budget-friendly tool prompted us to develop our own JMeter plugin to link with CITRIX, OCR, and synchronization.
Our plugin work produced a highly versatile and easy-to-use tool for working with Citrix protocol, which almost entirely covers LoadRunner’s functionality when working with this protocol, and in some instances surpasses it. But the most important advantage for our customers is the fact that it is free.
Differences from LoadRunner-based Solutions
If you’ve ever written terminal scripts for LR, then you are familiar with the agony of debugging, which requires the synchronization of every little thing, every image, and where being off by a couple of pixels can break everything. Tedious, slow, unreliable, devilishly torturous – these are the epithets you want to cry out during coding and debugging. The pain and suffering literally permeate you when you have to support this model.
Knowing this, when developing the plugin to work with Citrix we placed great emphasis on achieving the following benefits:
Features
And now we’ll briefly explain how it all works.
Anybody not directly associated with load testing can skip this section in order to avoid brain trauma.
Preparing JMeter
What you need to get started
You’ll need the following in order for the plugin to work properly:
Installing the plugin
I won’t describe how to install JMeter and JRE for the Citrix load testing. I assume everybody is already familiar with these procedures. Installing Citrix Receiver is also straightforward, so we’ll get right to installation of the plugin. To install the plugin itself, simply copy the plugin’s .jar files and com4j library files to the lib folder in JMeter’s root directory. But to be able to record and playback you also need to add a registry entry (the path given is for Windows x64; for 32-bit versions, remove the Wow6432Node subdirectory from the path):
This text can be saved in a text file with the .reg extension and then added to the registry by executing the file.
To record scripts, you must add the Citrix ICA Recorder element (located in the Non-Test Elements category) to WorkBench and there indicate the correct connection parameters:
The recording parameters must also be specified:
Recording a session
Once all of the settings have been entered, you can connect to the a session and start recording. To start/stop recording, use the Recording controls button group in the recorder.
The Connect button simply connects to the session. This does not start recording. You can start recording later.
The Record button connects and immediately starts recording the selected actions.
The Stop button stops recording and disconnects from the session. The session itself (Citrix Receiver window) is not closed. Moreover, when the session is exited (for example, by closing the application) recording is stopped automatically based on the LogOut event.
The Step controls button group is for managing steps. It becomes available once recording has begun.
The Add Step button saves the current step and adds it to the selected controller. Additionally, the recording begins to be received by a new sampler.
The Restart Step button simply clears all actions from the current step (helpful if you make a mistake). The Start Tag button inserts a tag so that all commands recorded until the End Tag is pressed will be ignored and instead the text entered by the user will be inserted. Accordingly, the End Tag button inserts a tag that terminates the effect of the Start Tag command.
The Synchronize button creates a screenshot of the Citrix session’s entire workspace and brings up a dialog with synchronization settings, which I would like to discuss in more detail.
Synchronization
There are two 2 synchronization options available at present: screenshot-based synchronization and text-based synchronization (using OCR).
Screenshot-based synchronization
To synchronize using screenshots, select the synchronization area (red semitransparent rectangle) and specify the right parameters.
This type of synchronization involves comparing checksums of the specified screenshot areas during recording and playback. As a result, changing the color of a single pixel during playback will change the screenshot’s checksum, which will cause the comparison to fail and the area will be considered missing.
Text-based synchronization
For text-based synchronization, you also must choose the area of the screen in which text will be recognized. You also need to specify two parameters: OCR config handle – A string indicated in the OCR Config test element in the OCR Handle field (more about this later), used to select the customized OCR engine to be used for text recognition; and OCR result variable – The name of a variable used to record the result. If nothing is recognized, the variable will be created and initialized with an empty string.
Preparing for playback
Once the script has been recorded, you should have a test plan with a controller and Citrix ICA Samplers inside. At least one Citrix ICA Config element is required for playback.
Configuring playback
A Citrix ICA config looks very similar to a Citrix ICA Recorder. It has exactly the same connection parameters, so there’s no point in describing them here. I’ll only mention that all fields in the connection configuration can be parametrized, and these values will be used when initializing the session, i.e. when executing the first Citrix ICA Sampler encountered. For example, don’t be afraid to use CSV Data Set Config to parametrize fields. The Citrix Config Handle field must be unique for all such configs. The samplers use it to determine which connection settings to use, and to connect to the session to continue the test.
The settings to playback sessions differ from those in the recorder. I’ll point out the differences:
Configuring OCR
If text recognition actions were recorded during recording, then an OCR Config element must also be added. It is quite simple and has only two parameters:
The training images are images with sequentially printed symbols (in a line or in a table). The range of codes for the specified symbols must be uninterrupted and in increasing order. A single range should be indicated for each image.
Description of samplers
In addition to the standard names and comments, the Citrix ICA Sampler has two fields. First, Citrix Config Handle indicates which settings to use to start a session or the pool from which to take an existing active session. The second field contains a list of all the commands it will execute. This field may be fully parametrized using JMeter variables.
Script playback
To playback a recorded and parametrized script, simply run the test. After each sampler has run, you can view the result of each command in Response Data.
Tips
If an .ica file is used to initialize sessions, it’s best to parametrize the field that corresponds to it. Because the file is single-use, the second stream (iteration) cannot be started using the same file. To solve this problem, you can get these files immediately before starting a session, e.g. using a web protocol.
If a sampler is unable to gain control of a session within 120 seconds, it will attempt to logout and disconnect the session for another 60 seconds. Then another attempt is connect will be made automatically.
If in rare instances an error occurs during screenshot-based synchronization, don’t worry. Try adding sampler the checksum obtained during playback (you can view the checksum on the Response Data tab in the View Results Tree, if the error occurred during synchronization).
If one of several actions is required based on what is displayed on the screen, you can use synchronization without a timeout and conditional controllers in the cycle in order to execute the desired action.
When using Citrix Receiver v4.4+, you cannot start a session with “direct” connection parameters. Only through .ica files.
Advantages of Using Citrix during Performance Testing
Using Citrix technologies in performance testing has its advantages over traditional approaches.
Benefits of using Citrix:
During this type of testing, interaction with the system takes place through the client application. Data sent from the client to the server are not emulated, but rather the user’s actions on the client. As a result, there is no need to analyze the details of the client’s interaction with the system. And this increases information security, makes it possible to test systems with an extremely complex/closed protocol, and saves time on development of test scenarios. This approach also provides the ability to assess the client application’s perfomance (for example, the time require to render information, the time required to write data to disk, etc.). What’s more, you can test the scalability of the Citrix farm itself.
Deficiencies of performance testing using Citrix:
The main deficiency is that to achieve universality you have to pay for load bench capacity and a Citrix farm. This technology works well when you have hundreds or thousands of users, but when there are tens of thousands or hundreds of thousands — alas, you have to think of something on the protocol level during Citrix load testing software. (We’ll tell you about this another time.)
Summary
Choosing a performance testing tool is crucial when creating a testing strategy. The individual specifics of the system being tested, the protocols being used, and the load on the system are highly important.
If a testing project involves Citrix, then we can boldly state that we can perform the testing using our in-house tools just as well as LoadRunner, which can yield significant savings on testing costs.
This extension is a truly “fresh” solution, which was developed at the end of 2015 but has already proven itself well in performance testing at a major bank.
A comparison with other solutions revealed no material disadvantages, so if the difference is trivial, why pay more?
Related insights in blog articles
JMeter vs LoadRunner vs PFLB: Which Tool to Choose?
Overview Performance testing is essential for evaluating the response time, scalability, reliability, speed, and resource usage of applications and web services under their expected workloads. The software market currently offers a variety of performance testing tools, but two names that automatically come to mind are Apache JMeter and LoadRunner. The JMeter and LoadRunner difference is […]
10 Top Performance Testing Companies: Detailed Overview
Did you know the software testing market is already worth $51.8 billion, and it’s set to almost double by 2032, reaching $97.3 billion? With so many platforms and companies springing up, how do you choose the best performance testing company for your business? That’s exactly what we’re here to help you with. In this article, […]
TOP 10 Best Online Load Testing Tools for 2024
In this article, we will go through our favourite features of each of these cloud-based load testing tools, while in the end you will find a parameterized comparison of all of them in one table.
Essential Guide to ITSM Change Management: Processes, Benefits, and Tips
ITSM change management is essential for managing and implementing IT changes smoothly. It focuses on minimizing risks and aligning changes with business goals. In this guide, we’ll explore what ITSM change management entails, discuss its benefits, and provide practical tips for implementation. Key Takeaways What is ITSM Change Management? ITSM change management is a key […]
Be the first one to know
We’ll send you a monthly e-mail with all the useful insights that we will have found and analyzed
People love to read
Explore the most popular articles we’ve written so far
- TOP 10 Best Online Load Testing Tools for 2024 Nov 7, 2024
- Benefits of Performance Testing for Businesses Sep 4, 2024
- Android vs iOS App Performance Testing: What’s the Difference? Dec 9, 2022
- How to Save Money on Performance Testing? Dec 5, 2022
- Cloud-based Application Testing: Features & Types Apr 15, 2020