No Protocol Docs, No Recorder, No Paid License: Load Testing Microsoft Dynamics NAV Anyway

Sona Hakobyan

Author

Sona Hakobyan

Sona Hakobyan is a Senior Copywriter at PFLB. She writes and edits content for websites, blogs, and internal platforms. Sona participates in cross-functional content planning and production. Her experience includes work on international content teams and B2B communications.

Senior Copywriter

Reviewed by Boris Seleznev

Reviewed by

Boris Seleznev

Boris Seleznev is a seasoned performance engineer with over 10 years of experience in the field. Throughout his career, he has successfully delivered more than 200 load testing projects, both as an engineer and in managerial roles. Currently, Boris serves as the Professional Services Director at PFLB, where he leads a team of 150 skilled performance engineers.

When the tool cannot see the traffic

Microsoft Dynamics NAV here is not a web application. It is a thick client talking to a server application over a database, and the customer wanted two things: a first real measurement of how the system performs, and a comparison of runtime across three different configurations, using one representative scenario - demonstrating and creating advance payment reports with different contents.

The obstacle arrived immediately and did not go away. Client-server communication is carried over msbin1, a proprietary Microsoft binary protocol. That meant the standard approach - record the traffic, parameterise it, replay it - was simply unavailable: neither LoadRunner nor Fiddler nor Wireshark could intercept it. Queries and responses could be observed only through the application log and through Canape.

The second obstacle compounded the first. There was no documentation for the Microsoft Dynamics NAV API, and Microsoft support did not supply it on request. So the alternative route - drive the system through its API instead of its wire protocol - had to be reverse-engineered rather than read.

This is the situation in which a buyer actually learns what a performance testing team is worth. A vendor whose method depends on a recorder has nothing to offer here. The work that followed is entirely tooling the team had to build.

What was built

• The database was filled with test data.
• Telegraf + InfluxDB + Grafana monitoring was deployed, including transaction monitoring driven from Grafana scripts.
• Load testing scripts were developed using the .NET protocol in LoadRunner.
• A separate C# application was developed that uses the Microsoft Dynamics NAV API client.
• Query monitoring was configured using Canape, which - with the application log - was the only window onto msbin1 traffic.
• Because the free LoadRunner license limits the number of sessions, a proprietary metrics collection system was written to gather transaction metrics from scripts running on different hosts through InfluxDB and Grafana.
• The final testing ran on a free LoadRunner license across 8 hosts with 50 threads each, with a proprietary automatic metric collector.

What could not be solved - stated plainly

Two problems were not fully overcome, and the case records them rather than smoothing them over:

• The correlation of an encrypted binary field with its data when calling a write command to the database could not be resolved. In the final version of the script, writes to the database were performed by direct SQL, bypassing the application server.
• A similar problem arose when using the Microsoft Dynamics NAV API.

The consequence matters and should be read honestly: write operations in the final scripts did not exercise the application server path. The measurement is a valid initial evaluation and a valid comparison between configurations, because every configuration was measured the same way - but it is not a full end-to-end write-path measurement, and it was not presented as one.

Results

• The customer received an initial evaluation of system runtime for the three different configurations - the configuration comparison they asked for.
• The errors discovered required DB table blocking, session terminations and similar interventions until they were eliminated.
• PFLB gained working experience with Microsoft proprietary solutions based on Microsoft Dynamics, including the msbin1 protocol and the undocumented NAV API.

Questions this engagement answers

What should I look for when choosing a performance testing vendor?

This case is a checklist in the form of a project. Ask four things.

Can a thick-client ERP be load tested at all?

Yes, but not by the default method. Expect scripts at the protocol or API layer, custom code to drive the client's API, and monitoring assembled specifically for the engagement. Budget for building tools, not just running them.

What does "compare three configurations" require that a single test does not?

Identical treatment. The value of this engagement is the comparison, and a comparison only holds if every configuration is driven by the same scenario through the same instrumented path - which is why the same advance-payment-report scenario ran against all three, and why the known limitation applied equally to each.

What do you do when the vendor of the system will not give you API documentation?

You reverse the interface from the log and from a proxy that can read it. Here Canape plus the application log were the only sources of query and response visibility, and the performance metrics were reconstructed from a purpose-built collector on top of them.