Skip to content

Tool Comparison Matrix ​

Category: Reference

This page provides a side-by-side comparison of all tools in the Azure Local Load Testing Framework. For selection guidance, see the Tool Selection Guide.


Capability Matrix ​

DimensionfioiPerf3HammerDBstress-ngVMFleet
Target OS — Linux✅✅—✅—
Target OS — Windows—✅✅—✅
Network testing—✅———
Storage I/O testing✅——✅ (VFS)✅
Database workload testing——✅——
CPU stress testing———✅—
Memory stress testing———✅—
VM fleet workload simulation————✅
Install script included✅—✅—✅
CI/CD pipeline included✅✅✅✅✅
Ansible role included✅————

Output and Metrics ​

ToolOutput FormatPrimary MetricSecondary Metrics
fioJSONIOPSThroughput MB/s, Latency P99 ms
iPerf3JSONThroughput MB/sJitter ms, Retransmits, Packet loss %
HammerDBLog → JSONNOPMTPM, Peak NOPM, Sample count
stress-ngYAML → JSONbogo-ops/secTotal bogo-ops, Real time, Usr/sys time
VMFleetCSV → JSONCluster IOPSCPU %, Latency ms, Queue depth

Profile Summary ​

ToolProfile CountProfile Files Location
fio5config/profiles/fio/
iPerf33config/profiles/iperf/
HammerDB2config/profiles/hammerdb/
stress-ng3config/profiles/stress-ng/
VMFleetN/Aconfig/ (config-driven, not profile-driven)

Monitoring Coverage ​

ToolAlert RulesCPU AlertsMemory AlertsStorage AlertsNetwork Alerts
fio7✅✅✅✅
iPerf36✅✅—✅
HammerDB7✅✅✅✅
stress-ng6✅✅✅—
VMFleet(shared PerfMon)✅✅✅✅

Connectivity Requirements ​

ToolProtocolPort(s)Auth Method
fioSSHTCP 22Key-based (linux_ssh)
iPerf3 (server)SSHTCP 22Key-based (linux_ssh)
iPerf3 (test traffic)TCP / UDPTCP/UDP 5201None (direct)
HammerDBWinRM / PowerShell remotingTCP 5985 / 5986Windows credentials (windows_winrm)
stress-ngSSHTCP 22Key-based (linux_ssh)
VMFleetWinRM / PowerShell remotingTCP 5985 / 5986Windows credentials (windows_winrm)

Report Generation ​

All tools write an aggregate JSON file consumed by New-LoadReport.ps1. The report template and placeholder mapping differ by tool:

ToolTemplate FileKey Placeholders
fioreports/templates/fio-report.adoc{seq_read_iops}, {rand_write_iops}, {p99_latency_ms}
iPerf3reports/templates/iperf-report.adoc{tcp_avg_throughput_mbps}, {udp_avg_jitter_ms}
HammerDBreports/templates/hammerdb-report.adoc{peak_nopm}, {avg_nopm}, {peak_tpm}
stress-ngreports/templates/stress-ng-report.adoc{cpu_avg_bogo_ops_per_sec}, {memory_avg_bogo_ops_per_sec}
VMFleetreports/templates/vmfleet-report.adoc{cluster_iops}, {avg_latency_ms}

When to Use Multiple Tools Together ​

See the Tool Selection Guide — Combining Tools section for the recommended test sequence for production validation.

Released under the MIT License.