Part 7: Cluster Testing & Validation
DOCUMENT CATEGORY: Runbook SCOPE: Production readiness validation PURPOSE: Validate cluster for production workloads and customer handover MASTER REFERENCE: Microsoft Learn - Validate Deployment
Status: Active
Overview
This section covers comprehensive technical validation of the Azure Local cluster to ensure production readiness before customer handover. Each step generates a validation report that feeds into the consolidated handover documentation.
Purpose
- Validate all cluster components are functioning correctly
- Document performance baselines for storage and network
- Test high availability and disaster recovery capabilities
- Verify security posture and compliance
- Generate reports for customer handover package
Validation Steps
| Step | Description | Duration | Deliverable |
|---|---|---|---|
| 1 | Infrastructure Health Validation | 1-2 hours | Health validation report |
| 2 | VMFleet Storage Performance Testing | 4-6 hours | Storage performance baseline report |
| 3 | Network & RDMA Validation | 1-2 hours | Network validation report |
| 4 | High Availability Testing | 2-4 hours | HA/failover test results, RTO documentation |
| 5 | Security & Compliance Validation | 1-2 hours | Security posture report |
| 6 | Backup & DR Validation | 2-4 hours | Backup/restore test results |
Total Estimated Duration: 12-20 hours (steps can be parallelized)
Prerequisites
Before starting validation:
- All cluster deployment stages complete (Parts 1-6)
- Administrative access to all cluster nodes
- Windows Server 2022 Core VHD available (for VMFleet)
- Maintenance window scheduled (VMFleet creates significant I/O load)
- Test VMs will be created during validation (deleted after)
Test Environment Setup
Dedicated test VMs are created during Step 4 (HA Testing):
| Test VM | OS | Purpose | Delete After |
|---|---|---|---|
TEST-WIN-01 | Windows Server 2022 | HA testing, backup testing, connectivity | Yes |
TEST-WIN-02 | Windows Server 2022 | Live migration target, failover testing | Yes |
TEST-LNX-01 | Ubuntu 22.04 LTS | Cross-platform validation, network testing | Yes |
Validation Reports
Each step generates a separate report stored in a central location:
\\<ClusterName>\ClusterStorage$\Collect\validation-reports\
├── 01-infrastructure-health-report-YYYYMMDD.txt
├── 02-vmfleet-storage-baseline-YYYYMMDD.csv
├── 02-vmfleet-storage-summary-YYYYMMDD.txt
├── 03-network-rdma-validation-YYYYMMDD.txt
├── 04-ha-failover-test-results-YYYYMMDD.txt
├── 05-security-compliance-report-YYYYMMDD.txt
├── 06-backup-dr-validation-YYYYMMDD.txt
└── CONSOLIDATED-VALIDATION-REPORT-YYYYMMDD.pdf
Consolidated Handover Report
After all steps complete, generate consolidated report:
# Script to consolidate all validation reports
# Located in: scripts/validation/New-ConsolidatedValidationReport.ps1
The consolidated report includes:
- Executive summary (pass/fail status for each category)
- Storage performance baseline metrics
- Network performance metrics
- HA/failover test results with RTO/RPO
- Security compliance status
- Backup validation results
- Recommendations and known issues
Success Criteria
Production Readiness Checklist
| Category | Criteria | Status |
|---|---|---|
| Infrastructure | All nodes healthy, Test-Cluster passes, no health faults | ☐ |
| Storage | VMFleet baseline documented, performance within expected range | ☐ |
| Network | RDMA operational, DCB configured, all VLANs accessible | ☐ |
| High Availability | Live migration < 5s, node failover < 2 min, quorum maintained | ☐ |
| Security | Defender score > 80%, no critical recommendations, RBAC verified | ☐ |
| Backup | Backup job successful, restore test passed, RPO/RTO documented | ☐ |
Validation Sign-Off
| Role | Name | Date | Signature |
|---|---|---|---|
| Azure Local Cloud Engineer | |||
| Azure Local Cloud QA | |||
| Project Manager |
Next Steps
After all validation steps complete with passing status:
- Generate consolidated validation report
- Archive individual reports to handover package
- Clean up test VMs and VMFleet resources
- Proceed to Part 8: Validation & Handover for team sign-offs