Vendor Management

For outsourced software delivery, it is a best practice to specify software quality requirements in the vendor’s agreement and to perform due diligence by reviewing technical deliverables at regular intervals during development and maintenance. A contract or service level agreement (SLA) may also contain incentives for incrementally improving code quality over time.

 

The CISQ quality characteristic measures for Security, Reliability, Performance Efficiency and Maintainability are strong indicators of the level of risk in system architecture, design and code. Explicit use of the standards with vendors will prioritize software quality measurement earlier in the software development lifecycle to reduce risk and cost.

 

For more guidance, read:
Example Acceptance Criteria with CISQ Standardized Metrics
CISQ Recommendation Guide: Effective Software Quality Metrics for Use in ADM Service Level Agreements

 

 

Example Portfolio Analysis: Summary Results for a 3-Month Period

 

 

Assets under management
Vendor Number of Applications Volume (AFP) Portfolio Percentage % Δ (AEP) Security Reliability Performance Efficiency Maintainability
1 23 110,000 34% 15% 2.77 (+2%) 3.05 (+3%) 2.80 (+1%) 3.02 (+2%)
2 10 45,000 14% 9% 2.78 (+.08%) 2.81 (+.05%) 3.02 (-.02%) 2.86 (-.04%)
3 7 30,000 9% 6% 3.05 (+1.5%) 3.03 (-1%) 3.00 (+.08%) 2.91 (-.05%)
4 29 140,000 43% 4% 2.99 (-2%) 2.97 (-5%) 3.03 (+1%) 3.05 (+2%)

 

Measuring software and contracting with vendors to these standards will ensure quality in applications and help teams detect any software quality risks that could otherwise go unnoticed and cause sudden expensive outages or security issues.

 

Technical deliverables can be reviewed at regular intervals during development, build, maintenance and acquisition. Activities include:

  • Measuring the quality and quantity of software delivered
  • Analyzing the root cause of risk and prioritizing corrective actions
  • Monitoring application/product quality and improving goals over time
  • Allocating team resources appropriately