CISQ hosted its latest Seminar at the HYATT Reston Town Center in Reston, VA, USA. The topic for this installment was “Software Quality in Federal Acquisitions”, and included the following speakers:
- David Herron, David Consulting Group
- Robert Martin, Project Lead, Common Weakness Enumeration, MITRE Corp.
- John Keane, Military Health Systems
- Dr. Bill Curtis, Director, CISQ
- John Weiler, CIO Interop. Clearinghouse
- Joe Jarzombek, Director for Software & Supply Chain Assurance, DHS
- Dr. William Nichols, Software Engineering Institute
Over 75 senior leaders from public and private sector organizations such as BSAF, MITRE, US Department of Defense, Northrop Grumman, NSA, Fannie Mae, US Army, and NIST were in attendance listening to presentations, engaging in discussions, and networking with peers.
Dr. Curtis began the day by discussing the recent changes in the regulatory environment at the Federal level, especially as they relate to software risk prevention. Kevin Jackson (IT-AAC) stressed how innovation cannot be adopted if it cannot be measured.
Mr. Herron introduced the uses of productivity analysis and Function Points to more effectively and efficiently manage portfolios. He noted that a baseline provides a “stake in the ground” measure of performance, help to identify opportunities for optimized development practices, and enables the organization to manage risks and establish reasonable service level measures. Mr. Herron also discussed how automation will change the game with respect to software sizing and Function Points, including increased coupling with structural quality and improved vendor management.
Mr. Martin led a lively session on identifying and eliminating the causes of security breaches through the development of the Common Weakness Enumeration repository. He described the best practices for using information in the repository for improving the security of software, noting that everything is based on software today and that any flaws in that software within today’s highly connected universe will magnify the issues. Different assessment methods are effective at finding different types of weaknesses, and some are good at finding the cause while others can find the effect. So it’s ok to use different methods together.
Mr. Keane then spoke about the tools and processes his team uses to measure and manage structural quality on DoD contracts. He noted the importance of strong vendor contract language dictating the quality and performance standards required. Performing static code analysis correctly has great benefits, and Mr. Keane stated that static analysis prior to testing is very quick and about 85% efficient. His team measures code quality using technical debt, architectural standards, and architectural dependencies.
Mr. Jarzombek showed how security automation, software assurance and supply chain risk management can enable enterprise resilience. He noted that there is an increased risk from supply chains due to: increasing dependence on commercial ICT for mission critical systems; increasing reliance on globally-sourced ICT hardware, software, and services; residual risk, such as counterfeit products and products tainted with malware, passed to the end-user’s enterprise; growing technological sophistication among adversaries. Mr. Jarzombek also noted that the ICT/software security risk landscape is a convergence between “defense in depth” and “defense in breadth”.
Dr. Nichols presented new research on the measurement of agile projects, noting that the agile community lacks hard data regarding which agile practices provide the best outcomes. He identified some trends and distributions but determined that there were no strong correlations between specific agile practices and measures. Great care must be taken when combining agile metrics because the variables often combine in ways that are not intuitively obvious and can easily become misleading when used in different contexts.
Dr. Curtis then concluded the event by talking about the importance of software productivity and quality measures in every kind of contract, and discussing the important work that CISQ is doing on creating specifications for automated software quality measures. He noted the need to curb technical debt in order to reduce the need for future rework, and made some recommendations for acquisition which include:
- Setting structural quality objectives
- Using a common vocabulary
- Measuring quality at the system level
- Evaluating contract deliverables
- Using rewards and penalties wisely
A fantastic Cocktail Social followed the event, facilitating great networking between the speakers and attendees. We received many positive statements from attendees throughout the event, noting the wealth of valuable information that was disseminated through engaging presentations and Q&A.