Managing Flaw Review with a Large Multi-Vendor Application

The previous blog post in this series discussed strategies for the large-scale deployment of the Veracode static code analysis tool across a large enterprise, focusing on strategies and techniques for ensuring rapid adoption within individual development teams typically responsible for self-contained homogenous applications. However, in a large enterprise, there are applications that are developed by multiple vendors and consist of a number of divergent codebases – this blog post discusses techniques for tailoring your Veracode process to accommodate larger multi-vendor applications. multi-vendor-applications

Late Programme Lifecycle Implementation of a Static Tool

A flagship programme within my organisation was a harmonised platform for our European operations; the development timescale spans several years, comprises multiple teams in multiple regions, and consists of a combination of bespoke application code, and third-party and open source software. The overarching objective for the use of Veracode within this programme was to obtain a summary view of the security posture of the overall application by performing a Veracode scan at a regular cadence as part of the standard build and deploy process. The decision to adopt Veracode analysis was made during a relatively late stage of the programme lifecycle, meaning that existing contractual agreements with vendors did not mandate any kind of code analysis – how were we to get them to cooperate in this process?

Gaining Vendor Cooperation

The most common concern for a vendor unfamiliar with Veracode is the perceived high false positive rates associated with a static code analysis tool. The best way to allay such fears is to provide the vendor access to the platform at the earliest opportunity and to allow it to perform its own scans in order to gain familiarity with the platform and the quality of the findings.

There was a marked change in attitude when vendors saw the number of tracked flaws reduced from several thousand to a few hundred…

The next concern raised was the sheer volume of flaws identified – in one case a single vendor contributed over 8 million lines of source code. The key message to our vendors was that the objective is to identify and remediate exploitable risk, rather than producing a perfect result sheet. To facilitate this, our team worked to produce a mitigation workflow that broke down various flaw categories (based on CWE) and exploitability (as determined by the Veracode analysis), resulting in a rule set that could be applied to a Veracode result set to produce a filtered set of flaws based on risk rating. There was a marked change in attitude when vendors saw the number of tracked flaws reduced from several thousand to a few hundred; immediately the response was more pro-active, especially when the flaws could be confirmed in consultation calls. In summary: Establish realistic and achievable targets for remediation, and ensure that you are able to measure and track progress.

Customising Vendor-Specific Reports

Being able to provide customised reporting for each vendor from a single Veracode scan result set presented some unique challenges: it was important to provide each vendor with visibility of their flaws (and only their flaws) and to ensure that flaws that were not managed (due to their lower risk rating) were not reported. Additionally, flaws within third-party and open source code were to be identified and tracked, but no action was expected of the vendor; rather, these items were flagged for specific focus during the manual penetration testing phase. The Veracode platform detailed report provides a verbose description of all flaws in XML format; this was imported into Excel and a number of VBA macros were run to assign vendor names (based on module names) and to remove all items that were not tracked. The platform is able to accurately distinguish between flaws that have been newly detected and those that existed previously; a macro flagged new items for review and these were reviewed in vendor-specific consultation calls with the Veracode application security experts. A consultation call can involve a significant investment in time from several parties; the ability to sharply focus these calls on only new high-risk items was important in ensuring active vendor participation.

Communicating Remediation Upstream

Finally, the Veracode platform provides an API specifically for the management of flaw remediation and comments. A Microsoft Visual Studio Tools for Office (VSTO) plugin was adapted to use this API to update flaw status on the platform with the mitigation status within the vendor spreadsheet; this was important to ensure that any high-level reporting done for the programme governance would report the current overall security position across all vendors with trending information. The ability of the platform to track flaw locations within successive scans meant that these mitigations were then propagated into downstream scans. Several vendors were able to recognise the value of this programme to the quality of their codebase and were pro-active in ensuring the flaw reports were fed back to their upstream development teams, and in some cases tailored consultation calls were held to address specific vendor concerns, or issues emerging from their scans. This post has given a brief overview of some of the challenges faced in our multi-vendor scenario: the key to success is to gain the cooperation of your vendors by demystifying the process, demonstrating actionable findings and showing a pragmatic and risk-based approach to the prioritisation of their remediation effort while stressing the benefits for both parties.

For more details on the development of my application security progamme, see Ad Hoc to Advanced Application Security: Your Path to a Mature AppSec Program.


Leave a Reply