GitHub and Microsoft Pilot “Codename MDASH” to Streamline Open Source Security

In the rapidly evolving landscape of software development, artificial intelligence has become a double-edged sword. While AI-driven tools have made it faster and more affordable to identify potential vulnerabilities in code, they have also created a significant operational bottleneck for the developers who maintain the world’s most critical open source projects. A new collaborative pilot between GitHub and Microsoft, known as "Codename MDASH," aims to shift the paradigm from simply flooding maintainers with raw security findings to providing them with actionable, verified, and ready-to-implement solutions.

The pilot project, which focused on six widely used open source projects, successfully converted thousands of raw AI-generated security signals into 99 high-quality, private reports. These reports did not merely alert maintainers to a problem; they included suggested fixes, significantly reducing the burden of triage. As of mid-November, maintainers had already addressed more than half of these findings, with several fixes reaching production environments, demonstrating the effectiveness of an agentic, human-in-the-loop approach to security.

The Problem: When AI Becomes Noise

For years, the open source community has struggled with the sheer volume of security alerts. Modern automated scanning tools, while capable, often produce a high rate of false positives and duplicate reports. For maintainers—many of whom are volunteers managing projects on their own time—this "noise" can be overwhelming. They are frequently inundated with automated scans that lack context, forcing them to spend hours verifying whether a finding is a genuine vulnerability or an artifact of an imprecise algorithm.

Maintainers don’t need more AI bug reports. They need fixes. 

This situation has created a growing frustration among the maintainers of foundational software. Christian Grobmeier, a maintainer of the widely used Apache Log4j project, has previously emphasized the urgency of this disparity, noting that the AI used to defend open source software must be more sophisticated and accurate than the AI used to attack it. When an automated scanner reports a vulnerability without a clear path to resolution, it doesn’t solve a problem; it merely adds to a maintainer’s mounting workload. The consensus among the developer community is clear: anyone leveraging AI to scan a project should prioritize the maintainer’s time by coming forward with a solution, not just a list of potential issues.

The Codename MDASH Approach

GitHub and Microsoft initiated the Codename MDASH pilot with two primary objectives: to ensure that only relevant, high-impact findings are escalated, and to guarantee that no maintainer is ever sent an unverified finding. Codename MDASH, which stands for Microsoft’s multi-model agentic scanning harness, utilizes the MAI Cyber model to perform deep analysis on open source repositories.

The process was designed to integrate seamlessly into the existing workflows of maintainers. Rather than forcing them to navigate external platforms, all reports and proposed fixes remained within the GitHub environment where they already conduct their daily development. The pipeline began with the AI agent flagging thousands of possible issues. These raw findings then underwent a rigorous triage process, followed by manual verification by a team of seven researchers from the Microsoft FORGE Lab.

Maintainers don’t need more AI bug reports. They need fixes. 

This human-led verification layer was crucial to the pilot’s success. It ensured that the findings reaching the maintainers were not just "possible" bugs, but real, actionable vulnerabilities. By the time a report was delivered to a maintainer, it included a well-scoped patch—a suggested fix that the maintainer could review, adapt, or merge as they saw fit. This effectively flipped the script: instead of starting with a raw alert, the maintainer started with a solution.

Lessons from the Pilot

The results of the pilot were telling. By November 15, of the 99 private reports sent to the six participating projects—AutoGPT, Caddy, Express, FastMCP, ImageMagick, and Netty—maintainers had already fixed 55 of them, with 14 of those patches successfully deployed in official releases.

The feedback from project maintainers highlighted that the value of the initiative lay in its ability to prioritize fixes over raw findings. For instance, the maintainers of Caddy were able to address all 22 of the reports they received within a week. Similarly, the AutoGPT team noted that the AI-generated report was so precise that it identified a specific line of code that had been impacted by a recent update, allowing them to implement a fix within days.

Maintainers don’t need more AI bug reports. They need fixes. 

One of the most striking takeaways from the experiment was the efficiency gain. An ImageMagick maintainer reported that a typical security advisory usually requires between 30 minutes and an hour of intensive work to research, understand, and address. By providing a succinct, accurate summary and a suggested fix, the Codename MDASH process drastically reduced this time, in one instance allowing for a fix to be pushed in just 15 minutes. This suggests that when AI is used to support rather than replace human judgment, it can become a powerful force multiplier for security.

However, the pilot also underscored the limitations of current technology. The process was highly labor-intensive, with researchers spending roughly two days per project manually vetting the findings. While this level of care was essential for the pilot, it highlights a broader, unresolved challenge: the current methodology does not yet scale to cover the millions of open source projects that constitute the global digital supply chain. Furthermore, while the pilot focused on private reporting, the broader ecosystem continues to struggle with a daily influx of low-quality, automated security reports that offer no guidance to the maintainers on the receiving end.

Building a Future for Secure Open Source

The success of the Codename MDASH pilot serves as a proof of concept for a more collaborative, human-centric model of software security. The clear takeaway for the industry is that the goal of security automation should be to reduce the "security tax" paid by maintainers. A report that includes a verified fix is a service; a raw scan that requires manual triage is an burden.

Maintainers don’t need more AI bug reports. They need fixes. 

This philosophy is closely aligned with the broader efforts of the GitHub Secure Open Source Fund. By providing financial support, security training, and a community of practice, the fund aims to bolster the security posture of projects that underpin much of the modern web. The pilot projects in the Codename MDASH initiative are all alumni of this program, which has already supported 216 projects, helping maintainers address more than 4,000 vulnerabilities in software that is downloaded over 20 billion times each month.

As the industry moves forward, the integration of "cyber-capable" AI models will likely become a standard tool for maintainers to verify proofs of concept both before and after a patch is applied. The vision articulated by the participating maintainers is a future where the primary challenge of software development is not merely shipping new features, but stewarding the security of the underlying infrastructure with the support of advanced, responsible tools.

The collaboration between GitHub, Microsoft, and the open source community demonstrates that when security is treated as a shared responsibility—rather than a burden placed solely on the shoulders of individual maintainers—the results can be transformative. As the tools for identifying vulnerabilities become more powerful, the ability to turn those findings into robust, reliable, and timely fixes will determine the resilience of the digital ecosystem for years to come. Ultimately, the success of these efforts hinges on the principle that while AI can do the heavy lifting of scanning, it is the human maintainers who do the final, critical work of securing the code that powers the world. The goal, as defined by those leading the initiative, remains clear: ensure that no maintainer ever has to fix a vulnerability alone.

Share:

Ali Ikhwan writes for Tech Maze.