Managing a sprawling digital organization often feels like a constant battle against digital clutter. For repository owners, the task of maintaining security—confirming who truly needs access, adjusting permissions as project requirements shift, and archiving repositories that have outlived their utility—is frequently a manual, tedious process. As projects scale, this housekeeping often falls by the wayside, leading to security risks and organizational bloat. To address this, GitHub engineers have developed "Evergreen," an internal tool that leverages native platform features to automate these critical maintenance tasks, ensuring that organizational hygiene remains consistent and secure.
The challenge of repository maintenance is particularly acute in shared environments where employees collaborate with external partners, customers, and contractors. At GitHub, for example, staff often work alongside external entities on product trials or specialized professional service engagements. Managing these outside collaborator permissions manually is unsustainable, yet neglecting them creates potential security vulnerabilities. To solve this, the team built Evergreen, a GitHub App designed to handle organization-wide maintenance through automated workflows that prioritize security and user experience.
The Design Philosophy of Evergreen
At the heart of the Evergreen project are two fundamental design principles: "fail secure" and "never destroy." In the context of access management, "fail secure" means that the default outcome is always the safest one. If a review process is initiated and no one provides a definitive response regarding a collaborator’s access, the system does not simply leave that access in place; it automatically revokes it. The system assumes that if an owner cannot justify why an external user still needs access, that access should be removed to protect the repository’s integrity.
Simultaneously, the "never destroy" principle ensures that the system is not a destructive force. Evergreen is architected to archive repositories rather than delete them. This allows for recovery should a project ever need to be resurrected. Furthermore, every access change is meticulously documented within the repository’s tracking issue. This transparency means that if an error occurs—such as a collaborator being removed by mistake—reverting that change is a simple, trackable process. By integrating these principles, GitHub has created a system that balances rigorous security with operational flexibility.
Choreographing Native GitHub Features
Technically, Evergreen operates as a lightweight Go-based webhook server hosted within a container. However, the true ingenuity of the tool lies in its "choreography"—the way it orchestrates existing, native GitHub features to perform complex tasks. Rather than relying on exotic, external infrastructure, the system utilizes a combination of GitHub Apps, scheduled Actions, webhooks, tracking issues, and custom repository properties.
The system is managed through a central controller repository that functions as both the "clock" and the "memory" for the entire organization. This controller uses scheduled GitHub Actions workflows to trigger repository dispatch events on a predictable, quarterly cadence. By spacing out the work, the system avoids overwhelming the platform or the users.
For instance, at the start of each quarter, the system initiates a stale repository review to identify projects that have seen no human activity. A week later, it pivots to an access review, where it snapshots the current list of outside collaborators for every repository and opens a checklist issue. As the month progresses, the system executes the necessary cleanup: archiving repositories that remain stale after a designated grace period and revoking access for collaborators whose status was not renewed by an authorized owner. Because each of these workflows also supports manual triggering, administrators retain the power to run these tasks on demand if an urgent situation arises.
Maintaining an Audit Trail Through State Management
One of the most elegant aspects of Evergreen is how it tracks the state of each repository. Instead of relying on an external database that could become desynchronized, the system stores the status of every repository as a JSON file within the controller repository. Each file acts as a persistent record of the repository’s history, including when it was last seen, the status of its access reviews, and which users were granted or denied continued access.
Because this state is maintained as a file within a Git repository, the commit history serves as an immutable, transparent audit trail. If a security auditor or a project manager needs to investigate why a specific user lost access to a project months ago, they can simply check the git log of the relevant state file. This eliminates the need for complex, bespoke audit pipelines or secondary database management, effectively turning the platform’s core version control functionality into an administrative tool.
Scaling "IssueOps" for Large Organizations
The user interface for Evergreen is intentionally simple, relying on what the team calls "IssueOps." When an access review is triggered, the system automatically opens an issue inside the specific repository containing a checklist of all current outside collaborators. The instructions are straightforward: the repository owner simply selects the checkboxes for the individuals who require continued access and then closes the issue.
When the issue is closed, the system captures the issues.closed webhook event, parses the checkboxes to determine which users to retain, and then executes the access revocation for those not selected. Finally, it posts a comment to the issue that provides a clear, documented receipt of exactly which permissions were modified. This workflow is designed to be as frictionless as possible for the human owners involved.
To maintain security, the tool strictly validates the identity of the person closing the issue. If an unauthorized user attempts to close a review issue, the application will automatically reopen it and provide a notification explaining that the action must be performed by an authorized owner. This ensures that the responsibility for access control remains firmly with those who have the authority to grant or revoke it.
Informed Decisions and Graceful Failures
A common hurdle in automation is determining the correct point of contact for a repository. Evergreen addresses this through a cascading fallback mechanism. It first looks for specific owners associated with the repository, and if that information is missing, it attempts to identify the appropriate team leads or project managers. This ensures that the people receiving the notifications are actually empowered to make the necessary decisions, preventing the "notification fatigue" that often occurs when automated alerts are sent to the wrong people.
The system is also designed to "fail closed" to prevent inconsistent states. If the system encounters a transient API error while attempting to revoke access for several collaborators, it will not mark the task as complete. Instead, it leaves the issue open and the review pending, allowing the system to retry the operation in subsequent cycles. A task is only marked as finished when the system has successfully completed every step, ensuring that the audit log remains accurate and no security gaps are left behind.
Lessons in Automation and Rollout
Rolling out such a powerful automation tool across a large organization with thousands of employees and even more external partners requires significant caution. GitHub’s team prioritized a "dry-run" phase, where the system logged every action it would take without actually modifying any permissions. This allowed the engineers to monitor the system’s decision-making process in real-time, ensuring that the logic was sound before it was given the authority to make changes.
Communication was another critical component of the rollout. By keeping the organization informed through clear, repeated messaging, the team ensured that the automated issue notifications were never a surprise to the developers. Furthermore, the team utilized deep observability through structured logging, which allowed them to identify and resolve edge cases—such as repositories with unusual configurations—before they caused widespread disruption.
Finally, the team was mindful of the platform’s operational limits. Because the system performs thousands of actions across many repositories, it had to be designed to respect GitHub’s secondary rate limits. By building in deliberate pacing and batching, the tool avoids triggering system abuse protections, ensuring that the automation remains reliable and efficient without needing constant human intervention.
By prioritizing transparency, auditability, and simplicity, GitHub has turned the often-neglected task of organizational hygiene into a seamless, automated process. The result is a more secure, well-maintained environment that proves how existing platform tools can be leveraged to solve the most persistent challenges in modern software development.
