In September, over 180 NPM packages were compromised by the Shai-Hulud worm, which exfiltrated secrets to 207 public GitHub repositories before most teams even realized they were affected. The attack's speed outpaced detection, but the forensic trail it left in GitHub's public event archive provides a model for effective real-time monitoring.
This checklist is designed to help your team build detection and forensic capabilities that operate at the speed of supply chain attacks. Each item addresses specific gaps revealed by the Shai-Hulud incident.
What This Checklist Covers
You'll implement controls to:
- Monitor GitHub event streams for unauthorized activity
- Detect exfiltration attempts quickly
- Conduct post-incident forensics using public data
- Identify propagation paths in a supply chain compromise
This isn't about preventing package compromise; it's about detecting when your secrets leave your perimeter and tracing the impact when a supplier's package turns malicious.
Prerequisites
Before you start:
- Maintain an inventory of GitHub users with access to organizational secrets
- Document which NPM packages your applications depend on
- Have logging infrastructure capable of ingesting and correlating external event streams
- Define notification timelines for secret exposure incidents
If you're missing any of these, pause here. Detection without context creates noise, not intelligence.
Checklist
1. Establish GitHub Event Stream Monitoring
Subscribe to the GitHub public timeline or configure access to gharchive.org data feeds. Ingest events for repositories owned by users with organizational access.
Done when: You're receiving CreateEvent, PublicEvent, and PushEvent data for all relevant user accounts within 15 minutes of occurrence, and you've validated the feed with a test repository creation.
2. Define Repository Behavior Baselines
Document normal patterns: how often your developers create repos, typical naming conventions, standard visibility settings, and authorized repository descriptions.
Done when: You can programmatically flag a repository with an anomalous description, unexpected visibility changes, or unrecognized naming patterns.
3. Implement Description-Based Anomaly Detection
Configure alerts for repositories created or modified with descriptions that don't match your naming standards. The Shai-Hulud attack used exactly two descriptions across all 207 compromised repos.
Done when: A test repository with the description "Data Exfiltration Test" triggers an alert within your defined SLA, and you've tuned thresholds to eliminate false positives.
4. Monitor Sudden Visibility Changes
Track when private repositories become public. The attack renamed existing repos and exposed them. This pattern is forensically distinct.
Done when: You receive alerts for any private-to-public transition, have documented which teams can make repos public, and can correlate visibility changes with repo rename events.
5. Catalog "Super Spreader" Risk
Identify which developers maintain NPM packages and have write access to multiple internal repositories. In the Shai-Hulud case, a few users were responsible for the largest number of compromised repos.
Done when: You have a list of users who meet both criteria, have assigned them to a higher-risk monitoring tier, and have briefed them on their role in potential propagation scenarios.
6. Build Deleted Repository Forensics Capability
GitHub Search only indexes live content. The event archive persists metadata even after deletion. Configure your tooling to query gharchive.org for historical events tied to now-deleted repositories.
Done when: You can reconstruct the creation timestamp, description, and owner of a repository deleted 48 hours ago using only event archive data.
7. Map Employer Attribution Paths
Document how you'll identify which company a compromised GitHub user works for. The Shai-Hulud investigation manually verified employer data due to high false-positive rates in automated attribution.
Done when: You've defined your verification process, tested it on sample users, and documented acceptable confidence thresholds before escalating to a vendor.
8. Establish Vendor Notification Protocols
Define how you'll contact a supplier when you detect indicators of compromise tied to their employees' GitHub accounts. You'll need contact details, evidence packages, and escalation paths.
Done when: You have pre-drafted notification templates, identified the correct contact for your top 20 critical suppliers, and documented what evidence you'll share versus withhold.
9. Integrate NPM Dependency Monitoring
Cross-reference compromised package lists against your dependency inventory. The attack affected packages your applications might consume.
Done when: Within four hours of a public disclosure, you can generate a report showing which of your applications depend on affected packages, their deployment environments, and who owns remediation.
10. Test Your Detection Latency
Run a controlled test: have an authorized user create a repository with an anomalous description and flip a private repo to public. Measure time-to-alert.
Done when: You detect both actions within your target SLA, and you've documented gaps where detection failed.
Common Mistakes
Relying solely on GitHub Search for forensics. By the time Shai-Hulud was widely reported, most compromised repos had been deleted. Search wouldn't find them, but the event archive did.
Treating all developers equally. Users who maintained NPM packages and had access to many repos caused the most damage. Risk-tier your monitoring.
Waiting for vendor disclosure. Companies affected by Shai-Hulud learned about it from public reporting, not from their suppliers. Your detection can't depend on someone else's notification timeline.
Ignoring repo rename events. A repository that suddenly gets renamed with "-migration" appended and goes public is a bright red flag. Most monitoring focuses on creation, not modification.
Next Steps
Start with items 1, 2, and 10. You need the data feed, the baseline, and proof that detection works before building out the rest.
Then prioritize item 5. Identifying your super spreaders takes manual work, but it's the highest-leverage control. A single compromised maintainer with broad access can propagate malware to dozens of packages.
Finally, integrate this checklist with your existing Incident Escalation procedures. Detection without response is just expensive logging.
