Runtime vs Build-Time Supply Chain Security: What Your Scanner Might Miss

Profile
Yves SoeteFollow
7 min read · Apr 21, 2026

APR 21, 2026 - Written by Yves SoeteBlacksight LLC — scan your site free atscanner.blacksight.io

The supply chain security conversation has been dominated by build-time tools: npm audit, Snyk, Socket, Dependabot. These tools are essential. They catch known CVEs in your dependencies before you ship. They flag license violations, detect typosquatting attempts, and warn you when a package suddenly starts making suspicious network requests in its install scripts. But they share a blind spot that almost nobody talks about: they only see code that lives in your repository. The code that actually runs in your visitors' browsers — the third-party scripts, CDN-hosted libraries, analytics pixels, and embedded widgets — is invisible to every build-time scanner on the market. This is not a limitation anyone is trying to hide. It is a fundamental architectural boundary. Build-time tools analyze your source tree. Runtime threats live outside it.



What build-time scanners do well



Build-time dependency scanning has matured significantly over the last five years, and the tools deserve credit for what they do accomplish. Dependency vulnerability scanning against CVE databases is the core use case. You declare your dependencies in package.json, Gemfile, requirements.txt, or go.mod, and the scanner cross-references every package and version against known vulnerabilities. When a CVE drops for a package in your tree, you get an alert. This is genuine, measurable security value.

License compliance checking prevents legal exposure. Socket's behavioral analysis catches packages that suddenly add network calls, filesystem access, or shell execution to their install hooks — behavior that correlates strongly with supply chain attacks. Dependency confusion detection stops attackers from tricking your package manager into pulling a malicious public package instead of your internal one. Typosquatting alerts catch packages named to look like popular libraries — crossenv instead of cross-env, electorn instead of electron.

You should be running these tools. They are table stakes for any team shipping software in 2026. But understanding what they cover is just as important as running them, because their coverage boundary creates a false sense of completeness.



The runtime gap



Open your browser's DevTools on any production website. Go to the Network tab and filter by JavaScript. Count the domains. On a typical marketing site or SaaS application, you will find scripts loading from a dozen or more external origins that never appear anywhere in the site's source repository.

Google Tag Manager is the most common entry point. A single GTM container can inject analytics pixels, remarketing tags, heatmap recorders, A/B testing scripts, and ad network code — all configured through a web UI that lives entirely outside your version control. Your developers may not even know what GTM is loading this week. CDN-hosted libraries are another category. jQuery loaded from cdnjs.cloudflare.com, Bootstrap pulled from cdn.jsdelivr.net, Font Awesome from its own CDN. These are dependencies in every meaningful sense, but they do not appear in any lockfile.

Payment widgets add Stripe.js, PayPal buttons, or Braintree's hosted fields. Chat and support widgets inject Intercom, Drift, or Zendesk scripts. A/B testing platforms like Optimizely and VWO load client-side experiment code. Ad network scripts from Google Ads, Meta Pixel, and programmatic exchanges execute on every page load.

Every one of these scripts runs with the same privileges as your own code. They can read the DOM, access form data, set cookies, make network requests, and redirect users. A compromised analytics script can exfiltrate credit card numbers from your checkout page. A hijacked chat widget can inject phishing overlays. A tampered CDN library can modify your application's behavior in ways that are invisible to your server-side logs. Build-time tools see none of this. They cannot, because these scripts are not in your repository.



Case study: Polyfill.io (2024)



The polyfill.io incident is the clearest illustration of the runtime gap. Over 100,000 websites loaded polyfill.io through a script tag to provide browser compatibility shims. The domain was acquired by a new owner in early 2024, and the served script was modified to detect mobile User-Agents and redirect visitors to scam and betting sites. On desktop, the script behaved normally, which is why manual spot-checks missed it.

No build-time scanner flagged this. Not npm audit, not Snyk, not Socket, not Dependabot. Because polyfill.io was not in anyone's package.json. It was a CDN script tag, added directly to HTML templates or injected through a tag manager. It existed entirely in the runtime layer.

The attack was eventually caught by security researchers performing runtime behavioral analysis — observing that the JavaScript served by the CDN had changed and was now making requests to domains associated with malicious activity. Runtime monitoring on affected sites would have detected the behavior change within a single scan cycle. Instead, most site operators found out from news articles days or weeks after the compromise began.



Case study: Magecart (ongoing)



Magecart is not a single group or a single attack — it is a class of e-commerce skimming operations that has been active since 2015 and continues to evolve. The common thread is injecting payment card skimmers into online checkout pages by compromising third-party scripts that those pages load.

Early Magecart attacks targeted JavaScript files hosted on e-commerce platforms directly. By 2023, attackers had shifted to compromising third-party vendors whose scripts were loaded by thousands of merchant sites. A single compromised analytics provider or tag management account could deploy skimming code across an entire portfolio of sites simultaneously.

In 2025, the techniques evolved further. Attackers began hiding skimmer payloads in CSS files and SVG images, using atypical delivery mechanisms that most JavaScript-focused security tools did not inspect. Some campaigns used legitimate-looking Google Tag Manager containers to deploy skimming code, making the malicious scripts appear authorized.

Build-time scanners cannot detect any of these attack patterns. The skimmers arrive through third-party scripts that load at runtime, through tag managers configured outside the codebase, or through compromised CDN resources. The malicious code never touches the victim's repository.



What runtime monitoring actually does



Runtime supply chain monitoring takes a fundamentally different approach. Instead of analyzing source code and dependency manifests, it visits your live site in a real browser — the same way your users do — and observes what actually loads and executes.

A runtime scanner catalogs every script, stylesheet, image, font, and external resource that loads during a page visit. It records the origin domain, the full URL, the content hash, and whether Subresource Integrity attributes are present. It checks SRI hashes against the actual served content to verify they match. On subsequent scans, it compares the current inventory against previous baselines and flags any changes: new scripts that appeared, existing scripts whose content changed, scripts that disappeared, or SRI hashes that no longer match.

More advanced runtime monitoring cross-references discovered vendors against known breach databases. If a third-party vendor you depend on has disclosed a compromise, you get an alert tied to the specific script on the specific page of your site. You do not have to read every vendor's security blog and manually check whether you use their code.

The output is an actionable inventory: here are the 47 external resources loading on your checkout page, here is which ones have SRI, here is which ones changed since last week, and here is which vendors have active security advisories. That inventory is the artifact your security team, your compliance auditor, and your incident response process all need.



You need both — here is how to think about it



Build-time scanning protects your code supply chain — the packages you install, the libraries you compile, the dependencies declared in your manifest files. Runtime monitoring protects your user-facing supply chain — the scripts your visitors' browsers actually download and execute when they visit your site.

These are two different attack surfaces with effectively zero overlap. A compromised npm package is invisible to runtime monitoring until it ships. A compromised CDN script is invisible to build-time scanning forever. If you only run one category of tool, you have a blind spot covering roughly half of your total supply chain exposure.

The mental model is straightforward. Build-time tools answer the question: "Is the code I wrote and the packages I chose safe?" Runtime tools answer the question: "Is the code my users' browsers actually execute safe?" Both questions matter. Neither answer substitutes for the other.



The compliance angle



PCI DSS 4.0 made this distinction concrete. Requirement 6.4.3, which became mandatory on March 31, 2025, explicitly requires three things for every payment page: an inventory of all scripts loaded and executed, a mechanism to confirm each script is authorized, and a mechanism to assure the integrity of each script.

Read that requirement carefully. It says "scripts loaded and executed" — not "scripts declared in your package.json." The inventory must cover runtime behavior, not build-time declarations. A QSA assessing your compliance will ask for the list of scripts that load on your checkout page in a real browser, not a printout of your lockfile.

This requirement is only satisfiable with runtime monitoring. Build-time tools cannot produce the evidence a QSA needs because they fundamentally cannot see what loads in the browser. Organizations that assumed their Snyk or Dependabot setup covered PCI 6.4.3 are discovering during 2026 audits that they have a gap — and that gap requires a different category of tooling to close.



Closing the gap



Build-time scanning is table stakes. Keep running npm audit, Snyk, Socket, Dependabot — whatever fits your workflow. These tools have prevented thousands of compromises and they remain essential.

Runtime monitoring is the piece most organizations are still missing. It covers the attack surface that build-time tools architecturally cannot reach: the third-party scripts, CDN resources, tag manager payloads, and embedded widgets that execute in your users' browsers every day.

BlackSight Scanner covers the runtime side. Point it at your URL and get a full third-party script inventory with SRI status, content-change detection, and vendor breach flags in 90 seconds. Plans start at $29/mo for continuous monitoring, and the supply chain scan is free to run right now — no account required.

Run a free scan at scanner.blacksight.io

Liked this article? Get notified when new articles drop — visitblacksight.io/blogto subscribe.

Version 1.0.68