The pipeline does not understand: it asks questions and reads an exit code
A human reviewer does not see everything, and does not see the same things every time. A security pipeline repeats the same questions on every pull request, and what comes out of it for the rest of the system is a single number: the exit code. If it is 0 the build continues, otherwise it stops. The questions come in two families. SAST (static analysis) reads the code and looks for dangerous shapes; DAST (dynamic analysis) queries the running application and observes the responses. They see different things, and the experiment that follows shows it on a concrete case: the flaw that one finds and the other does not.
The target is a three-route Express application written for the purpose, with three flaws, and its fixed version. Neither is started as a server: the test DAST calls the application function with light-my-request, with no ports and no network. The SAST is Semgrep 1.185.0, installed from PyPI in a temporary virtual environment, with rules written in the listing and with --metrics=off, which the manual indicates to avoid sending usage metrics. Versions read on 2026-11-13: Node v24.11.0, Express 5.3.0, light-my-request 6.8.0. Every output is identical across two consecutive runs.
The target: three flaws and their fix
The first route puts the name parameter into the HTML without escaping (reflected XSS, CWE-79). The second passes the expr parameter to eval (CWE-95): whoever calls the endpoint runs JavaScript on the server. The third sets a session cookie without protection attributes (CWE-1004 and CWE-614). They are textbook errors, chosen because each has a different way of being seen from the code or from the response.
import express from 'express';
export const app = express();
// 1) Reflected XSS: the parameter ends up in the HTML without escaping
app.get('/hello', (req, res) => {
res.send(`<h1>Hello ${req.query.name}</h1>`);
});
// 2) Code execution: the parameter is evaluated as JavaScript
app.get('/calc', (req, res) => {
res.send(String(eval(req.query.expr)));
});
// 3) Session cookie without protection attributes
app.post('/login', (req, res) => {
res.cookie('sid', 'abc123');
res.json({ ok: true });
});
The fixed version does four things, and it is worth reading them one by one. A middleware adds to every response a restrictive Content-Security-Policy, X-Content-Type-Options: nosniff and Referrer-Policy; escapeHtml neutralizes the five special characters; the /calc route loses eval and accepts only two integers of at most nine digits, with an anchored regular expression; the cookie receives httpOnly, secure and sameSite: 'strict'. The value to choose with most care is sameSite: strict is the most protective, but a link from another site will arrive without the cookie, and if you need to come back with a session from external links you use lax.
import express from 'express';
export const app = express();
const escapeHtml = (s) => String(s).replace(/[&<>"']/g, (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c]);
// Security headers on every response
app.use((req, res, next) => {
res.set('Content-Security-Policy', "default-src 'none'; frame-ancestors 'none'");
res.set('X-Content-Type-Options', 'nosniff');
res.set('Referrer-Policy', 'no-referrer');
next();
});
app.get('/hello', (req, res) => {
res.send(`<h1>Hello ${escapeHtml(req.query.name ?? '')}</h1>`);
});
// No eval: only two integers and an addition
app.get('/calc', (req, res) => {
const { a, b } = req.query;
if (!/^\d{1,9}$/.test(a ?? '') || !/^\d{1,9}$/.test(b ?? '')) return res.status(400).send('invalid parameters');
res.send(String(Number(a) + Number(b)));
});
app.post('/login', (req, res) => {
res.cookie('sid', 'abc123', { httpOnly: true, secure: true, sameSite: 'strict', path: '/' });
res.json({ ok: true });
});
SAST: a rule is a question about the code
A Semgrep rule describes a shape of code. The three in the listing are deliberately simple: eval(...) inside a route handler, res.send with a template literal containing req.query.something, and res.cookie called with only two arguments, that is without options. The severity (ERROR or WARNING) is a choice of whoever writes the rule and is then used to decide what blocks the build. A note of honesty: the eval rule fires for any eval in a handler, not only for those fed by user data; that is why the message says "inside a route handler" and not "with request data".
rules:
- id: eval-in-route-handler
languages: [javascript]
severity: ERROR
message: eval() inside a route handler (CWE-95)
patterns:
- pattern: eval(...)
- pattern-inside: |
app.$METHOD($PATH, ($REQ, $RES) => { ... })
- id: html-with-unescaped-input
languages: [javascript]
severity: ERROR
message: request parameter interpolated into an HTML response without escaping (CWE-79)
pattern: $RES.send(`...${$REQ.query.$P}...`)
- id: cookie-without-options
languages: [javascript]
severity: WARNING
message: res.cookie() without httpOnly/secure/sameSite (CWE-1004, CWE-614)
pattern: $RES.cookie($N, $V)
The script calls semgrep scan with execFileSync and prints the findings sorted by line. The detail that surprises is in the catch: without --error Semgrep exits with 0 even when it finds something, but recovering stdout from the exception avoids surprises if one day someone adds that flag. On the vulnerable version three findings appear, with line and severity, and on the fixed version zero.
import { execFileSync } from 'node:child_process';
// Static analysis with Semgrep and local rules (no network, no telemetry)
function scan(path) {
let out;
try {
out = execFileSync('semgrep', ['scan', '--config', 'rules.yml', '--metrics=off', '--disable-version-check', '--quiet', '--json', path],
{ encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] });
} catch (e) { out = e.stdout; }
return JSON.parse(out).results
.sort((a, b) => a.start.line - b.start.line)
.map((r) => `${r.path}:${String(r.start.line).padStart(2)} ${r.extra.severity.padEnd(7)} ${r.check_id}`);
}
for (const file of ['target/app-vuln.mjs', 'target/app-fixed.mjs']) {
const findings = scan(file);
console.log(`${file}: ${findings.length} findings`);
for (const f of findings) console.log(' ' + f);
}
// Output:
// target/app-vuln.mjs: 3 findings
// target/app-vuln.mjs: 7 ERROR html-with-unescaped-input
// target/app-vuln.mjs:12 ERROR eval-in-route-handler
// target/app-vuln.mjs:17 WARNING cookie-without-options
// target/app-fixed.mjs: 0 findings
The three lines coincide with the three flaws and say where to intervene: it is the advantage of SAST, which indicates the line and does not require a running application, so it runs in a few seconds on every pull request. Its limit lies in what the rule recognizes. The next listing writes the same XSS in three ways: the template version is flagged, the string concatenation one is not, and the one with the nosemgrep comment is not either, by construction, because the comment suppresses the rule. The second absence is a true false negative: the rule recognizes a shape, not the meaning, and to cover concatenation as well you have to write another rule or use Semgrep's taint mode (which I did not use here).
import { execFileSync } from 'node:child_process';
import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
// The same XSS vulnerability, written in three ways. Which one does the html-with-unescaped-input rule see?
const dir = mkdtempSync(join(tmpdir(), 'sast-var-'));
const handler = (name, body) => `app.get('/${name}', (req, res) => {\n${body}\n});\n`;
const files = {
'template.mjs': handler('a', " res.send(`<h1>Hello ${req.query.name}</h1>`);"),
'concatenation.mjs': handler('b', " res.send('<h1>Hello ' + req.query.name + '</h1>');"),
'suppressed.mjs': handler('c', " // nosemgrep: html-with-unescaped-input\n res.send(`<h1>Hello ${req.query.name}</h1>`);"),
};
for (const [name, src] of Object.entries(files)) writeFileSync(join(dir, name), src);
const raw = execFileSync('semgrep', ['scan', '--config', 'rules.yml', '--metrics=off', '--disable-version-check', '--quiet', '--json', dir], { encoding: 'utf8' });
const hit = new Set(JSON.parse(raw).results.map((r) => r.path.split('/').pop()));
for (const name of Object.keys(files)) console.log(name.padEnd(20), hit.has(name) ? 'flagged' : 'NOT flagged');
rmSync(dir, { recursive: true, force: true });
// Output:
// template.mjs flagged
// concatenation.mjs NOT flagged
// suppressed.mjs NOT flagged
Two habits follow from this. The first: a new rule is always tried on a variant that must escape it, as here, to know what it does not see. The second: every nosemgrep is a decision to review in code review, with the reason written beside it, otherwise suppression becomes the quickest way to turn the pipeline green again.
DAST: the same application seen from outside
The dynamic probe does not read the code: it sends requests and compares the responses. It sends the payload <script>alert(1)</script> and checks whether it comes back intact in the body, asks /calc for typeof process (a harmless expression, but one that succeeds only if the server executes it), checks the presence of three security headers and reads the cookie's attributes. It is a toy: it does not crawl, does not try variations, does not look for unknown vulnerabilities. A real scanner such as OWASP ZAP, which its site presents as a free and open source web application scanner, works against an application on the network with far greater coverage. I did not run it here.
import inject from 'light-my-request';
// In-process dynamic probe: it calls the application, it does not read the code. No port, no server.
const PAYLOAD = '<script>alert(1)</script>';
const REQUIRED_HEADERS = ['content-security-policy', 'x-content-type-options', 'referrer-policy'];
async function probe(label, app) {
console.log(`== ${label}`);
const xss = await inject(app, { url: '/hello?name=' + encodeURIComponent(PAYLOAD) });
console.log('reflected XSS :', xss.body.includes(PAYLOAD) ? 'VULNERABLE (payload in the body, not escaped)' : 'ok (payload neutralized)');
const rce = await inject(app, { url: '/calc?expr=' + encodeURIComponent('typeof process') });
console.log('server eval :', rce.body === 'object' ? 'VULNERABLE (the server ran my code)' : `ok (response ${rce.statusCode})`);
const missing = REQUIRED_HEADERS.filter((h) => !(h in xss.headers));
console.log('headers :', missing.length ? 'missing ' + missing.join(', ') : 'all present');
const login = await inject(app, { method: 'POST', url: '/login' });
const cookie = [login.headers['set-cookie']].flat().join(';').toLowerCase();
const lacks = ['httponly', 'secure', 'samesite'].filter((a) => !cookie.includes(a));
console.log('session cookie :', lacks.length ? 'missing ' + lacks.join(', ') : 'httponly, secure, samesite present');
}
for (const [label, file] of [['app-vuln', './target/app-vuln.mjs'], ['app-fixed', './target/app-fixed.mjs']]) {
await probe(label, (await import(file)).app);
}
// Output:
// == app-vuln
// reflected XSS : VULNERABLE (payload in the body, not escaped)
// server eval : VULNERABLE (the server ran my code)
// headers : missing content-security-policy, x-content-type-options, referrer-policy
// session cookie : missing httponly, secure, samesite
// == app-fixed
// reflected XSS : ok (payload neutralized)
// server eval : ok (response 400)
// headers : all present
// session cookie : httponly, secure, samesite present
The vulnerable version fails four checks out of four; the fixed one passes them all. The difference from SAST is instructive: the test SAST did not flag the missing security headers at all (the cookie yes), because the absence of a line of code is hard to describe with a rule, while in the response it is evident. In the opposite direction the DAST says "the server ran my code" but not in which line: you then have to go and look for it. That is why the two analyses add up instead of replacing each other.
The gate: from the result to the exit code
With --error Semgrep exits with 1 if there are findings, as the corresponding line of the manual indicates. It is the piece that connects the analysis to the pipeline. The listing tries it on the two versions and uses --severity, which limits the analysis to the rules of the indicated level, to show a gradual adoption: you start by blocking only ERROR and leave the others as a warning.
import { spawnSync } from 'node:child_process';
// All the pipeline sees is the exit code: 0 the build goes on, non-zero it stops
const gate = (path, extra = []) => spawnSync('semgrep',
['scan', '--config', 'rules.yml', '--metrics=off', '--disable-version-check', '--quiet', '--error', ...extra, path],
{ stdio: 'ignore' }).status;
console.log('app-vuln , all rules -> exit', gate('target/app-vuln.mjs'));
console.log('app-fixed , all rules -> exit', gate('target/app-fixed.mjs'));
// --severity filters the rules by severity: with WARNING only the cookie rule fires
console.log('app-vuln , severity ERROR only -> exit', gate('target/app-vuln.mjs', ['--severity', 'ERROR']));
console.log('app-vuln , severity WARNING only -> exit', gate('target/app-vuln.mjs', ['--severity', 'WARNING']));
// Output:
// app-vuln , all rules -> exit 1
// app-fixed , all rules -> exit 0
// app-vuln , severity ERROR only -> exit 1
// app-vuln , severity WARNING only -> exit 1
The output shows the main constraint of a gate: on the fixed version the code is 0, on the vulnerable one it is 1 in both modes, because the ERROR severity flaws are there. The trade-off to choose is the severity that blocks. A gate that blocks everything risks being switched off in the first week of false positives; a gate that never blocks is not a gate. The reasonable path is to start from the highest level, measure how many findings are true and lower the threshold only when the team can keep up.
The workflow: the same thing in CI, with minimal permissions
The following file is a template to adapt, it was not run: GitHub was not consulted, and the SHAs of the actions are placeholders to be replaced with real, verified ones. I only checked that it is valid YAML with a parser. There are four points to look at: the top-level permissions reduced to contents: read, the actions pinned to a SHA (GitHub's guide, already cited in the article on the supply chain, recommends this and indicates workflow lockfiles as the strongest protection), the pinned Semgrep version and npm ci --ignore-scripts for installation.
# Template to adapt: NOT run in this article. The SHAs must be replaced with the real ones
# of the versions you have verified; semgrep must be installed at a pinned version.
name: security
on:
pull_request:
permissions:
contents: read # the GITHUB_TOKEN starts from the minimum, each job asks for more only if needed
jobs:
sast:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@<full-40-character-sha> # action pinned to a SHA
with:
persist-credentials: false
- uses: actions/setup-python@<full-40-character-sha>
with:
python-version: '3.12'
- run: pip install semgrep==1.185.0 # pinned version, not "latest"
- run: semgrep scan --config rules.yml --metrics=off --error --severity ERROR .
dependencies:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@<full-40-character-sha>
with:
persist-credentials: false
- uses: actions/setup-node@<full-40-character-sha>
with:
node-version: 24
- run: npm ci --ignore-scripts # lockfile respected, scripts disabled
- run: npm audit --audit-level=high
The sast job reproduces the gate on the code, and the dependencies job applies what the article Supply chain security proved with npm ci and npm audit. The choice not to gate on DAST, absent here, is deliberate: it requires a running test environment, and it is worth adding only when there is a stable staging, so as not to make the pipeline slow and unstable.
What this experiment does not cover, and where to start
Left out are secret scanning, infrastructure-as-code analysis, container image analysis and real DAST: each would deserve its own experiment, and here they are only named. To decide what to verify, the reference is the OWASP ASVS, whose page indicates 5.0.0 as the latest stable version: it is a list of numbered requirements with which to turn "secure" into verifiable checks. The rules this pipeline enforces come from the previous articles: authorization and token errors in API Security, those on encryption and hashing in Cryptographic failures, XSS and headers in XSS, CSRF and CSP. The series starts with the framing in OWASP Top 10 2025.
What you can do now
You can write a SAST rule, try it also on a variant that must escape it and know what it does not see. You can write a dynamic probe that checks an application's responses without opening ports, and explain why SAST and DAST find different flaws. You can turn the result into an exit code, choose the severity that blocks and build a workflow with minimal permissions and pinned actions. The real work begins afterwards: maintaining the rules, reviewing suppressions and reading the findings before someone switches them off.
Frequently asked questions
What is the difference between SAST and DAST?
SAST analyzes the code without running it and indicates the line, DAST queries the running application and observes the responses. The first sees flaws in the code, the second flaws of configuration and behavior; they find different sets and are used together.
Does a SAST rule that reports nothing mean the code is secure?
No. A rule recognizes a shape of code, not its meaning: in the experiment the same XSS written with string concatenation instead of a template is not flagged. Every rule should also be tried on a variant that must escape it.
How do you make the pipeline fail on a Semgrep finding?
With the --error flag, which according to the manual makes the command exit with code 1 if there are findings. With --severity you choose the severity of the rules to run, to block only the most serious ones first.
Why pin GitHub actions to a SHA?
Because a tag can be moved to other code, while a full SHA identifies a precise commit. GitHub's guide recommends it and indicates workflow lockfiles, which record the commit of every action, as the strongest protection.
The same series also includes SQL injection and input validation and sessions and cookies.
Sources
- Semgrep: CLI reference - the --error, --severity and --metrics flags used in the listings (page read), consulted on 2026-11-13
- semgrep on PyPI - version 1.185.0 installed for the experiment, read through the registry's JSON API, consulted on 2026-11-13
- GitHub Docs: Secure use reference - pinning to a full SHA, workflow dependency lockfiles and minimal permissions of the GITHUB_TOKEN (text read), consulted on 2026-11-13
- OWASP Application Security Verification Standard - 5.0.0 as the latest stable version and the format of the requirements (page read), consulted on 2026-11-13
- ZAP (Zed Attack Proxy) - the project's description as a free and open source web application scanner (home page read; tool not run), consulted on 2026-11-13
- CWE-79: Improper Neutralization of Input During Web Page Generation - the official name of the XSS weakness; CWE-95, 1004 and 614 cited in the text are analogous, consulted on 2026-11-13
The JavaScript listings and the rules were each run twice with identical output, with Semgrep in a temporary virtual environment. The YAML workflow was not run (only parsed syntactically) and ZAP was not used.

Commenti
Caricamento commenti...
Sign in to comment