RORO's blog

Semgrep (Semantic Grep) is a multi-language static analysis tool that matches code by its structure – not just its text – using a unified Abstract Syntax Tree and a rich pattern language. This document covers the full internal architecture from CLI entry-point to taint sink detection.

Table of Contents

0. Introduction
   • What is SAST?
   • History of Semgrep
1. High-Level Architecture
2. Component Breakdown
3. The Full Analysis Pipeline
4. Target Discovery & Filtering
5. Rule Parsing & Optimization
6. Parsing & the Universal AST
7. The Matching Engine
8. The Intermediate Language (IL) & CFG
9. Taint Analysis (Dataflow)
10. Output & Reporting Pipeline
11. OSemgrep / RPC Architecture
12. Key Data Structures

-1. Why?

I have been working as a DevSecOps engineer for nearly four years now. When I started, I had little exposure to tooling like SAST, SCA, or DAST. My mindset was firmly rooted in offensive security. Penetration testing was the goal, the dream.

Executive Summary

• Challenge: Thread of Doom
• Category: Reverse Engineering
• Flags: NHK26{VirtualProtect_Overwritten}
• Binary: NHK_CrackMe_V3.exe (PE32, x86, 43520 bytes)

Overview

Thread of Doom is a Windows crackme that presents a dialog with a “Demo” button. Clicking the button displays an error: “Tu n’es pas premium ! Prix : 2 BTC”. The goal is to understand the binary’s protection mechanisms and extract the hidden flag.

The flag is XOR-encrypted in memory with a single-byte key (0x55). The binary only decrypts it at runtime when several anti-tampering checks pass, but since both the ciphertext and key are visible in the decompilation, we can extract it statically without running the binary at all.

Introduction

Original post: OffenSkill - Enketo 6.2.1 - Auth-Bypass, SSRF, and XXE Browser Abuse to File Read

This training session was focused on white-box code review, application, and system runtime introspection.
We wanted to work on a JavaScript backend framework and Enketo Express seemed to be a good candidate. The source code is available on GitHub - enketo/enketo-express and the version we assessed was the version 6.2.1, built with the official Dockerfiles.

Enketo is a cross platform software used to (quoting):

Executive Summary

• Challenge: HalCrypto
• Category: Web Security
• Vulnerability: JWT validation bypass via URL confusion with @ symbol
• Impact: Authentication bypass leading to admin access
• Flag: HTB{r3d1r3c73d_70_my_s3cr37s}

Vulnerability Overview

Attack Flow Diagram

Source-to-Sink Analysis

1. Entry Point - JWT Authentication (Source)

The vulnerability starts when the AuthMiddleware processes JWT tokens:

// middleware/AuthMiddleware.js:5-34
module.exports = async (req, res, next) => {
    try {
        if (req.cookies.session === undefined) {
            if (!req.is('application/json')) return res.redirect('/');
            return res.status(401).send(response('Authentication required!'));
        }
        return JWTHelper.getHeader(req.cookies.session)
            .then(header => {
                if (header.jku && header.kid) {
                    // VULNERABILITY: Weak URL validation using lastIndexOf
                    if (header.jku.lastIndexOf(config.AUTH_PROVIDER, 0) !== 0) {
                        return res.status(500).send(response('The JWKS endpoint is not from localhost!'));
                    }
                    return JWTHelper.getPublicKey(header.jku, header.kid)
                        .then(pubkey => {
                            return JWTHelper.verify(req.cookies.session, pubkey)
                                .then(data => {
                                    req.user = data.user;
                                    return next();
                                })
                                .catch(() => res.status(403).send(response('Authentication token could not be verified!')));
                        })
                        .catch(() => res.redirect('/logout'));
                }
                return res.status(500).send(response('Missing required claims in JWT!'));
            })
            .catch(err => res.status(500).send(response("Invalid session token supplied!")));
    } catch (e) {
        return res.status(500).send(response(e.toString()));
    }
}

2. The Vulnerable Validation - String-based URL Check

The critical vulnerability lies in line 14 of AuthMiddleware.js:

Executive Summary

• Challenge: PageOneHTML
• Category: Web Security
• Vulnerability: Server-Side Request Forgery (SSRF) via gopher:// protocol
• Impact: Access to internal API endpoint leading to flag disclosure
• Flags:
  • Local: HTB{f4k3_fl4g_f0r_t3st1ng}
  • Remote: HTB{l1bcurL_pla7h0r4_0f_pr0tocOl5}

Source-to-Sink Analysis

1. Entry Point - User Input (Source)

The vulnerability starts at /api/convert endpoint which accepts user-controlled markdown content:

// routes/index.js:15-28
router.post('/api/convert', async (req, res) => {
    const { markdown_content, port_images } = req.body;  // User input

    if (markdown_content) {
        html = MDHelper.makeHtml(markdown_content);      // Convert MD to HTML
        if (port_images) {                               // If port_images is true
            return ImageConverter.PortImages(html)       // Process images
                .then(newHTML => res.json({ content: newHTML }))
                .catch(() => res.json({ content: html }));
        }
        return res.json({ content: html });
    }
    return res.status(403).send(response('Missing parameters!'));
});

2. Image Processing - Protocol Confusion

The ImageConverter extracts all <img> tags and processes their src attributes:

When I first started developing tools for source code auditing, my primary need was to track tainted data flows through complex codebases during manual code reviews. Initially, I turned to Tree-Sitter, which proved excellent for single-file analysis with its fast, incremental parsing capabilities. However, as I scaled to larger codebases with complex cross-file dependencies and data flows, Tree-Sitter’s AST-only approach became limiting. The challenge wasn’t just parsing individual files. It was understanding how data flows between functions, across modules, and through various execution paths during thorough manual security assessments.

Topics covered

This post explores the evolution from manual code review to automated security testing, covering:

• The reality of manual code review and its limitations
• Understanding vulnerabilities vs weaknesses
• How SAST tools work under the hood
• Taint analysis and data flow tracking
• Sink-to-source vs source-to-sink methodologies
• Mitigation strategies: whitelisting vs blacklisting
• Dealing with false positives in practice
• Choosing and implementing SAST tools at scale
• The complementary relationship between manual and automated testing

It’s 3 AM. You’re on your fifth cup of coffee, eyes bloodshot, staring at line 2,847 of a 10,000-line pull request. Somewhere in this maze of curly braces and semicolons lurks a SQL injection vulnerability that could bring down your entire application. Welcome to the glamorous world of manual code review!