Cheerio: build a source-linked page brief for an AI worker

Give the worker a defined input

Before asking an AI worker to summarize a service page, decide what it should receive. A full page can mix the service description with navigation, scripts, unrelated sidebars and footer text. A small extraction contract makes the input easier to inspect and gives reviewers a source to check.

This AI-assisted editorial guide builds that contract with Cheerio and ten local checks. The HTML and example.com URLs are invented fixtures; the script fetches no website and calls no AI service. It demonstrates extraction behavior, including a hidden-text negative control, rather than claiming a complete crawler or a verified business summary.

1. Pin the package used in the example

The official cheeriojs/cheerio project publishes cheerio under the MIT license. We verified version 1.2.0 and ran it with Node.js 24.19.0 on October 24, 2026. This pinned package declares Node.js 20.18.1 or later; use its own metadata when reproducing the example, because current documentation may describe a newer development state. [1]

Use these PowerShell commands in a new scratch directory. Save both JavaScript blocks below, in order, as page-brief.mjs before the last command. Installation requires internet access. Running the example needs no credentials and prints only synthetic fixture data.

mkdir page-brief-demo
cd page-brief-demo
npm install --ignore-scripts "cheerio@1.2.0"
# Save both JavaScript blocks below as page-brief.mjs first.
node page-brief.mjs

2. Extract a deliberately narrow brief

Cheerio’s load method parses a markup string into a queryable tree. This helper uses the first main element, selects paragraphs and reads the title and first heading. A missing main stops extraction; silently falling back to the entire page could turn a layout change into a different input contract. [2]

The cleanup removes script, style, nav and aside nodes inside that selection before reading paragraph text. Cheerio’s text method includes descendant text, including script and style source when present, while remove deletes selected nodes and their children. These fixed selectors are our fixture policy, not a general content-recognition algorithm. [3]

The link policy resolves raw href values against the supplied source URL and retains only same-origin HTTPS links without embedded credentials. It removes fragments and returns labels and addresses; it neither visits links nor authorizes future requests. The 100,000-byte ceiling is an example input bound to tune for your own workflow.

import assert from 'node:assert/strict';
import * as cheerio from 'cheerio';

const clean = text => text.replace(/\s+/g, ' ').trim();
function pageBrief(html, source) {
  if (Buffer.byteLength(html, 'utf8') > 100_000) {
    throw new Error('Markup exceeds fixture limit');
  }
  const base = new URL(source);
  const $ = cheerio.load(html);
  const main = $('main').first();
  if (!main.length) throw new Error('Missing main: review manually');
  main.find('script, style, nav, aside').remove();
  const paragraphs = main.find('p').toArray()
    .map(element => clean($(element).text())).filter(Boolean);
  const links = main.find('a[href]').toArray().flatMap(element => {
    try {
      const url = new URL($(element).attr('href'), base);
      if (url.protocol !== 'https:' || url.origin !== base.origin
        || url.username || url.password) return [];
      url.hash = '';
      return [{ label: clean($(element).text()), url: url.href }];
    } catch { return []; }
  });
  return {
    source, title: clean($('title').first().text()),
    heading: clean(main.find('h1').first().text()),
    paragraphs, links
  };
}

3. Check the output and its failure cases

Append this block and run the file. The checks cover source and headings, entity decoding, whitespace, boilerplate, script and style exclusion, link selection, a missing main, oversized input, JavaScript-generated content and hidden text.

The last two checks intentionally expect different outcomes. A script that would create a paragraph in a browser does not add that paragraph here. A paragraph already present with the hidden attribute remains in the extracted list. Keeping both cases visible avoids mistaking an HTML parser for a rendered-page inspection.

// Invented HTML and reserved example domains; no page is fetched.
const source = 'https://example.com/services/';
const html = `<!doctype html><html><head>
<title>Automation services | Example</title></head><body>
<header><p>HEADER_MARKER</p></header>
<main><nav><p>NAV_MARKER</p></nav>
<h1>Workflow support</h1>
<p>Map &amp; review one workflow.</p>
<p> Bring  one
  process. </p>
<p hidden>HIDDEN_MARKER</p>
<script>const value = 'SCRIPT_MARKER';</script>
<style>.example { --label: STYLE_MARKER; }</style>
<a href="/contact#start">Contact</a>
<a href="mailto:demo@example.com">Email</a>
<a href="javascript:void(0)">Action</a>
<a href="https://example.net/">External</a>
</main><footer><p>FOOTER_MARKER</p></footer></body></html>`;
const brief = pageBrief(html, source);
const serialized = JSON.stringify(brief);
const cases = [
  ['source and headings', () => {
    assert.equal(brief.source, source);
    assert.equal(brief.title, 'Automation services | Example');
    assert.equal(brief.heading, 'Workflow support');
  }],
  ['decoded entity', () => assert.equal(
    brief.paragraphs[0], 'Map & review one workflow.')],
  ['normalized whitespace', () => assert.equal(
    brief.paragraphs[1], 'Bring one process.')],
  ['boilerplate excluded', () => {
    for (const word of ['HEADER_MARKER', 'NAV_MARKER', 'FOOTER_MARKER'])
      assert(!serialized.includes(word));
  }],
  ['script and style excluded', () => {
    assert(!serialized.includes('SCRIPT_MARKER'));
    assert(!serialized.includes('STYLE_MARKER'));
  }],
  ['only selected links', () => assert.deepEqual(brief.links, [
    { label: 'Contact', url: 'https://example.com/contact' }
  ])],
  ['missing main stops extraction', () => assert.throws(
    () => pageBrief('<h1>No main</h1>', source), /Missing main/)],
  ['oversized input stops extraction', () => assert.throws(
    () => pageBrief('x'.repeat(100_001), source), /fixture limit/)],
  ['JavaScript content absent', () => {
    const dynamic = pageBrief(`<main><div id="app"></div>
      <script>document.querySelector('#app').innerHTML =
      '<p>DYNAMIC_MARKER</p>';</script></main>`, source);
    assert.deepEqual(dynamic.paragraphs, []);
  }],
  ['hidden-text negative control', () => {
    assert(brief.paragraphs.includes('HIDDEN_MARKER'));
  }]
];
for (const [name, check] of cases) {
  check(); console.log('PASS:', name);
}
console.log(JSON.stringify(brief, null, 2));

4. Interpret the ten observed results

All ten checks passed locally. The brief retained the fixture source, title and heading. It decoded the ampersand, normalized whitespace and returned the two ordinary paragraphs plus HIDDEN_MARKER. Header, navigation, footer, script and style markers were absent. Only the resolved contact link remained.

Missing-main and oversized inputs stopped with the expected errors. The JavaScript fixture produced no paragraph. These outcomes apply to this small fixture and chosen selectors; they do not establish extraction quality across arbitrary websites.

Cheerio does not execute JavaScript, apply CSS or visually render a page. The hidden-text result demonstrates one practical consequence: markup that a visitor cannot see may still enter a text extraction. Check a representative rendered page when visibility matters, and use an appropriate rendering step when required content exists only after JavaScript runs. [4]

5. Keep provenance and authority separate

In a real pipeline, store the validated final source URL, retrieval time and an input snapshot or digest alongside the extracted brief. Review missing fields and changed layouts before passing the result to a summarizer. A URL preserved in JSON helps trace an input; it does not verify the page’s claims.

Extracted text can still contain instructions written by the page author. Treat it as source material for the worker, not permission to send messages, change customer records or execute tools. Removing script tags does not make ordinary prose trustworthy or prevent prompt injection.

Cheerio is also not an HTML sanitizer. If you display this brief in a dashboard, use a text-safe rendering path rather than inserting returned strings as raw HTML. If you later add network fetching, validate destinations and redirects separately and enforce request and response limits. The documentation calls out both untrusted markup and user-supplied fetch destinations as boundaries to handle explicitly. [5]

Sources & further reading

1. Cheerio 1.2.0 package metadata, runtime requirement and license
2. Cheerio documentation: loading an HTML document
3. Cheerio documentation: reading text and removing elements
4. Cheerio introduction: parser behavior and browser limitations
5. Cheerio security: untrusted markup, text sinks and network destinations

Turn business information into a reviewable AI input

Sociologix can help connect approved sources to AI research, customer-care and workflow tools, with extraction checks and human review at the right handoffs. Bring one source page and the decision your worker needs to support.