The naive extraction, and why it fails on real pages
function extractPageText(): string {
return document.body.innerText;
}
On a clean documentation page this works fine. On a typical news site, document.body.innerText includes the navigation menu, a cookie consent banner, related-article widgets, comment sections, and footer links -- all interleaved with the actual article text, with no structural signal telling your AI feature which part is the content the user actually wants summarized.
This is exactly the "garbage in, garbage out" problem from Course 03's preprocessors
The same lesson from Course 03's article extractor applies here, at DOM level instead of HTTP level: strip out nav, footer, header, ads, and cookie banners before extracting text, and prefer the actual article/main content container when the page has one. A production-grade version of this would use a real readability algorithm; for this course, a heuristic that strips known noise elements and prefers <article>/<main> when present gets you most of the value without an extra dependency.
function extractPageText(): string {
const clone = document.body.cloneNode(true) as HTMLElement;
clone.querySelectorAll('nav, footer, header, script, style, [class*="cookie"], [class*="banner"], [class*="ad-"]').forEach((el) => el.remove());
const mainContent = clone.querySelector('article, main, [role="main"]');
const target = mainContent ?? clone;
return target.innerText.replace(/\s+/g, ' ').trim();
}
Cloning the body before removing elements is deliberate -- mutating the REAL page's DOM to strip elements would visibly break the page the user is looking at, which is exactly the kind of side effect a "just reads the page" extension should never have. Preferring an <article>/<main> container when one exists handles the common case where the page's own HTML already marks its primary content, which is more reliable than any heuristic you'd write yourself.
You test extraction on a single-page app (like a modern web dashboard) and get almost no text back, even though the page is clearly full of content. What's the most likely cause?
Content scripts can run before, at, or after a page's own JavaScript finishes rendering, depending on configuration (document_idle vs document_end vs document_start) -- a SPA that renders its content client-side after initial load can easily have little-to-no text present if your script runs too early. This is worth testing explicitly against at least one JS-heavy page, not just static content sites, precisely because it's a real and common failure mode, not a hypothetical one.