

0 / 2 embers
0 / 3000 xp
click for more info
Complete a lesson to start your streak
click for more info
Still calibrating
click for more info
Not enough gems
Cost: 6 gems
1: Collecting Page Data
incomplete
2: JSON Report
incomplete
3: Submit the Link to Your Repository!
incomplete
This lesson's interactive features are locked, please to keep using them
We're almost done! Our web crawler now extracts rich data from every page and stores it efficiently in an object. Let's export it to JSON so it's easy to read and share.
For example, one page record in that object might look like this:
export const pageData: Record<string, ExtractedPageData> = {
"learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001":
{
url: "https://learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001/",
heading: "Ashenfang Longsword",
first_paragraph:
"A balanced battlefield blade with a smoldering fuller and leather-wrapped grip.",
outgoing_links: [
"https://learnwebscraping.dev/practice/ecommerce/",
"https://learnwebscraping.dev/practice/ecommerce/categories/",
"https://learnwebscraping.dev/practice/ecommerce/categories/longswords/",
],
image_urls: [
"https://learnwebscraping.dev/images/catalog/longswords.svg",
],
},
};
We want to create a report.json file containing a sorted array of all page records.
export function writeJSONReport(
pageData: Record<string, ExtractedPageData>,
filename = "report.json",
): void;
pageData is the object returned by your crawlerfilename is the JSON file to create (defaults to "report.json")url for deterministic outputJSON.stringify using 2-space indentationfs.writeFileSyncHere are some tips:
const sorted = Object.values(pageData).sort((a, b) => a.url.localeCompare(b.url))JSON.stringify(sorted, null, 2)path.resolve(process.cwd(), filename)writeJSONReport from ./reportwriteJSONReport(pages, "report.json") after crawling completesreport.json is created after running the crawlerRun and submit the CLI tests.