

0 / 2 embers
0 / 3000 xp
click for more info
Complete a lesson to start your streak
click for more info
Difficulty: 4
click for more info
Not enough gems
Cost: 6 gems
1: Collecting Page Data
incomplete
2: JSON Report
incomplete
3: Submit the Link to Your Repository!
incomplete
This lesson's interactive features are locked, please to keep using them
We're almost done! Our web crawler now extracts rich data from every page and stores it efficiently in an object. Let's export it to JSON so it's easy to read and share.
For example, one page record in that object might look like this:
export const pageData: Record<string, ExtractedPageData> = {
"learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001":
{
url: "https://learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001/",
heading: "Ashenfang Longsword",
first_paragraph:
"A balanced battlefield blade with a smoldering fuller and leather-wrapped grip.",
outgoing_links: [
"https://learnwebscraping.dev/practice/ecommerce/",
"https://learnwebscraping.dev/practice/ecommerce/categories/",
"https://learnwebscraping.dev/practice/ecommerce/categories/longswords/",
],
image_urls: [
"https://learnwebscraping.dev/images/catalog/longswords.svg",
],
},
};We want to create a report.json file containing a sorted array of all page records.
export function writeJSONReport(
pageData: Record<string, ExtractedPageData>,
filename = "report.json",
): void;pageData is the object returned by your crawler
filename is the JSON file to create (defaults to "report.json")
Sort the pages by url for deterministic output
Serialize the sorted array with JSON.stringify using 2-space indentation
Write the file to disk using fs.writeFileSync
Here are some tips:
Sort: const sorted = Object.values(pageData).sort((a, b) => a.url.localeCompare(b.url))
Serialize: JSON.stringify(sorted, null, 2)
Resolve path: path.resolve(process.cwd(), filename)
Import writeJSONReport from ./report
Call writeJSONReport(pages, "report.json") after crawling completes
Verify report.json is created after running the crawler
Check that it contains a valid JSON array of page objects
Run and submit the CLI tests.