We're sorry but this app doesn't work properly without JavaScript enabled. Please enable it to continue.

This lesson's interactive features are locked, please to keep using them

JSON Report

We're almost done! Our web crawler now extracts rich data from every page and stores it efficiently in an object. Let's export it to JSON so it's easy to read and share.

For example, one page record in that object might look like this:

export const pageData: Record<string, ExtractedPageData> = {
  "learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001":
    {
      url: "https://learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001/",
      heading: "Ashenfang Longsword",
      first_paragraph:
        "A balanced battlefield blade with a smoldering fuller and leather-wrapped grip.",
      outgoing_links: [
        "https://learnwebscraping.dev/practice/ecommerce/",
        "https://learnwebscraping.dev/practice/ecommerce/categories/",
        "https://learnwebscraping.dev/practice/ecommerce/categories/longswords/",
      ],
      image_urls: [
        "https://learnwebscraping.dev/images/catalog/longswords.svg",
      ],
    },
};

We want to create a report.json file containing a sorted array of all page records.

Assignment

  1. export function writeJSONReport(
      pageData: Record<string, ExtractedPageData>,
      filename = "report.json",
    ): void;
    
    • pageData is the object returned by your crawler
    • filename is the JSON file to create (defaults to "report.json")
    • Sort the pages by url for deterministic output
    • Serialize the sorted array with JSON.stringify using 2-space indentation
    • Write the file to disk using fs.writeFileSync

Here are some tips:

  • Sort: const sorted = Object.values(pageData).sort((a, b) => a.url.localeCompare(b.url))
  • Serialize: JSON.stringify(sorted, null, 2)
  • Resolve path: path.resolve(process.cwd(), filename)
    • Import writeJSONReport from ./report
    • Call writeJSONReport(pages, "report.json") after crawling completes
    • Verify report.json is created after running the crawler
    • Check that it contains a valid JSON array of page objects

Run and submit the CLI tests.