This lesson's interactive features are locked, please to keep using them

JSON Report

We're almost done! Our web crawler now extracts rich data from every page and stores it efficiently in a map. Let's export it to JSON so it's easy to read and share.

For example, one page record in that map might look like this:

pages := map[string]PageData{
    "learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001": {
        URL:            "https://learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001/",
        Heading:        "Ashenfang Longsword",
        FirstParagraph: "A balanced battlefield blade with a smoldering fuller and leather-wrapped grip.",
        OutgoingLinks: []string{
            "https://learnwebscraping.dev/practice/ecommerce/",
            "https://learnwebscraping.dev/practice/ecommerce/categories/",
            "https://learnwebscraping.dev/practice/ecommerce/categories/longswords/",
        },
        ImageURLs: []string{"https://learnwebscraping.dev/images/catalog/longswords.svg"},
    },
}

We want to create a report.json file containing a sorted array of all page records.

Assignment

Add JSON struct tags to PageData in extract_page_data.go:

type PageData struct {
    URL            string   `json:"url"`
    Heading        string   `json:"heading"`
    FirstParagraph string   `json:"first_paragraph"`
    OutgoingLinks  []string `json:"outgoing_links"`
    ImageURLs      []string `json:"image_urls"`
}

Create a new file called json_report.go.
Import Go's built-in encoding/json package:

import "encoding/json"

Create a new function called writeJSONReport. Here's the signature:

func writeJSONReport(pages map[string]PageData, filename string) error

Implement writeJSONReport. It should:
- Sort the map keys for deterministic output
- Build a []PageData slice in sorted order
- Marshal it with json.MarshalIndent using indent=2
- Write the result to disk with os.WriteFile

Here are some tips:

Sort keys: sort.Strings(keys)
Marshal: data, err := json.MarshalIndent(sorted, "", " ")
Write: os.WriteFile(filename, data, 0644)

Update your main.go to use the JSON report:
- Call writeJSONReport(cfg.pages, "report.json") after crawling completes
Test your implementation:
- Verify report.json is created after running the crawler
- Check that it contains a valid JSON array of page objects

Run and submit the CLI tests.