This lesson's interactive features are locked, please to keep using them

Structure Page Data

So far we've built functions to help us normalize URLs and extract links/text from HTML. Now let's structure that data in a way that's much more usable.

Assignment

Write tests for a new function called extractPageData. Here's the signature:

func extractPageData(html, pageURL string) PageData {

html is an HTML string
pageURL is the absolute URL of the page (used for converting relative URLs)
It returns a struct with fields: URL, Heading, FirstParagraph, OutgoingLinks, ImageURLs

Here's one example test case to get you started:

func TestExtractPageData(t *testing.T) {
	inputURL := "https://crawler-test.com"
	inputBody := `<html><body>
        <h1>Test Title</h1>
        <p>This is the first paragraph.</p>
        <a href="/link1">Link 1</a>
        <img src="/image1.jpg" alt="Image 1">
    </body></html>`

	actual := extractPageData(inputBody, inputURL)

	expected := PageData{
		URL:             "https://crawler-test.com",
		Heading:         "Test Title",
		FirstParagraph: "This is the first paragraph.",
		OutgoingLinks:  []string{"https://crawler-test.com/link1"},
		ImageURLs:      []string{"https://crawler-test.com/image1.jpg"},
	}

	if !reflect.DeepEqual(actual, expected) {
		t.Errorf("expected %+v, got %+v", expected, actual)
	}
}

Implement the extractPageData function. You have already built all the logic for this function, it just needs to return the data you're already gathering in a struct!

Run and submit the CLI tests to verify your extraction logic works correctly!