This lesson's interactive features are locked, please to keep using them

Extract Links and Images

We need to extract links from the HTML both the links people can click AND the images that are displayed. This gives us a complete picture of what resources each page references.

For example, this HTML page has both a link and an image:

<html>
  <body>
    <a href="https://crawler-test.com">Go to Boot.dev</a>
    <img src="/logo.png" alt="Boot.dev Logo" />
  </body>
</html>

We need to extract both https://crawler-test.com (from the link) and /logo.png (from the image).

Assignment

Write tests for a new function called getURLsFromHTML. Here's the signature:

function getURLsFromHTML(html: string, baseURL: string): string[];

html is an HTML string
baseURL is the root URL of the website we're crawling. This will allow us to rewrite relative URLs into absolute URLs.
It returns an un-normalized list of all the URLs found within the HTML, and an error if one occurs.

In your tests, make sure that:

relative URLs are converted to absolute URLs
you find all the <a> tags in a body of HTML

Here's one example test case to give you an idea:

test("getURLsFromHTML absolute", () => {
  const inputURL = "https://crawler-test.com";
  const inputBody = `<html><body><a href="/path/one"><span>Boot.dev</span></a></body></html>`;

  const actual = getURLsFromHTML(inputBody, inputURL);
  const expected = ["https://crawler-test.com/path/one"];

  expect(actual).toEqual(expected);
});

Implement the getURLsFromHTML function. Here are a few hints you may find useful:

querySelectorAll() returns all elements that match the given selector.
You can use the .get() method to retrieve the value of a specific attribute.
getAttribute() returns element's attribute or null if there is no such attribute.
In HTML, "anchor" elements are links. With their href attribute containing the actual URL. e.g:

<a href="https://www.boot.dev">Learn Backend Development</a>

Write tests for a new function called getImagesFromHTML. Here's the signature:

function getImagesFromHTML(html: string, baseURL: string): string[];

html is an HTML string
baseURL is the root URL of the website we're crawling. This will allow us to rewrite relative URLs into absolute URLs.
It returns an un-normalized list of all the image URLs found within the HTML, and an error if one occurs.

In your tests, make sure:

relative URLs are converted to absolute URLs
Handle cases where attributes might be missing

Here's one example test case to give you an idea:

test("getImagesFromHTML relative", () => {
  const inputURL = "https://crawler-test.com";
  const inputBody = `<html><body><img src="/logo.png" alt="Logo"></body></html>`;

  const actual = getImagesFromHTML(inputBody, inputURL);
  const expected = ["https://crawler-test.com/logo.png"];

  expect(actual).toEqual(expected);
});

Implement the getImagesFromHTML function.
Make sure you have at least 3 test cases for each function you're testing.

Run and submit the CLI tests from the root of your module.