We're sorry but this app doesn't work properly without JavaScript enabled. Please enable it to continue.

This lesson's interactive features are locked, please to keep using them

Welcome

In this project, we'll be building a Web Crawler in TypeScript using Node.js! To rank well in Google Search, websites need to internally link pages one to another. For example, my blog post about the benefits of cheap haircuts should probably have a link to my other blog post about the best places to get haircuts.

We're going to write a TypeScript application that generates an "internal links" report for any website on the internet by crawling each page of the site.

for Windows Users

If you try to complete this course without WSL 2 installed on Windows, you're gonna have a bad time.

Learning Goals
  • Get hands-on practice with local TypeScript development and tooling
  • Practice making HTTP requests in TypeScript
  • Learn how to parse HTML
  • Practice unit testing

TypeScript Setup

Before we dive into the project, let's make sure you are all set up properly. You will need:

  1. A code editor. I use VS code, though you can use whatever you're comfortable with.
  2. The Boot.dev CLI installed, and you'll need to be logged in.
  3. NVM installed.

Installing Node.js With NVM

I recommend NVM because it allows you to have multiple installations of Node.js on the same machine. After installing NVM, add an .nvmrc file to the root of your project directory that contains a snippet of text:

22.15.0

This allows you to simply type nvm use in your CLI while in the root of your project to activate the correct version of node! You may get an installation command to run if you don't yet have that version of node, but it's just another one-liner.

Assignment

Check to make sure you've activated the correct version of node by typing:

node --version
# Prints: v22.15.0

Submit the CLI tests. There's no penalty on failure for this lesson.