We're sorry but this app doesn't work properly without JavaScript enabled. Please enable it to continue.

This lesson's interactive features are locked, please to keep using them

Welcome

In this project, we'll build a Web Crawler in Golang! To rank well in Google Search, websites need to internally link pages one to another. For example, a blog post about the benefits of haircuts should probably link to my post about the best places to get haircuts.

We're going to write a Golang CLI application that generates an "internal links" report for any website on the internet by crawling each page of the site.

for Windows Users

If you try to complete this course without WSL 2 installed on Windows, you're gonna have a bad time.

Learning Goals

  • Get hands-on practice with local Go development and tooling

  • Practice making HTTP requests in Go

  • Learn how to parse HTML with Golang

  • Practice unit testing

Setup

Before we dive into the project, let's make sure you are all set up properly. You will need:

  1. The latest Go toolchain installed.

  2. If you're using VS Code, I recommend installing the Go extension. It's not required, but it makes working with Go a lot easier.

  3. You will need the Boot.dev CLI installed, and you'll need to be logged in.

Assignment

Type go version to make sure that you have Go installed.

Submit the CLI tests. There's no penalty on failure for this lesson.