Main

Let's get to work on the actual application! Our program takes as a command line argument the website URL we want to crawl (or at least the page we want to start from).

go build -o crawler && ./crawler BASE_URL

go run . BASE_URL

Where BASE_URL is the root URL of the website we want to crawl. For example, you can test with my blog if you want: https://wagslane.dev.

You will need to process command line arguments for this step.

All your main function should do at this point is:

If the number of CLI arguments is less than 1, print "no website provided" and exit with code 1.
If the number of CLI arguments is more than 1, print "too many arguments provided" and exit with code 1.
If we have exactly one CLI argument, it's the BASE_URL, so print a message letting the user know the crawler is starting at that URL: starting crawl of: BASE_URL, where BASE_URL is the URL provided as the CLI argument.

Test by Running

This is a good example of some code that's a bit weird to test with unit tests. Just run the dang thing with too few, just the right amount, and too many arguments to ensure it behaves as expected.

Run and submit the CLI tests from the root of your main package.

Tips

I used os.Exit to exit the program with a status code of 1. This is a good way to signal to the operating system that something went wrong.
os.Args holds the CLI arguments, with the first item as the program path and the rest being the actual arguments, os.Args[1:].