CPMagical CP Library

Problem solution · C++

Web Crawler

Web Crawler: a C++ solution using breadth-first search. Learn the idea, check the complexity, and read the full code, with credit to walkccc LeetCode Solutions.

Technique
Breadth-first search
Source
walkccc LeetCode Solutions
Length
40 lines
Start with the idea.

Try the problem first. If you get stuck, read the approach below, then write your own solution. The full code is at the bottom.

Approach

Breadth-first search

For Web Crawler, the implementation explores reachable states in layers, which is the standard shape for unweighted shortest paths and minimum-step transitions.

  1. Model each valid configuration as a state and each legal move as an edge.
  2. Seed the queue with the starting state and mark it immediately.
  3. Expand each state once, recording distance or reachability for unseen neighbours.

Code notes

  • 40 lines of C++ from the credited upstream file 1236.cpp.
  • The implementation visibly relies on sequence storage, hash lookup, work queue.
  • 2 loop blocks detected.

Complexity

Verify that each state and transition is processed only a bounded number of times; that determines the traversal cost.

Check the problem constraints before deciding whether this complexity will pass.

Source

Code and credit

This code comes from walkccc LeetCode Solutions by P.-Y. Chen (walkccc) and is used under the MIT licence.

Open problem ↗Original source ↗
Full codeWeb Crawler · C++C++
Use this to learn the idea, then write your own version.
/** * // This is the HtmlParser's API interface. * // You should not implement it, or speculate about its implementation * class HtmlParser { *  public: *   vector<string> getUrls(string url); * }; */ class Solution { public:  vector<string> crawl(string startUrl, HtmlParser htmlParser) {    queue<string> q{{startUrl}};    unordered_set<string> seen{{startUrl}};    const string& hostname = getHostname(startUrl);     while (!q.empty()) {      const string currUrl = q.front();      q.pop();      for (const string& url : htmlParser.getUrls(currUrl)) {        if (seen.contains(url))          continue;        if (url.find(hostname) != string::npos) {          q.push(url);          seen.insert(url);        }      }    }     return {seen.begin(), seen.end()};  }  private:  string getHostname(const string& url) {    const int firstSlash = url.find_first_of('/');    const int thirdSlash = url.find_first_of('/', firstSlash + 2);    return url.substr(firstSlash + 2, thirdSlash - firstSlash - 2);  }};

Keep practising

Did this explanation save you time? I'm a Grade 11 student building this free library to make difficult algorithms easier to understand.

Buy me a coffee ↗