Progress

Tracking Progress Towards Becoming the Best Search Engine in the World #

We are developing a metric to describe how far along we are towards becoming the best search engine in the world. Whilst this will never be perfect, it is useful as a motivating factor, since progress by this metric can always be achieved by taking concrete steps. It also allows us to explain to people the enormity of the task we are undertaking, how far we have come, and how far we still have to go.

TLDR; we are 1.2154% complete.

Philosophy and methodology #

We generally prefer goals that are directly attainable, for example, “writing a blog post that meets our criteria” instead of “getting 1,000 views on a blog post” since the former is under our control whereas the latter depends on a host of other conditions outside our control. We will also have goals that are outside of our direct control where it makes sense.

The numbers we set as our goals are arbitrary, but should move us in the right direction. The “correct” number is the number we expect to reach once we achieve our goal of being the best search engine in the world. Of course, we don’t know what that is, but a guess is fine for now. We will sometimes use a benchmark by referring to metrics that are available for other organisations that have reached a level that is in some way comparable to what we are trying to achieve.

Progress is tracked by “points” assigned by making progress towards a goal. Once points are assigned, they cannot be taken away, however new goals can be added. This means that the percentage complete value can decrease with time as new values are added, but the score does not decrease.

Version 0.1 (January 2023) #

The criteria for each of these to be satisfactory and count towards the goal is still to be documented. For now use common sense, e.g. a ten word blog post will not count.

Technology #

Git commits:

  • Goal: 100,000 across all repos
  • Points per commit: 1
  • Max total points: 100,000
  • Benchmarks: MediaWiki - 114,000 commits, Elasticsearch - 67,000 commits
  • Current: 225 (225 points)

Number of pages crawled per day:

  • Goal: 3 billion
  • Points per million pages: 50
  • Max total points: 150,000
  • Benchmark: Google apparently indexed around 100 billion pages at the time of writing, and we need to refresh our index at least once a month.
  • Current: 1 million (50 points)

NDCG score in our offline evaluation:

  • Goal: 80%
  • Points per percentage point: 1,000
  • Max total points: 80,000
  • Benchmark: our dataset is derived from Bing queries and results
  • Current: 10% (10,000 points)

Community #

Blog posts:

  • Goal: 250
  • Points per post: 100
  • Max total points: 25,000
  • Benchmarks: None, this is based on roughly one post a week for five years
  • Current: 2 (200 points)

Videos:

  • Goal: 1000
  • Points per video: 100
  • Max total points: 100,000
  • Benchmarks: Analysis by TubeBuddy (source), average number of videos on channels with 100,000 to 1,000,000 subscribers
  • Current: 6 (600 points)

Number of volunteers:

  • Goal: 100,000 active volunteers each month
  • Points per volunteer: 1
  • Max total points: 100,000
  • Benchmarks: Number of Wikipedia editors: 126,000 active per month
  • Current: 26 (based off Firefox extension stats) (26 points)

Organisation #

Number of employees in the parent organisation:

  • Goal: 20
  • Points per employee: 5,000
  • Max total points: 100,000
  • Benchmarks: None
  • Current: None

Incorporation goals:

  • Incorporate the main organisation in the UK as a non-profit (20,000 points)
  • Get registered charity status in the UK (20,000 points)
  • Incorporate a non-profit in the US (20,000 points)
  • Max total points: 60,000
  • Current: None

Number of officially affiliated organisations:

Number of commits to The Book of Mwmbl:

  • Goal: 10,000
  • Points per commit: 10
  • Max total points: 100,000
  • Current: 20 (200 points)

Grand totals #

  • Current points TOTAL: 11,121
  • Maximum points TOTAL: 915,000
  • Percentage complete: 1.2154%