×

What is ContextualWeb and how does it work?

The Memory Indexing Theory

The human brain has a marvelous capacity to record everyday experiences and retrieve these memories much later in all manner of situations.

Recent studies in Neuroscience and Brain Science have considerably advanced our understanding of the distinct roles of the brain areas responsible for the encoding and retrieval of memories.

An enormous amount of research has led to a general understanding of how the brain forms memory traces for new events. Substantial progress has been made in discovering how the brain organizes new memories into networks of knowledge that can be accessed flexibly.

ContextualWeb is an implementation of the memory system of the brain: It indexes web pages in a way similar to the way the brain indexes memories, forming a brain-like network of knowledge based on the World Wide Web.

There is a striking difference between existing implementations of search engines based on the inverted indexing scheme (like Google or Bing) and the new brain-like model: The intrinsic inefficiency of the inverted indexing scheme is bypassed!

Inverted indexing (for each word, hold a list of the documents in which it appears) requires high maintenance costs on infrastructures due to the overwhelming computing power necessary for supporting massive list intersections via MapReduce jobs. ContextualWeb indexes and retrieves webpages without list intersections!

The Technology

By mimicking the brain we can build search engines which require storage and computation power of orders of magnitude lower than any other known search engine.

ContextualWeb runs on a small cluster of commodity servers with a ridiculously low maintenance costs of only a few hundred of dollars a month.

The advantage of the new model is twofold: Significantly smaller index size (as the index is a cognitive compact representation of the memory) and fast search query response (as the retrieval of a memory is a sequence of pattern activation of low computational cost). This makes the brain-like model extremely economic and highly scalable for handling large number of requests simultaneously.

As another example, the technology allows us to use a disk-based index architecture while keeping highly competitive performance.

When comparing with other search engines, such as Google and Bing, we should take into account the fact that we are comparing a new search engine running on only a few hundred of dollars a month with search engines running on hundreds of millions of dollars a month for more than two decades…

Our ultimate goal is to exploit the recent technological advances to bring down the costs of the search business: Making large scale indexing systems cheap and easy, so that eventually every company (big and small) will be able to keep its own large-scale Googe-like search engine.

Peter Thiel (PayPal cofounder and an early investor in Facebook) described it in the best way:

“If we’re going to have a real transformation in search, you have to try to be building technologies that will enable you to take something like 10,000 servers that are used for search today and do it with one hundred servers.”

How Does It Work?

ContextualWeb is an implementation of the Hippocampal Memory Indexing Theory (1986, 2007, 2013-present).

The theory describes the hippocampus and the prefrontal cortex which are the parts of the brain that support rapid encoding of new information and consolidation and organization of the memory networks.

It works completely different from any known search engine: There are no inverted indexing, BrainRank, NLP (natural language processing) or standard Machine Learning or Deep Learning models here…

Every event experienced in our life is recorded forming a memory trace and indexed forming a memory index for fast future recollection.

The content of our experience is stored in the neocortex. The index is stored in the hippocampus.

The inverted indexing scheme is replaced in the brain-like model by the Hippocampus Index, whose role is to form and retrieve memories. The Hippocampus Index represents direct associations and co-occurring patterns that compose the experienced event.

It is a an auto-associative network that is continually altering, and supports features like pattern separation of overlapping neocortical inputs at encoding, and pattern completion of partial neocortical input at retrieval.

Google’s BrainRank is replaced in the brain-like model by the prefrontal cortex, whose role is to select the appropriate memory for the context. It accumulates features of related memories that compose the context of a set of connected experiences. When subsequently cued to a context (search query), the prefrontal cortex is viewed as biasing the retrieval of context-appropriate memories in the hippocampus as well as other brain areas.

NLP has no role in the brain-like model, as after all, the brain’s “natural order” is independent of language.

The learning models used by the brain-like model are intrinsically different from the standard techniques we are used to in Machine Learning and Deep Learning (the latter simply violates basic biological facts). The brain heavily uses one-shot learning models for rapid encoding of memories: Events are experienced all the time and are indexed when occurred.

References:

A great blog on memory indexing:

  1. https://camphippo.wordpress.com/…

Academic papers

  1. The Hippocampal Memory Indexing Theory.
  2. The Hippocampal Indexing Theory and Episodic Memory: Updating the Index.
  3. A Theory of Hippocampal Function in Memory.
  4. The Form and Function of Hippocampal Context Representations.
  5. The CA3 network as a memory store for spatial representations.
  6. Heteroassociative storage of hippocampal pattern sequences in the CA3 subregion.
  7. Hippocampal Representation of Related and Opposing Memories Develop within Distinct, Hierarchically Organized Neural Schemas.
  8. Interplay of Hippocampus and Prefrontal Cortex in Memory.
  9. Learning by Example in the Hippocampus.
  10. Computational Analysis of the Role of the Hippocampus in Memory.