OpenCog Brainwave » AttentionAllocation

Spaced repetition and memory

Joel Pitt — Wed, 09 Jul 2008 00:17:43 +0000

An article in Wired from a while back on Piotr Wozniak (no relation to Steve), a researcher of optimal memory and learning strategies, got me thinking about learning theory and memorization in the context of OpenCog. From the article (emphasis mine):

Long-term memory, the Bjorks said, can be characterized by two components, which they named retrieval strength and storage strength. Retrieval strength measures how likely you are to recall something right now, how close it is to the surface of your mind. Storage strength measures how deeply the memory is rooted. Some memories may have high storage strength but low retrieval strength. Take an old address or phone number. Try to think of it; you may feel that it’s gone. But a single reminder could be enough to restore it for months or years. Conversely, some memories have high retrieval strength but low storage strength. Perhaps you’ve recently been told the names of the children of a new acquaintance. At this moment they may be easily accessible, but they are likely to be utterly forgotten in a few days, and a single repetition a month from now won’t do much to strengthen them at all.

So, in memory studies, they talk of storage strength and retrieval strength. In my observation, retrieval strength is analogous to the distance of an atom’s short term importance to the attentional focus. Atoms just below the attentional focus threshold will be much easier to retrieve, both because there is more chance that they are stored locally and in memory and because they are more likely to be used by mind agents than atom’s with lower short term importance. Storage strength, on the other hand, is related to the long term importance of an atom. Atom’s with very low long term importance are unlikely to persist in the atom space, or more accurately they’ll be preferentially forgotten over atoms with higher long term importance.

One of the problems [with learning] is that the amount of storage strength you gain from practice is inversely correlated with the current retrieval strength. In other words, the harder you have to work to get the right answer, the more the answer is sealed in memory.

Perhaps they’ll be a need to incorporate this. Something that is persistently of use, but only becomes useful at significantly spaced periods of time. Requiring an OpenCog instance to reason about or research the fact again each time would be inefficient if there is some way of recognizing this long term trend. In a way, this is where something like a System Activity Mining agent might come in to play, to data mine such trends, however… I’m personally unsure about whether such an agent will scale to working on an entire atom space, particularly if it’s trying to detect these long term and infrequent trends.

One way to implement the storage of these infrequently but consistently used atoms is to assess the velocity at which short term importance changes when bringing an atom into the attention focus. This velocity would be greater for atoms which are harder to recall since they have further to travel to reach the attentional focus. A higher velocity could then confer more long term importance when an atom entires the attentional focus rather than getting long term importance purely from the stimulus reward system.

New importance spread system

Joel Pitt — Sun, 06 Jul 2008 23:16:34 +0000

Well, I’d noticed it’s been a while since we’ve had a post here and thought I’d rectify it with a brief note of what I’ve been up to. Posts of more substance are on their way I promise!

Recently we’ve been trying to play with some new mechanisms for the spread of Short Term Importance in OpenCog. The initial implementation worked alright, but not sufficiently well to make the Hopfield network emulation perform admirably. Ideally we’d like to have a emulator comparable to traditional Hopfield networks that has the added benefit of effective continuous learning. Now, perhaps this focus on a toy problem is unnecessary, but if we can come up with a continuous learning Hopfield network then they’ll be a scientific paper in there somewhere, which will lend to OpenCog’s credibility in the future.

Initially I implemented an importance spreading mechanism that ensured importance couldn’t spread uphill to atoms with higher importance. This performed somewhat better than the existing system (based on very cursory evaluation), but was still somewhat hacky.

Ben suggested an example analogous to diffusion, which uses a Markov matrix to represent the transition probabilities of STI moving along a Hebbian link from one atom to another. Thus I went and implemented that, which works super effectively when imprinting a single pattern – but doesn’t work so well with more than one. I’m now going to modify how the the transition matrix is constructed based on what atoms are in the attentional focus, which should prevent importance diffusion being the free for all it is at the moment.

And in between doing this Attention Allocation stuff, I’m trying to port an implementation of Probabilistic Logic Networks from Novamente (OpenCog’s benevolent ancestor) to OpenCog. Theoretically that shouldn’t be too hard… but it’s quite complex code in places. Smart pointers to vectors of trees of predicates, oh my!

Hopfield network example

Joel Pitt — Sun, 08 Jun 2008 11:52:37 +0000

As a toy problem for playing/testing/understanding attention allocation in OpenCog, I’ve been emulating the behaviour of a Hopfield network within the OpenCog AtomSpace.

For those not already aware, a Hopfield network is a kind of recurrent neural network that acts as an associative memory. It consist of a number of units linked together. These units store and display the patterns in memory. The patterns are stored in a Hopfield network by adjusting the weights of links between units, based on the association between active units in a given pattern.

A result of this, is that if you give the Hopfield network a partial pattern, that is slightly incorrect, it should still retrieve the complete pattern if it’s in memory. The memorised patterns are in fact minimal energy states for the network (see the wikipedia entry for details).

Using this traditional method, approximately 0.14N patterns can be stored (where N = number of units/nodes) in a fully connected network. However trying to teach a network already loaded with patterns a new one results in aberrant patterns. There are ways of allowing continuous learning, the most well known probably being the Palimpsest scheme, where the weights between units are capped. This reduces the number of storable patterns down to 0.05N however. We theorise, that using the mechanism of attention allocation, we’ll be able to achieve comparable if not improved results.

The process of emulating a Hopfield network is described on the OpenCog wiki.

In my next post, I’ll explain an example of Hopfield network emulation using attention allocation.

A week in Maryland

Joel Pitt — Thu, 22 May 2008 22:17:10 +0000

I’ve been spending the last week at Ben Goertzel’s place to discuss the work I’ve done on attention allocation and to get up to speed on Probabilistic Logic Networks (PLN). There was also an introduction to Relex and RelexToFrame and how PLN will be of use in normalising the Frame representation of a sentence.

The ideas we generated for attention allocation, to improve the performance of the Hopfield network emulator, are as follows:

increase the amount of atom sti and lti significantly to avoid rounding error.
use the same stimulus divided across the entire pattern.
directly imprint the pattern, but balance the +ve and -ve perception nodes. i.e. (the importance of all the nodes not in the pattern + the importance of all the nodes in the pattern == 0)
gradually update the rent, i.e. make it a decaying Util::recent_val

Most of these all address the issue of rent updates (after AtomSpace funds leave homeostatic bounds) significantly disrupting the retrieval process leading to erratic performance.

Unrelated to the Hopfield network, but still to do with attention allocation:

record stimulus applied per mindagent, and then convert to sti/lti after each mindagent runs. this will perhaps require a StimulusToImportanceAgent separate from the ImportanceUpdatingAgent – since the latter collects rent and carries out other per cycle activities. Or the stimulus is converted to sti/lti directly by the AtomSpace.
Inferring Hebbian links from inheritance and other PLN links. The truth value for the inferred Hebbian link will have a wider interval/lower confidence, possibly using the same formula as the inh2mem TruthValue formula. When spreading importance, the truth value, if it isn’t a simple truth value, will be randomly sampled to work out the amount of importance to spread.

The remaining days were devoted to learning aspects of PLN (indefinite probability intervals, quantifiers, count conversion, and inference control), and discussing the involvement of MOSES. I got some guidance being shown through existing PLN code, discussed creating SWIG bindings for the OpenCog code base, and subsequently create a shell that can operate both locally and remotely.

I also met some familiar online identities (from SL4, singularity and openog mailing lists) in the flesh at the DC future salon dinner.

Those who were around and involved with discussions during the week are shown in the below group photo:

From left to right: Eddie Munroe (Novamente VP of Engineering, HoL), Matt Ikle (PLN co-author, GUG), Moshe Looks (MOSES author), Ben Goertzel (SIAI Director of Research, Novamente CEO), Bruce Klein (Novemente president, HLD), Joel Pitt (me, OpenCog researcher, AoE). In front: Crunchkin (Novamente Virtual Pet Advisor). Thanks to Izabella for taking the photo!

(Some of the 3 letter acronyms at the end of the names are made up, I’ll leave you to work which ones and what they stand for

Hebbian Links and Importance Spreading

Joel Pitt — Mon, 19 May 2008 22:28:45 +0000

This post will cover the HebbianLink, and two MindAgents, the ImportanceSpreadingAgent and the HebbianLearningAgent. The former agent spreads Short term importance (STI) along HebbianLinks, while the later updates the HebbianLink truth values.

Let’s look at the HebbianLink shall we?

There are three types: symmetric, asymmetric and inverse. I’ll concentrate on the symmetric HebbianLink here, but there is a paragraph about each of the others on the wiki. HebbianLinks basically indicate which concepts are important at the same time, or to put it another more OpenCog specific way, they indicate which atoms tend to appear in the Attentional focus of the system at the same time. The Truth value of the link indicates the strength of this tendency.

How these HebbianLinks are created is an interesting challenge, because we can’t just add HebbianLinks between all pairs of atoms. To intelligently do this, there are plans for a HebbianMiningAgent that will look at AtomTable activity to determine what new links should be created. However, this has yet to implemented. Currently Hebbian links have to be manually added by the programmer/user (I’ll show how I’ve done this in a later post on Hopfield network emulation using the Attention allocation mechanisms).

Once the HebbianLinks are there however, the HebbianLearningAgent keeps them updated. Each cycle the agent runs, it updates the truth values of all the HebbianLinks by adjusting them slightly to reflect the conjunction of the linked atoms when one of them is in the attentional focus.

The ImportanceSpreadingAgent takes the amount of excess importance that an atom has and spreads it along the HebbianLinks, with more importance being spread along HebbianLinks with higher TruthValues. This excess importance is defined as the amount of STI above the importance spreading threshold and this is usually greater than the attentional focus boundary to ensure that importance spreading doesn’t remove atoms from the attentional focus. This transfer of importance can then result in these destination atoms being elevated into the attentional focus, increasing their chances of being used in inference and other cognitive processes.

Introducing the Importance Updating Agent

Joel Pitt — Fri, 09 May 2008 04:15:50 +0000

The ImportanceUpdatingAgent handles the exchange of currency between atoms. Every thought cycle It pays out wages in the form of Short Term Importance (STI) and Long Term Importance (LTI) to atoms that are currently in use, and also collects rent from an atom’s STI and LTI.

As atoms are used in mind processes they are endowed with stimulus. MindAgents should explicit grant atoms stimulus when they make use of them, although it’s left up to the MindAgent to decide how best to do this.

The ImportanceUpdatingAgent takes this stimulus and pays wages to atoms. So in a sense, the stimulus is a record of how much work the atom has been involved in. The currency of the wages that the ImportanceUpdatingAgent pays to an atom is in the form of both STI and LTI. The amount of STI and LTI conferred is based both on the total stimulus of the atom, and two internal multipliers that indicate the rate that stimulus is converted into STI and LTI. Atoms also have to pay LTI rent to exist in the the AtomSpace and STI rent to exist in the attentional focus.

For example, say atom A1 has 0 STI and 0 LTI, during some cognitive process it is given 10 stimulus. Now it comes time for the ImportanceUpdatingAgent to run. It’s internal conversion rate for stimulus into STI is 10 and for LTI the conversion rate is 3. At this stage a lower conversion rate for LTI is thought to be of more use because we want LTI to respond slower than the short term importance (rent for LTI is also lower, see below), for example, things flit in and out of your attention as you go through the day, but the things you remember long term are the things that you’ve spent more time thinking about. When the ImportanceUpdatingAgent pays wages, it gives the atom A1 100 STI and 30 LTI.

STI rent is charged from atoms that are above the attentional focus boundary and LTI rent for all atoms in the system (conceptually at least, once atoms start being swapped to disk, LTI rent for these atoms might only be charged periodically due to the expense involved in updating them). So, if the attentional focus boundary was set at zero, i.e. all atoms with positive STI are in the attention span of the OpenCog instance, then for our atom A1, it would be charged both STI and LTI rent. If STI was < 0 then atom A1 would not be charged STI rent, but would still be charged LTI rent regardless of the STI or LTI of A1.

Now, since both STI and LTI currency is conserved, the funds that the ImportanceUpdatingAgent uses to pay wages has to come from somewhere. In an ideal situation, this would be balanced by the rent charged to atoms. However, the number of atoms in the AtomSpace and in the attentional focus will be constantly changing. In fact, an OpenCog instance may even have some method of controlling the attentional focus boundary depending on how focussed or quick thought and reaction needs to be. more atoms in the attentional focus mean more atoms to consider during reasoning, although reasoning methods can and do use atoms outside of the attentional focus if the those in attention are insufficient to come up with suitable results. To manage the variable number of atoms in attention and in the Atom Space, the ImportanceUpdatingAgent draws STI and LTI from a pool of funds managed by the Atom Space. These pools have a homeostatic range that the ImportanceUpdatingAgent tries to keep the Atom Space funds within. If, at the end of an update cycle, the pools are outside of this range, then the agent taxes all atoms to bring the pools back to the middle of the range (Note that although it’s called a tax, it can actually result in a refund if the agent has been charging too much tax… just like real life ). The agent then recalculates the optimal rent based on decaying measures of the AtomSpace size and number of atoms in the attentional focus.

Further information will be available on the wiki about the agent, such as the default parameters, and the formulas for updating rent, etc.

Economic Attention Allocation

Joel Pitt — Tue, 06 May 2008 00:21:20 +0000

My task for the past couple of months has been to implement code to manage attention allocation within OpenCog. This process of deciding which atoms in the system are important is useful for a number of reasons, such as memory management and the forgetting process, and guiding the PLN (Probabilistic Logic Network) inference process.

Attention allocation in OpenCog has an economic framework initially conceptualized and prototyped by Ben Goertzel. Atoms have three types of importance currency associated with them:

Short term importance (STI) guides what concepts are in the attentional focus of an OpenCog instance.
Long term importance (LTI) is currently used for indicating what atoms might be worth forgetting or swapped out of memory.
Very long term importance (VLTI) is a flag which essentially indicates whether the atom can be forgotten or not.

Both STI and LTI are conserved throughout the system, and the AtomSpace maintains a pool of funds for paying wages to atoms that assist in achieving system goals. Atoms that are within the attentional focus have rent charged from their STI, and all atoms have rent charged from their LTI.

There are currently 4 MindAgents related to economic attention allocation. I plan to do a post on each, beginning with the ImportanceUpdatingAgent.