First up is a video demonstrating a human controlled player navigating and interacting with the world. This world is built in Unity3D, so eventually we’ll be able to put this environment online, make it an iPad app, or whatever else, and let you guys interact and teach OpenCog directly.

The things to note are that it’s based off a minecraft-like environment, which means the player and AI will be able to modify the terrain and build things. Other objects can also be moved around and interacted with. We’ve got a very flexible action system that allows new action types to easily be added, and OpenCog will be able to learn the causal effect of executing these previously unknown actions.

Next is a demonstration of 3D pathfinding (implemented by Troy Deheng), with the satisfaction of a single “demand” for energy by consuming the battery object. In addition, it also shows simple planning by asking for a battery from the player if OpenCog can’t find one in the world. After asking this, the player spawns a new battery with a switch, and OpenCog efficiently detects this correlation between using the switch and new batteries appearing. In essence learning a new behaviour to satisfy it’s goals.

Jared Wigmore has been working on Fishgram, which is a frequent sub-hypergraph miner that detects patterns in the AtomSpace (and so by extension, also in the virtual world). This is the component used to detect that pushing a switch creates a new battery.

Last is a shorter video showing demands beyond just energy. These are analogous to goals, but not quite. They are slightly different in that they impel the system towards certain goals. The new demand in this video is one for integrity, which is roughly analogous to fatigue/health. In the video, the house is known to satisfy this demand and increase integrity, but it oscillates between which demand is most important. Integrity then Energy, then back again. Zhenhua Cai has already added a couple of more demands: Competence and Certainty.

Thanks to Cord Krohn for putting the videos together, as well as doing environment and character design and thanks to Michael Jia for 3d modelling and art assets.

In early April, Ben visited Hong Kong to give guidance to the long-term plan for the HK project. We had many interesting discussions with plans for how to tackle a number of problems, including resolving issues in the language pipeline, frequent sub-graph mining, and recognizing event boundaries.

I implemented an initial workable version of OpenCog Python bindings, with support for MindAgents and more recently CogServer requests both written in Python.

Troy and Zhenhua developed the OpenCog embodiment workbench. The OpenCog workbench is designed to monitor the OpenPsi system, and other aspects of a running instance of OpenCog. Currently, the workbench has three modules: Psi monitor, Dialog system logger, and Psi Emotion Space. The Psi monitor is usable as it stands while the other two are under development. It’s written in Python and uses PyQt for the GUI library.

Jared has been doing a lot of background reading and has been experimenting with frequent sub-graph mining. He has currently decided on using SUBDUE to detect embodiment patterns, as it has a number of benefits over other packages.

Troy integrated minepackage, to provide a dynamic environment (with block destruction and creation) for OpenCog embodiment. Minepackage is an open source package for creating cubic type games, such as Minecraft and infiniminer. The full package supports (or will support): terrain generation, fluid physics, decoration objects and lighting systems. However, our research project will only need some of these features and will also be extending the game world significantly to provide an effective learning environment.

On this note, Cord Krohn has been working on the design side of things has put together some nice concept sheets for the characters in the demos. He is also planning how the learning we wish to demonstrate will be best represented in the minecraft-like environment.

Zhenhua is currently replacing the OpenPsi equations with more suitable expressions based on AtomSpace contents and feedback from the world. Zhenhua also started work on the Dialogue System, which is related to the Perception/Action system that OpenPsi will inform.

Together with Michel, Shuo Chen and others, Ben Goertzel worked out a detailed design for integrating DeSTIN (a variant of HTM) with OpenCog. In addition, a quite extensive document on Michel’s work to port DeSTIN to CUDA is expected by the end of June as a result of Michel’s thesis.

Ben Goertzel and Nil Geisweiller have designed and implemented a framework for “feature metalearning” (transferring information on feature quality from one categorization problem to another), integrated it with MOSES, and are currently testing it on a large database of text categorization problems.

Matt Ikle’ prototyped a simple, non-scalable version of information geometry based economic attention allocation in OpenCog, and found dramatic intelligence improvements over the standard ECAN version for these simple cases. A draft paper is available here.

So, progress on a number of fronts, hopefully by the next recap we’ll have a video of embodied learning to show

There have been a number of contributors so far, including:

• Ted Sanders took the lead with tidying things up and getting things working.
• The Xiamen BLISS lab provided a collection of gesture videos to train and test gesture recognition with DeSTIN.
• Michel Drenthe, also located at the BLISS lab, is planning to port DeSTIN to take advantage of CUDA on NVIDIA graphics cards.
• Amritpal Singh has been developing a wrapper around a SIFT library to do pre-processing of images to extract keypoints and do rudimentary object detection before feeding the annotated image into DeSTIN.

And because a picture is worth a thousand words (or perhaps a thousand keypoints?), here are a couple of images with Amritpal’s SIFT object recognition successfully detecting an apropos textbook.

We’ve got ourselves established in the M-Lab currently, although Gino may want to relocate us to the School of Design on the PolyU campus. We’ve submitted an order to the University requisition people in the University for 3 reasonably specc’ed workstations, we’ll no doubt order more for the rest of the team when they arrive (see below). In the mean time, there are older computers that are functional enough for now and we have our personal laptops which we’ve been using.

After a review of available game engines, and based on the expertise of the team, we’ve decided that Unity3D will be the best engine to use for the project. While Lucid2 was initially proposed, it only supports an old version of DirectX and the original development team isn’t available for round the clock support. Unity3D has an active community and also supports cross-platform development, including iOS (although no Linux support ;-( ).

Initially we were focused on speech interaction in Unity3D, and we managed to get pocketsphinx, the speech recognition library, working as a DLL plugin for Unity3D and controlling a robot character with very basic commands (“go left”, “go right”, “explode”, “stop”, etc.). I started playing with festival as a speech synthesis plugin but then we realised that these aspects should probably not be a primary focus at the beginning due to the trickiness of getting speech working right. Instead and in the mean time,
we want to just use text input/output. We can build on that once the core dynamics are working correctly (or we have more people involved).

Thus, I am focused on creating a network interface for connecting Unity3D to OpenCog. This has involved understanding the existing network architecture of the embodiment system. At this stage it seems it will first involve making a NetworkElement class in C#, then generating the right XML messages. I think this can be improved with zeromq and
protocol buffers, but we’ll wait until the existing pipeline works and I/we better understand the embodiment system.

Lester has been getting more familiar with Unity3D development. Importing game characters from another in-house game project so that we have models to prototype with (until we get some graphic-artists/modellers). He’ll also help with the Unity side of the OpenCog-Unity connection.

Cord is busy with game design. Loosely we’ve been thinking about a game around the idea of the Incredible Machine meets Creatures (well, very roughly at least). There will be levels to solve, where you have to teach the AI character(s) to do certain behaviors and can interact with them by talking… and to a limited extent by “playing god” (limited, because “playing god” uses “magic”/”psi” which you’ve only got a certain amount of). Each level your AI characters will remember the stuff you taught them earlier, you’ll be able to tell
them to interact with new objects the AI hasn’t seen before by telling it that the object is similar to another one it’s already seen. This will be more fully detailed by the end of the week.

Progress is happening, and on a personal note it feels good to be spending all my time and mental energy on OpenCog rather than on distracting contracts to pay the bills

I’ll endeavour to regular post updates and perhaps a video or two once we start having a functional demo.

The team involved were: Professor Min Jiang and students Huang Deheng, Ye Yang, and Shuo Chen.

Raj actually completed these early 2010, but due to myself and others being very busy at the time, we didn’t post them here. However, we continue to get many requests for an easy introduction and a tutorial to OpenCog. Fortunately one of the videos is an introduction to the software framework:

The OpenCog Software Framework – presented by Joel Pitt (me) and Ben Goertzel.

Of course, this is a the high level overview, and the other videos available on the AGI Summer School site focus on more specific aspects of OpenCog, such as:

• Automated Program Learning: the MOSES Algorithm presented by Nil Geisweiller.
• An introduction to PLN by myself.

…among many others!

There’s a new ‘pln fc’ command, in the CogServer shell, which runs some FC inference.

Here are some pics of the new forward chainer on a demo dataset about toys, object persistence etc. They show the atomspace before and after inference.

Following is a general explanation of how it works. The idea is that each PLN Rule can provide templates for the Atoms it requires as input. In each inference step, the forward chainer picks a Rule, and then looks up a sequence of
Atoms that match the input templates.

Here’s an example with DeductionRule, solving the classic “Mortal Socrates” problem, explained on the OpenCog wiki.

DeductionRule requires two Atoms, in the form:

(Inheritance A B)
(Inheritance B C)

which basically means, A is a B and B is a C. It then produces:

(Inheritance A C)

For the first argument, the forward chainer looks up any Atom that matches (Inheritance A B), that is, any InheritanceLink in the system.
Suppose it finds “Socrates is a man”:

(Inheritance Socrates man)

Now it has A = Socrates and B = man. So to find the second argument, it looks for:

(Inheritance man C)

i.e. “man/men is/are <something>”. Suppose it finds “Men are mortal”:

(Inheritance man mortal)

Then it feeds these two premises into the DeductionRule, which produces:

(Inheritance Socrates mortal)

Remember that since this is forward chaining, it could have found all sorts of other things. If it had found, for the second argument, “Men tend to be bald”, then it would have produced “Socrates is probably bald”

Instructions are on the wiki although I’ll also place these in a README.osx file within the repository too.

There are bugs, mostly around library linking (i.e. I recently found that the Ubigraph module isn’t linked to libxmlrpc_client) – so if you find anything that’s broken, please file a bug at Launchpad.

The ultimate goal is to package OpenCog as an OSX application package (and an Ubuntu deb too) – however, I am but one man! If you are familiar with either packaging process, and willing to lend an indirect hand towards developing human-level(+) intelligence, then your help would be most appreciated.

• make POST requests to create a new atom.
• make POST requests to a specific atom URL to update an atom’s truth value, STI, or LTI.
• make a query via a GET request for a particular type of atom, sorted by a variety of attributes and paginated so that client’s can specify the maximum results returned and get more if necessary.
• initiate shell requests, such as telling PLN to backwards chain to increase the confidence of a target atom.

There is the potential to add many more utility methods, such as a url to get all neighbours within a number of hops from a focus atom. Also getting the configuration status and finding out what modules are currently
loaded.

If someone feels like helping, it’s pretty easy to add a new URL/target to the web module and I can help guide them.

More details on the wiki.

There is also now a basic web interface which tabulates queries and has hyperlinks to allow textual navigation of the AtomSpace. This is a prelude to having HTML5 canvas based graph visualisation via the OpenCog web interface.

(Email samir@vettalabs.com if you have detailed questions)

The main update is that the “virtual pet QA system” now answers questions regarding many more spatial relationships; for the full list see

http://www.opencog.org/wiki/EmbodimentLanguageComprehension_Questions

(The work here was in creating rules to identify when these relationships hold between objects as perceived and recorded in the LocalSpaceMap. Ideally these rules would be learned of course, but for a short-cut we’ve hand-coded them…. Making these rules is easy in itself, but required various improvements to the virtual-pet perception infrastructure, which were useful improvements anyway…)

Along the way to doing this, the SpaceServer and LocalSpaceMap inside OpenCog were extended to use 3D rather than just 2D like before. What Samir and Fabricio are working on now is:

1) Loading FrameNet into the Atomspace to enable better reference resolution (this will also be useful for PLN work). They are using an XML version of FrameNet so they don’t need to spider the website…. Note that our license for FrameNet covers only research uses; you need to pay to use it for commercial apps…

2) Building a simple OpenGL-based visualizer so as to visualize what the virtual pet is seeing at each point in time (this should be useful for debugging issues with spatial reasoning, etc.). Basically this is a visualizer for the 3D LocalSpaceMap structure inside OpenCog…