Igor A. Mel’čuk and Alain Polguère, (1987) “A Formal Lexicon in Meaning-Text Theory”, Computational Linguistics, vol. 13, pp. 261-275.

portal.acm.org/citation.cfm?id=48160.48166
www.aclweb.org/anthology/J/J87/J87-3006.pdf

Within the context of that theory, the output of the Stanford parser is strictly at the SSynR or “surface syntactic representation” level, while, as a general rule Relex attempts to generate the DSynR or “Deep syntactic representation” structure.  Some of what I’ve been trying to do with opencog is towards the “SemR” structure, as described in that paper.

The more I read about MTT, the more it seems to capture some of what we are trying to do (defacto are doing) with NLP within opencog.  In particular, the MTT concept of a “lexical function” (which is not really described in that paper??) could be a particularly strong way of guaranteeing correct syntactic output for segsim, nlgen or NLGen2

– Linas Vepstas

I wrote up a few paragraphs on the most prominent differences; most of my updates were to the RelEx dependency relations page.

Here are the main bullet points:

• RelEx attempts basic entity extraction, and thus avoids generating nn noun modifier relations for named entities.
• RelEx will collapse the object and complement of a preposition into one. Stanford will do this for some, but not all relationships.
• RelEx will convert passive subjects into objects, and instead indicate passiveness by tagging the verb with a passive tense feature.
• RelEx avoids generating copulas, if at all possible, and instead indicates copular relations as predicative adjectives, or in other ways.
• RelEx extracts semantic variables from questions, with the intent of simplifying question answering. For example, “Where is the ball?” generates _pobj(_%atLocation, _$qVar) _psubj(_%atLocation, ball), which can then pattern-match a plausible answer: _pobj(under, couch).
• RelEx attempts to extract comparison variables.

Its also clear to me that I could split up the relex processing into two stages: one which generates stanford-style syntactic relations, and a second stage that generates the more abstract stuff. This might be a wise move … Since RelEx is already more than 3x faster than the Stanford parser, this could attract new users.

– Linas Vepstas

MVp- Js+

is a disjunct that says “there must be an MVp link from this word, going to the left, and an Js link, going to the right”. This disjunct commonly connects prepositions to a verb on their left (the MV- link) and the object of the preposition on the right (the J+ link).

A good way to think about disjuncts is to imagine them as very fine-grained part-of-speech tags. Thus, when one sees “MVp- Js+” associated to a word, one knows not only that the word is a preposition, but even a bit more: its a preposition that took a singular object.  Disjuncts classify words not just into crude part-of-speech categories, but much finer categories:  thus verbs are not just as transtivie or intransitive verbs, but mgiht be transitive verbs that take both direct and indirect objects, or participles, etc.

Siva Reddy, a GSOC 2009 summer student, prepared a table of the frequency of occurrence of different disjuncts in a large collection of text. The top six entries are

Ds+           950275.635843
Xp-           838569.90527
A+          616522.664867
AN+        566658.997313
MVp- Js+       563082.649325
MVp- Jp+      446487.310222

and these are exactly what one might expect:

• Ds+ connects the determiner “the” to nouns: and of course, “the” is the most frequent word in the English language.
• Xp- connects the period at the end of the sentence to the start of the sentence, so of course its frequently observed.
• A+ connects adjectives to nouns, AN+ connects noun modifiers to nouns.
• As noted above, MV connects verbs to modifying phrases, and J connects prepositions to objects, so that MV- J+ is the disjunct that most prepositions will get. Js connects to a singular object, Jp connects to a plural count or mass noun.

A graph of rank vs. frequency is shown below:

Disjunct rank vs. frequency of occurance

As can be seen, the distribution is more or less Zipfian, with a power-law exponent of 1.5.  The fact that the long tail appears to be linear indicates that grammatical construction in the English language appears to be more ore less scale-free: difficult and akward constructions are increasingly rare.  The fact that the graph is not purely Zipfian, but instead has a knee for the most common grammatical connections suggests that the most common grammatical constructions are “less common than they should be”: almost as if English speakers are resisting the use of formulaic sentence constructions. So, for example, since adjectives and noun-modifiers appear near the top of the rank, this suggests that English speakers “could have” used more adjectives and noun-modifiers, but didn’t. Quite why this is so is not clear.  Perhaps the use of anaphora and references in general  helps decrease the need for lots of modifiers.

The open questions are then:

1. Why a power law of 1.5?
2. Why is there a knee?
3. Does this result hold for other languages?

The corpus used here consists of approximately 1 million sentences, obtained by parsing entire Wikipedia articles, Voice of America news stories, and 10 books from Project Gutenberg, including War and Peace, Jane Austen, and some scientific or medical texts.

– Linas Vepstas

Mutual information of nearby words

Above is a graph showing the distribution of the mutual information of word pairs that occur in the same sentence. A number of texts were analyzed, including a portion of Wikipedia, some books from project Gutenberg, etc. A collection of all possible pairs of words was created, where each word in the pair occurs in the same sentence (with the left word of the pair having occurred in the sentence to the left of the right word in the pair). These were counted — about 10 million word pairs were observed — and their mutual information was calculated.

Mutual information is a measure of the likelihood of seeing two words occur together: thus, for example “Northern Ireland” will have a high mutual information, since the words “Northern” and “Ireland” are used together frequently.  By contrast, “Ireland is” will have negative mutual information, mostly because the word “is” is used with many, many other words besides “Ireland”; there is no special relationship between these words. High-mutual-information word pairs are typically noun phrases, often idioms and “collocations”, and almost always embody some concept (so, for example, “Northern Ireland” is the name of a place — the name of the conception of a particular country).

In mathematical terms, the mutual information of a word pair (x,y) is defined as:

M(x,y) = log_2  P(x,y) / P(x,*) P(*,y)

where P(x,y) is the probability of seeing the word pair (x,y), P(x,*) is the probability of seeing a word pair where the left word is x, and P(*,y) is the probability of seing a word pair where the right word is y.

The graph shows M(x,y) on the horizontal axis, and the probability of seeing such a value of M on the vertical axis. This is a bin-count of the distribution of possible values of mutual information, over all word pairs.  This is *NOT* a scatterplot of M(x,y) vs. P(x,y).

Here’s another graph: same as above, except that this time, only pairs of words that occur immediately next to one-another are considered.  The sample size is much smaller: only about 2.4M word-pairs were collected.

Mutual Information of Neighboring Word Pairs

The blue and green exponential lines are located in exactly the same place as in the previous graph. It’s humped in a different way than the previous graph. What is the shape of this hump?  Are the slopes characteristic, or do they vary from one corpus sample to another?  If anyone knows the answers to these questions, please let me know!

Notice the peaks off to the right, at high MI values, in the first graph. I think these are word pairs which are heavily used (topics/terms that are discussed) in one single contributing text, but in none of the others. That’s the hypothesis, I don’t know.

Here is a more detailed discussion, with many other additional figures.

– Linas Vepstas