I am interested in meaning and how individuals can interact meaningfully.
As a psychologist and cognitive scientist, I study meaning specifically in terms of semantic memory - I study how individuals gain and use their general knowledge about the world.
My work is largely driven by the distributional hypothesis, an idea first proposed by linguists that the meaning of a word depends on the context in which the word occurs. From this perspective, the meaning of a word depends on its statistical dependence on other words.
I apply the distributional hypothesis computationally through various Distributional Semantic Models (DSMs), which track the statistical distribution of words in a body of text. Typically, DSMs are dispatched to evaluate the relationships between words, thereby modelling human similarity judgments. I prefer instead to apply DSMs in order to build comprehension systems in order to answer the question of how humans go from words to understanding.
As a psychologist and cognitive scientist, I study meaning specifically in terms of semantic memory - I study how individuals gain and use their general knowledge about the world.
My work is largely driven by the distributional hypothesis, an idea first proposed by linguists that the meaning of a word depends on the context in which the word occurs. From this perspective, the meaning of a word depends on its statistical dependence on other words.
I apply the distributional hypothesis computationally through various Distributional Semantic Models (DSMs), which track the statistical distribution of words in a body of text. Typically, DSMs are dispatched to evaluate the relationships between words, thereby modelling human similarity judgments. I prefer instead to apply DSMs in order to build comprehension systems in order to answer the question of how humans go from words to understanding.
Instance Theory of Semantic MemoryDistributional semantic models (DSMs) specify learning mechanisms with which humans construct a deep representation of word meaning from statistical regularities in language. Despite their remarkable success at fitting human semantic data, virtually all DSMs may be classified as prototype models in that they try to construct a single representation for a word’s meaning aggregated across contexts. This prototype representation conflates multiple meanings and senses of words into a center of tendency, often losing the subordinate senses of a word in favor of more frequent ones. We present an alternative instance-based DSM based on the classic MINERVA 2 multiple-trace model of episodic memory. The model stores a representation of each language instance in a corpus, and a word’s meaning is constructed on-the-fly when presented with a retrieval cue. Across two experiments with homonyms in both an artificial and natural language corpus, we show how the instance-based model can naturally account for the subordinate meanings of words in appropriate context due to nonlinear activation over stored instances, but classic prototype DSMs cannot. The instance-based account suggests that meaning may not be something that is created during learning or stored per se, but may rather be an artifact of retrieval from an episodic memory store.
|
Foraging in Semantic MemoryHills, Jones, and Todd (2012) observed that response patterns during the semantic fluency task (e.g., “name all the animals you can in a minute”) display statistical signatures of memory search that mirror optimal foraging in physical space. They proposed a model of memory search based on exploration-exploitation tradeoffs known to produce optimal foraging patterns when animals search for food resources, applied to a spatial model of semantic memory. However, Abbott, Austerweil, and Griffiths (2015) demonstrated that optimal foraging behavior could also naturally emerge from a random walk applied to a network representation of semantic memory, without reliance on a foraging process. Since then, this has been a very active are of debate in the literature, but core confounds have prevented any clear conclusions between the random walk and cue switching model. We control confounds here by using a fixed training corpus and learning model to create both spatial and network representations, and evaluate the ability of the cue switching model and several variants of the random walk model to produce the behavioral characteristics seen in human data. Further, we use BIC to quantitatively compare the models’ ability to fit the human data, an obvious comparison that has never before been undertaken. The results suggest a clear superiority of the Hills et al. cue switching model. The mechanism used to search memory in the fluency task is likely to have been exapted from mechanisms evolved for foraging in spatial environments.
|
SCiP Poster
Code[link] |
Toolkit for Modeling Semantic Spaces
Numerous toolkits exist for the generation of semantic spaces. Popular models such as GenSim and TensorFlow are highly streamlined software packages that allow users to extract semantic similarities from a given corpus. However, these packages do not employ many of the most successful semantic models, leaving a wide swath of parameter space in semantic research under explored. For instance, GenSim employs continuous-bag-of-words (CBOW) and skip-gram, as well as other statistical techniques, such as LSA, LDA, and td-idf transformations, while TensorFlow is limited to the CBOW and skip-gram models. We present a generalized, modular toolkit that implements often overlooked models of semantic memory. These models include HAL, COALS, GLoVe, and BEAGLE. Additionally, the toolkit fosters an exploration of model combination and construction allowing for an effective framework for future research into the efficacy of techniques in semantic modeling.
|