Contents:
Computers can be trained to recognize typical participants in action events in written language, such as instruments – “knife” is a typical instrument of “cutting”. “Cake” is also something that is typically cut (a “patient” of “cutting”).
Humans are able to identify “cooking” as an action that typically takes place in a “kitchen”, or that a “kitchen” is for cooking. The problem is, other things can happen in a kitchen. Computers have particular difficulty with identifying typical location-action relationships in language – in existing work that mainly relies on machine learning over large volumes of text.
This project is about exploring ways to enhance typical location prediction through machine learning techniques (e.g. deep learning, reinforcement learning, etc) by adding image data to textual data and is part of an international collaboration with Prof. Vera Demberg at Saarland University, Germany.
You can think of meaning in language as a kind of “blob” that is roughly partitioned by words. The partitions vary in clarity – “table” is clearly different from “flower” but less different from “countertop”. Different languages partition the space differently, but people have hypothesized that there are common elements in this partitioning. For example, there are countless ways to divide up the space of possible colours by words, but languages seem to converge on the same partitioning – a similar arrangement of colour terms.
This project is about using machine learning techniques, particularly deep reinforcement learning in an agent-based paradigm, to simulate how language communities might arrive at a common partitioning of the “meaning blob” into words, based on theories of cognition and communication. In cooperation with Prof. Devdatt Dubhashi at Chalmers.
In the sentence, “Every pedestrian held an umbrella”, most English-speakers would assume that there was one umbrella per pedestrian, even though the syntax allows for there to be a single umbrella held by every pedestrian at once. On the other hand, “Every pilgrim visited a shrine” allows for a plausible interpretation that all the pilgrims visited a single shrine. Some combination of experience and common sense allows language users to assign relative probable set cardinalities to different pairs of objects: e.g. “pedestrian ~ umbrella” while “pilgrim > shrine”.
This project will attempt to use deep learning techniques over textual and image data sources as well as data collected from human experiments in order to build a preliminary general model of relative set cardinality assignment, in the context of universal quantifiers like “every”.
A “nominal predicate” is an action represented as a noun rather than a verb – for example, “destruction” is a noun representing the same type of event that the verb “destroy” represents. Something can destroy a city – we can thereafter refer to that as “the destruction of the city.” Unlike verbs, however, nominal predicates can omit all mention of the events participants, leaving them implicit (“There was destruction everywhere.”) Machine understanding of texts still requires the ability to identify the implicit participants of a nominal predicate.
This project will seek to develop, combine, and improve new and existing machine learning models of nominal predicate identification. The training and testing data come from existing large English text corpora. This is part of an international collaboration with Prof. Vera Demberg at Saarland University, Germany.
Words being more and less predictable in context have an effect on speaker behaviour – for example, taking longer to pronounce a word in order to enable the listener to more easily recognize it. The factors of predictability in conversational behaviour are challenging to identify. Very simple language models (machine-learned models of sequences) have been used to demonstrate the effect of predictability, but more sophisticated models that take into account syntax and semantics may provide more explanatory power.
This project involves applying recent machine learning techniques, particularly neural language modeling, to model effects of predictability over transcribed conversational text in existing collections (e.g. AMI Corpus). This is part of an international collaboration with Prof. Vera Demberg at Saarland University, Germany.
The political concept of “dogwhistling” in English is the use of language by politicians to use the same utterances to send different intended messages to different groups, e.g., to mislead the media about what they mean, while signalling a “real” message to a base of core followers “in the know”, so to speak.
A similar strategy is represented by the popular board game “Dixit”, where players are rewarded for deceiving SOME of their competitors about the identity of an abstract image BUT NOT ALL of their competitors – indeed, they are penalized if they deceive none OR ALL of their competitors.
This project is about taking the first steps to learn models of what might be called “adversarial communication” using human-identified image features. We will design an online interface intended to collect such features in the context of a game against a computer using a generative adversarial network (GAN) or reinforcement learning strategy, as appropriate.