Fetching the paper…

Reading the bibliography…

2016

Deep Embedding for Spatial Role Labeling

Ludwig, Oswaldo, Liu, Xiao, Kordjamshidi, Parisa et al.

Original
PDF
Post on X

Understand

This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description.

The model is composed of a deep multilayer perceptron (MLP) stacked on the top of a Long Short Term Memory (LSTM) network, the latter being preceded by an embedding layer.
The VIEW is applied to transferring multimodal background knowledge to Spatial Role Labeling (SpRL) algorithms, which recognize spatial relations between objects mentioned in the text.
This work also contributes with a new method to select complementary features and a fine-tuning method for MLP that improves the $F1$ measure in classifying the words into spatial roles.

Deep Embedding for Spatial Role Labeling · Around

Built on

Long short-term memory
Hochreiter, Sepp and Schmidhuber, Jürgen · 1997
Earlier work this paper cites.
A support vector method for multivariate performance measures
Joachims, Thorsten · 2005
Earlier work this paper cites.
Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy
Peng, H., Long, F., and Ding, C · 2005
Earlier work this paper cites.
Learning structured prediction models: A large margin approach
Original
Taskar, Ben, Chatalbashev, Vassil, Koller, Daphne, and Guestrin, Carlos · 2005
Earlier work this paper cites.

Similar

Novel maximum-margin training algorithms for supervised neural networks
Ludwig, Oswaldo and Nunes, Urbano · 2010
Cited alongside, same era.
Multimodal learning with deep boltzmann machines
Srivastava, Nitish and Salakhutdinov, Ruslan R · 2012
Cited alongside, same era.
Efficient estimation of word representations in vector space
Original
Mikolov, Tomas, Chen, Kai, Corrado, Greg, and Dean, Jeffrey · 2013
Cited alongside, same era.
Deep fragment embeddings for bidirectional image sentence mapping
Karpathy, Andrej, Joulin, Armand, and Li, Fei Fei F · 2014
Cited alongside, same era.

Then

Adam: A method for stochastic optimization
Original
Kingma, Diederik and Ba, Jimmy · 2014
Later among the works it cites.
Unifying visual-semantic embeddings with multimodal neural language models
Original
Kiros, Ryan, Salakhutdinov, Ruslan, and Zemel, Richard S · 2014
Later among the works it cites.
Microsoft coco: Common objects in context
Lin, Tsung-Yi, Maire, Michael, Belongie, Serge, Hays, James, Perona, Pietro, Ramanan, Deva, Dollár, Piotr, and Zitnick, C Lawrence · 2014
Later among the works it cites.
Global machine learning for spatial ontology population
Kordjamshidi, Parisa and Moens, Marie-Francine · 2015
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…