FIRE 2021
Forum for Information Retrieval Evaluation
Virtual Event
13th-17th December
Effective and Efficient Neural Re-Ranking
Allan Hanbury, TU Wien, Austria
Re-ranking in Information Retrieval involves using a more effective and usually more computationally expensive algorithm in a second stage to re-rank the top results returned by a very efficient retrieval algorithm in the first stage. Complex neural architectures such as BERT have recently shown very high effectiveness on the re-ranking task, but have the following disadvantages: (i) high computational cost requiring powerful hardware and leading to unacceptable waits before results are returned, (ii) lack of interpretability of the ranking produced, and (iii) a limit on the amount of document text that can be processed. These disadvantages limit the implementation of neural approaches in the more constrained environments with specific user requirements and user expectations often found in domain-specific or enterprise search. I present approaches developed by my group to overcome these disadvantages, including the Transformer-Kernel (TK) neural re-ranking model, its adaptation for long text (TKL), and Cross-Architecture Knowledge Distillation.
Topic-Driven Sentiment Analysis
Yulan He, University of Warwick, UK
Understanding opinions or sentiments expressed in text often goes beyond sentiment classification as it is more desirable to map the extracted opinions into various aspects. This is closely related to aspect-based sentiment analysis (ABSA) in which both opinion targets and target-specific opinion expressions are identified. Building an effective ABSA model, however, usually requires fine-grained annotations of both aspects and aspect-associated sentiments at the sentence level, involving heavy manual efforts. We instead explore alternative approaches in which we aim to automatically extract polarity-bearing topics discussed in text without requiring fine-grained aspect-level annotations for training. In this talk, I will present our recent work on topic-driven sentiment analysis. This includes (1) a disentangled adversarial neural topic models for separating opinions from plots in user reviews; (2) a brand-topic model which can infer real-valued brand-associated sentiment scores and generate fine-grained sentiment-topics with their associated sentiments gradually varying from negative to positive; and (3) a topic-driven knowledge-aware transformer for emotion detection in dialogues in which existing neural language models are extended by adding an additional topic layer to capture topic transition patterns in dialogues.
Fair and Desirable Allocation of Exposure in Ranking
Thorsten Joachims, Cornell University, USA
Search engines and recommender systems have become the dominant matchmaker for a wide range of human endeavors -- from online retail to finding romantic partners. Consequently, they carry substantial power in shaping markets and allocating opportunity to the participants. In this talk, I will discuss how the machine learning algorithms underlying these system can produce unfair ranking policies for both exogenous and endogenous reasons. Exogenous reasons often manifest themselves as biases in the training data, which then get reflected in the learned ranking policy and lead to rich-get-richer dynamics. But even when trained with unbiased data, reasons endogenous to the algorithms can lead to unfair or undesirable allocation of opportunity. To overcome these challenges, I will present new machine learning algorithms that directly address both endogenous and exogenous unfairness.
A Conceptual Framework for a Representational Approach to Information Retrieval
Jimmy Lin, University of Waterloo, Canada
In this talk, I will present a conceptual framework for understanding recent developments in information retrieval that attempts to integrate dense and sparse retrieval methods. I propose a representational approach that breaks the core text retrieval problem into a logical scoring model and a physical retrieval model. The scoring model is defined in terms of encoders, which map queries and documents into a representational space, and a comparison function that computes query-document scores. The physical retrieval model defines how a system produces the top-k scoring documents from an arbitrarily large corpus with respect to a query. The scoring model can be further analyzed along two dimensions: dense vs. sparse representations and supervised (learned) vs. unsupervised approaches. I show that many recently proposed retrieval methods, including multi-stage ranking designs, can be seen as different parameterizations in this framework, and that a unified view suggests a number of open research questions, providing a roadmap for future work.
