Source code for Transformer-based Multi-task Learning for Disaster Tweet Categorisation (UCD's participation in TREC-IS 2020A, 2020B and 2021A).

Overview

Source code for "UCD participation in TREC-IS 2020A, 2020B and 2021A".

*** update at: 2021/05/25

This repo so far relates to the following work:

  • Transformer-based Multi-task Learning for Disaster Tweet Categorisation, (WiP paper, ISCRAM 2021)
  • Multi-task transfer learning for finding actionable information from crisis-related messages on social media, (paper, TREC 2020)

Setup

git clone https://github.com/wangcongcong123/crisis-mtl.git
pip install -r requirements.txt

Dataset preparation

  • Download the splits prepared for the system from here that contains three subdirectories for 2020a, 2020b and 2021a respectively.
  • Unzip the file to data/.

Training and submitting

# for 2020a
python run.py --dataset_name 2020a --model_name bert-base-uncased

# or for 2020b
python run.py --edition 2020b --model_name bert-base-uncased
python run.py --edition 2020b --model_name google/electra-base-discriminator
python run.py --edition 2020b --model_name microsoft/deberta-base
python run.py --edition 2020b --model_name distilbert-base-uncased
python submit_ensemble.py --edition 2020b


# or for 2021a
python run.py --edition 2021a --model_name bert-base-uncased
python run.py --edition 2021a --model_name google/electra-base-discriminator
python run.py --edition 2021a --model_name microsoft/deberta-base
python run.py --edition 2021a --model_name distilbert-base-uncased
python submit_ensemble.py --edition 2021a

To see our results compared to other participating runs in 2020a and 2020b, check the appendix of this overview paper. To know the details of our approach, check this ISCRAM-2021 paper on 2020a and this TREC-2020 paper on 2020b. The evaluation for 2021a is still in process so the results will be added as soon as they come out.

Citation

If you use the code in your research, please consider citing the following papers:

@article{wang2021,
author = {Wang, Congcong and Nulty, Paul and Lillis, David},
journal = {Proceedings of the International ISCRAM Conference},
keywords = {18th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2021)},
number = {May},
title = {{Transformer-based Multi-task Learning for Disaster Tweet Categorisation}},
volume = {2021-May},
year = {2021}
}

@inproceedings{congcong2020multi,
 address = {Gaithersburg, MD},
 title = {Multi-task transfer learning for finding actionable information from crisis-related messages on social media},
 booktitle = {Proceedings of the Twenty-Nineth {{Text REtrieval Conference}} ({{TREC}} 2020)},
 author = {Wang, Congcong and Lillis, David},
 year = {2020},
}

Queries

Let me know if any questions via [email protected] or through creating an issue.

Owner
Congcong Wang
Ph.D [email protected], Crisis on Social Media, NLP, Machine Learning, IR
Congcong Wang
From Canonical Correlation Analysis to Self-supervised Graph Neural Networks

Code for CCA-SSG model proposed in the NeurIPS 2021 paper From Canonical Correlation Analysis to Self-supervised Graph Neural Networks.

Hengrui Zhang 44 Nov 27, 2022
Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

Rafael Berral Soler 71 Jan 05, 2023
MWPToolkit is a PyTorch-based toolkit for Math Word Problem (MWP) solving.

MWPToolkit is a PyTorch-based toolkit for Math Word Problem (MWP) solving. It is a comprehensive framework for research purpose that integrates popular MWP benchmark datasets and typical deep learnin

119 Jan 04, 2023
Code and Resources for the Transformer Encoder Reasoning Network (TERN)

Transformer Encoder Reasoning Network Code for the cross-modal visual-linguistic retrieval method from "Transformer Reasoning Network for Image-Text M

Nicola Messina 53 Dec 30, 2022
Semi-Supervised Learning with Ladder Networks in Keras. Get 98% test accuracy on MNIST with just 100 labeled examples !

Semi-Supervised Learning with Ladder Networks in Keras This is an implementation of Ladder Network in Keras. Ladder network is a model for semi-superv

Divam Gupta 101 Sep 07, 2022
An easy way to build PyTorch datasets. Modularly build datasets and automatically cache processed results

EasyDatas An easy way to build PyTorch datasets. Modularly build datasets and automatically cache processed results Installation pip install git+https

Ximing Yang 4 Dec 14, 2021
TensorFlow Metal Backend on Apple Silicon Experiments (just for fun)

tf-metal-experiments TensorFlow Metal Backend on Apple Silicon Experiments (just for fun) Setup This is tested on M1 series Apple Silicon SOC only. Te

Timothy Liu 161 Jan 03, 2023
This is the official PyTorch implementation for "Mesa: A Memory-saving Training Framework for Transformers".

A Memory-saving Training Framework for Transformers This is the official PyTorch implementation for Mesa: A Memory-saving Training Framework for Trans

Zhuang AI Group 105 Dec 06, 2022
The source code for CATSETMAT: Cross Attention for Set Matching in Bipartite Hypergraphs

catsetmat The source code for CATSETMAT: Cross Attention for Set Matching in Bipartite Hypergraphs To be able to run it, add catsetmat to PYTHONPATH H

2 Dec 19, 2022
A toolset of Python programs for signal modeling and indentification via sparse semilinear autoregressors.

SPAAR Description A toolset of Python programs for signal modeling via sparse semilinear autoregressors. References Vides, F. (2021). Computing Semili

Fredy Vides 0 Oct 30, 2021
(to be released) [NeurIPS'21] Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs

Higher-Order Transformers Kim J, Oh S, Hong S, Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs, NeurIPS 2021. [arxiv] W

Jinwoo Kim 44 Dec 28, 2022
The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color

The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color Overview Code and dataset for The World of an Octopus: H

1 Nov 13, 2021
Exploring the link between uncertainty estimates obtained via "exact" Bayesian inference and out-of-distribution (OOD) detection.

Uncertainty-based OOD detection Exploring the link between uncertainty estimates obtained by "exact" Bayesian inference and out-of-distribution (OOD)

Christian Henning 1 Nov 05, 2022
Robust fine-tuning of zero-shot models

Robust fine-tuning of zero-shot models This repository contains code for the paper Robust fine-tuning of zero-shot models by Mitchell Wortsman*, Gabri

224 Dec 29, 2022
Official implementation of "Learning Not to Reconstruct" (BMVC 2021)

Official PyTorch implementation of "Learning Not to Reconstruct Anomalies" This is the implementation of the paper "Learning Not to Reconstruct Anomal

Marcella Astrid 13 Dec 04, 2022
CCNet: Criss-Cross Attention for Semantic Segmentation (TPAMI 2020 & ICCV 2019).

CCNet: Criss-Cross Attention for Semantic Segmentation Paper Links: Our most recent TPAMI version with improvements and extensions (Earlier ICCV versi

Zilong Huang 1.3k Dec 27, 2022
Oriented Response Networks, in CVPR 2017

Oriented Response Networks [Home] [Project] [Paper] [Supp] [Poster] Torch Implementation The torch branch contains: the official torch implementation

ZhouYanzhao 217 Dec 12, 2022
Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces Installation After cloning the repo open

37 Dec 03, 2022
SmallInitEmb - LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence

SmallInitEmb LayerNorm(SmallInit(Embedding)) in a Transformer I find that when t

PENG Bo 11 Dec 25, 2022
This is the repository for Learning to Generate Piano Music With Sustain Pedals

SusPedal-Gen This is the official repository of Learning to Generate Piano Music With Sustain Pedals Demo Page Dataset The dataset used in this projec

Joann Ching 12 Sep 02, 2022