Publications

# denotes equal contribution, * denotes correspondence. See full publications in Google Scholar.
Jump to Preprint, Conference, Journal, Others.

▶ Preprint

•   VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing
     Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan
     arXiv    2024      [paper] [code] [新智元]

•   Modeling Unified Semantic Discourse Structure for High-quality Headline Generation
     Minghui Xu, Hao Fei, Fei Li, Shengqiong Wu, Rui Sun, Chong Teng, Donghong Ji
     arXiv    2024      [paper]  

•   Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
     Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat-Seng Chua, Yueting Zhuang, Siliang Tang
     arXiv    2024      [paper]  

•   De-fine: Decomposing and Refining Visual Programs with Auto-Feedback
     Minghe Gao, Juncheng Li, Hao Fei, Wei Ji, Guoming Wang, Wenqiao Zhang, Siliang Tang, Yueting Zhuang
     arXiv    2023      [paper]  

•   Towards Complex-query Referring Image Segmentation: A Novel Benchmark
     Wei Ji, Li Li, Hao Fei, Xiangyan Liu, Xun Yang, Juncheng Li, Roger Zimmermann
     arXiv    2023      [paper]  

•   ControlRetriever: Harnessing the Power of Instructions for Controllable Retrieval
     Kaihang Pan, Juncheng Li, Hongye Song, Hao Fei, Wei Ji, Shuo Zhang, Jun Lin, Xiaozhong Liu, Siliang Tang
     arXiv    2023      [paper]  

•   NExT-GPT: Any-to-Any Multimodal LLM
     Shengqiong Wu, Hao Fei^*, Leigang Qu, Wei Ji, Tat-Seng Chua
     arXiv    2023      [paper] [code] [新智元]

•   Revisiting Conversation Discourse for Dialogue Disentanglement
     Bobo Li, Hao Fei, Fei Li, Shengqiong Wu, Lizi Liao, Yinwei Wei, Tat-Seng Chua, Donghong Ji
     arXiv    2023      [paper]  

•   ECQED: Emotion-Cause Quadruple Extraction in Dialogs
     Li Zheng, Donghong Ji, Hao Fei, Shengqiong Wu, Jingye Li, Bobo Li, Chong Teng
     arXiv    2023      [paper]  

•   Subband-based Generative Adversarial Network for Non-parallel Many-to-many Voice Conversion
     Jian Ma, Zhedong Zheng, Hao Fei, Feng Zheng, Tat-seng Chua, Yi Yang
     arXiv    2022      [paper]  

▶ Conference

•   Multi-view Counterfactual Contrastive Learning for Fact-checking Fake News Detection
     Yongcheng Zhang, Lingou Kong, Sheng Tian, Hao Fei, Changpeng Xiang, Huan Wang, Xiaomei Wei
      ICMR    2024      [paper]  

•   Actively Learn from LLMs with Uncertainty Propagation for Generalized Category Discovery
     Jinggui Liang, Lizi Liao, Hao Fei, Bobo Li, Jing Jiang
      NAACL    2024      [paper]  

•   NUS-Emo at SemEval-2024 Task 3: Instruction-Tuning LLM for Multimodal Emotion-Cause Analysis in Conversations
     Meng Luo, Han Zhang, Shengqiong Wu, Bobo Li, Hong Han, Hao Fei^*
      SemEval Workshop    2024      (2nd Place)      [paper] [code] 

•   I3: Intent-Introspective Retrieval Conditioned on Instructions
     Kaihang Pan, Juncheng Li, Wenjie Wang, Hao Fei, Hongye Song, Wei Ji, Jun Lin, Xiaozhong Liu, Tat-Seng Chua, Siliang Tang
      SIGIR    2024      [paper]  

•   Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs
     Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Tat-Seng Chua
      CVPR    2024      [paper] [code] 

•   LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
     Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, Tao Chen
      CVPR    2024      [paper] [code] 

•   What Factors Influence LLMs’ Judgments? A Case Study on Question Answering
     Lei Chen, Bobo Li, Li Zheng, Haining Wang, Zixiang Meng, Runfeng Shi, Hao Fei, Jun Zhou, Fei Li, Chong Teng and Donghong Ji
      COLING    2024      [paper]  

•   MMLSCU: A Dataset for Multi-modal Multi-domain Live Streaming Comment Understanding
     Zixiang Meng, Qiang Gao, Di Guo, Yunlong Li, Bobo Li, Hao Fei, Shengqiong Wu, Fei Li, Chong Teng, Donghong Ji
      TheWebConf (WWW)    2024      [paper]  

•   In-Context Learning for Few-Shot Nested Named Entity Recognition
     Meishan Zhang, Bin Wang, Hao Fei^*, Min Zhang
      ICASSP    2024      [paper]  

•   Reverse Multi-Choice Dialogue Commonsense Inference with Graph-of-Thought
     Li Zheng, Hao Fei, Fei Li1, Bobo Li, Lizi Liao, Donghong Ji, Chong Teng
      AAAI    2024      [paper] [code] 

•   Harnessing Holistic Discourse Features and Triadic Interaction for Sentiment Quadruple Extraction in Dialogues
     Bobo Li, Hao Fei, Lizi Liao, Yu Zhao, Fangfang Su, Fei Li, Donghong Ji
      AAAI    2024      [paper]  

•   Improving Expressive Power of Spectral Graph Neural Networks with Eigenvalue Correction
     Kangkang Lu, Yanhua Yu, Hao Fei, Xuan Li, Zixuan Yang, Zirui Guo, Meiyu Liang, Mengran Yin, Tat-Seng Chua
      AAAI    2024      [paper]  

•   Molca: Molecular graph-language modeling with cross-modal projector and uni-modal adapter
     Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, Tat-Seng Chua
      EMNLP    2023      [paper] [code] 

•   VPGTrans: Transfer Visual Prompt Generator across LLMs
     Ao Zhang, Hao Fei^*, Yuan Yao^*, Wei Ji, Li Li, Zhiyuan Liu, Tat-Seng Chua
      NeurIPS    2023      [paper] [code] [新智元]

•   Imagine That! Abstract-to-Intricate Text-to-Image Synthesis with Scene Graph Hallucination Diffusion
     Shengqiong Wu, Hao Fei^*, Hanwang Zhang, Tat-Seng Chua
      NeurIPS    2023      [paper] [code] 

•   LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation
     Leigang Qu, Shengqiong Wu, Hao Fei^*, Liqiang Nie, Tat-Seng Chua
      ACM MM    2023      [paper] [code] 

•   Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition
     Bobo Li, Hao Fei, Lizi Liao, Yu Zhao, Chong Teng, Tat-Seng Chua, Donghong Ji, Fei Li
      ACM MM    2023      [paper]  

•   Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling
     Yu Zhao, Hao Fei^*, Yixin Cao, Bobo Li, Meishan Zhang, Jianguo Wei, Min Zhang, Tat-Seng Chua
      ACM MM    2023      [paper]  

•   Partial Annotation-based Video Moment Retrieval via Iterative Learning
     Wei Ji, Renjie Liang, Lizi Liao, Hao Fei^*, Fuli Feng
      ACM MM    2023      [paper]  

•   DialogRE^C+: An Extension of DialogRE to Investigate How Much Coreference Helps Relation Extraction in Dialogs
     Yiyun Xiong, Mengwei Dai, Fei Li, Hao Fei, Bobo Li, Shengqiong Wu, Chong Teng and Donghong Ji
      NLPCC    2023      [paper] [code] 

•   Reasoning Implicit Sentiment with Chain-of-Thought Prompting
     Hao Fei, Bobo Li, Qian Liu, Lidong Bing, Fei Li, Tat-Seng Chua
      ACL (Short)    2023      [paper] [code] [Paperweakly]

•   Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucination
     Hao Fei, Qian Liu, Meishan Zhang, Min Zhang, Tat-Seng Chua
      ACL    2023      [paper] [code] 

•   Constructing Code-mixed Universal Dependency Forest for Unbiased Cross-lingual Relation Extraction
     Hao Fei, Meishan Zhang, Min Zhang, Tat-Seng Chua
      ACL (Findings)    2023      [paper] [code] 

•   Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling
     Shengqiong Wu, Hao Fei^*, Yixin Cao, Lidong Bing, Tat-Seng Chua
      ACL    2023      (Paper Award Nomination, 1.6%)      [paper] [code] 

•   Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment
     Shengqiong Wu, Hao Fei^*, Wei Ji, Tat-Seng Chua
      ACL    2023      [paper]  

•   Generating Visual Spatial Description via Holistic 3D Scene Understanding
     Yu Zhao, Hao Fei^*, Wei Ji, Jianguo Wei, Meishan Zhang, Min Zhang, Tat-Seng Chua
      ACL    2023      [paper] [code] [Paperweakly]

•   DiaASQ: A Benchmark of Conversational Aspect-based Sentiment Quadruple Analysis
     Bobo Li, Hao Fei, Yuhan Wu, Jinsong Zhang, Shengqiong Wu, Jingye Li, Yijiang Liu, Lizi Liao, Tat-Seng Chua, Fei Li, Donghong Ji
      ACL (Findings)    2023      [paper] [code] [Paperweakly]

•   LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model
     Hao Fei, Shengqiong Wu, Jingye Li, Bobo Li, Fei Li, Libo Qin, Meishan Zhang, Min Zhang, Tat-Seng Chua
      NeurIPS    2022      (Spotlight)      [paper] [code] 

•   Matching Structure for Dual Learning
     Hao Fei, Shengqiong Wu, Yafeng Ren, Meishan Zhang
      ICML    2022      [paper] [code] 

•   Effective Token Graph Modeling using a Novel Labeling Strategy for Structured Sentiment Analysis
     Wenxuan Shi, Fei Li, Jingye Li, Hao Fei, Donghong Ji
      ACL    2022      [paper] [code] 

•   Unified Named Entity Recognition as Word-Word Relation Classification
     Jingye Li^#, Hao Fei^#, Jiang Liu, Shengqiong Wu, Meishan Zhang, Donghong Ji, Fei Li
      AAAI    2022      [paper] [code] [Paperweekly]

•   Mastering the Explicit Opinion-role Interaction: Syntax-aided Neural Transition System for Unified Opinion Role Labeling
     Shengqiong Wu, Hao Fei^*, Fei Li, Donghong Ji, Meishan Zhang, Yijiang Liu, Chong Teng
      AAAI    2022      [paper] [code] 

•   Mutual Disentanglement Learning for Joint Fine-Grained Sentiment Classification and Controllable Text Generation
     Hao Fei, Chenliang Li, Donghong Ji, Fei Li
      SIGIR    2022      [paper] [code] [情感计算]

•   Making Decision like Human: Joint Aspect Category Sentiment Analysis and Rating Prediction with Fine-to-Coarse Reasoning
     Hao Fei, Jingye Li, Yafeng Ren, Meishan Zhang, Donghong Ji
      TheWebConf (WWW)    2022      [paper]  

•   Global Inference with Explicit Syntactic and Discourse Structures for Dialogue-Level Relation Extraction
     Hao Fei, Jingye Li, Shengqiong Wu, Chenliang Li, Donghong Ji and Fei Li
      IJCAI    2022      [paper] [code] 

•   Inheriting the Wisdom of Predecessors: A Multiplex Cascade Framework for Unified Aspect-based Sentiment Analysis
     Hao Fei, Fei Li, Chenliang Li, Shengqiong Wu, Jingye Li and Donghong Ji
      IJCAI    2022      [paper] [code] 

•   Conversational Semantic Role Labeling with Predicate-Oriented Latent Graph
     Hao Fei, Shengqiong Wu, Meishan Zhang, Yafeng Ren and Donghong Ji
      IJCAI    2022      [paper]  

•   Entity-centered Cross-document Relation Extraction
     Fengqi Wang, Fei Li, Hao Fei, Jingye Li, Shengqiong Wu, Fangfang Su, Wenxuan Shi, Donghong Ji, Bo Cai
      EMNLP    2022      [paper] [code] 

•   Conversation Disentanglement with Bi-Level Contrastive Learning
     Chengyu Huang, Zheng Zhang, Hao Fei, Lizi Liao
      EMNLP (Findings)    2022      [paper]  

•   Joint Alignment of Multi-Task Feature and Label Spaces for Emotion Cause Pair Extraction
     Shunjie Chen, Xiaochuan Shi, Jingye Li, Shengqiong Wu, Hao Fei, Fei Li, Donghong Ji
      COLING    2022      [paper] [code] 

•   OneEE: A One-Stage Framework for Fast Overlapping and Nested Event Extraction
     Hu Cao, Jingye Li, Fangfang Su, Fei Li, Hao Fei, Shengqiong Wu, Bobo Li, Liang Zhao, Donghong Ji
      COLING    2022      [paper] [code] 

•   Prompt-Based Generative Multi-label Emotion Prediction with Label Contrastive Learning
     Yuyang Chai, Chong Teng, Hao Fei, Shengqiong Wu, Jingye Li, Ming Cheng, Donghong Ji, Fei Li
      NLPCC    2022      [paper] [code] 

•   Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extraction with Rich Syntactic Knowledge
     Shengqiong Wu^#, Hao Fei^#, Yafeng Ren, Donghong Ji, Jingye Li
      IJCAI    2021      [paper] [code] 

•   Rethinking Boundaries: End-To-End Recognition of Discontinuous Mentions with Pointer Networks
     Hao Fei, Donghong Ji, Bobo Li, Yijiang Liu, Yafeng Ren, Fei Li
      AAAI    2021      [paper] [code] 

•   Encoder-Decoder Based Unified Semantic Role Labeling with Label-Aware Syntax
     Hao Fei, Fei Li, Bobo Li, Donghong Ji
      AAAI    2021      [paper] [code] 

•   End-to-end Semantic Role Labeling with Neural Transition-based Model
     Hao Fei, Meishan Zhang, Bobo Li, Donghong Ji
      AAAI    2021      [paper] [code] 

•   Better Combine Them Together! Integrating Syntactic Constituency and Dependency Representations for Semantic Role Labeling
     Hao Fei, Shengqiong Wu, Yafeng Ren, Fei Li and Donghong Ji
      ACL (Findings)    2021      [paper] [code] 

•   MRN: A Locally and Globally Mention-Based Reasoning Network for Document-Level Relation Extraction
     Jingye Li, Kang Xu, Fei Li, Hao Fei, Yafeng Ren, Donghong Ji
      ACL (Findings)    2021      [paper] [code] 

•   Latent Target-Opinion as Prior for Document-Level Sentiment Classification: A Variational Approach from Fine-Grained Perspective
     Hao Fei, Yafeng Ren, Bobo Li, Shengqiong Wu, Donghong Ji
      TheWebConf (WWW)    2021      [paper]  

•   Latent Emotion Memory for Multi-Label Emotion Classification
     Hao Fei, Yue Zhang, Yafeng Ren, Donghong Ji
      AAAI    2020      [paper] [code] 

•   Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus
     Hao Fei, Meishan Zhang, Donghong Ji
      ACL    2020      [paper] [code] [社媒派SMP]

•   Modeling Local Contexts for Joint Dialogue Act Recognition and Sentiment Classification with Bi-channel Dynamic Convolutions
     Jingye Li, Hao Fei, Donghong Ji
      COLING    2020      [paper]  

•   Retrofitting Structure-aware Transformer Language Model for End Tasks
     Hao Fei, Yafeng Ren, Donghong Ji
      EMNLP    2020      [paper]  

•   Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP
     Hao Fei, Yafeng Ren, Donghong Ji
      EMNLP (Findings)    2020      [paper]  

•   Improving Text Understanding via Deep Syntax-Semantics Communication
     Hao Fei, Yafeng Ren, Donghong Ji
      EMNLP (Findings)    2020      [paper]  

•   Nominal Compound Chain Extraction: A New Task for Semantic-Enriched Lexical Chain
     Bobo Li, Hao Fei, Yafeng Ren, Donghong Ji
      NLPCC    2020      [paper] [code] 

•   Aggressive Language Detection with Joint Text Normalization via Adversarial Multi-task Learning
     Shengqiong Wu, Hao Fei, Donghong Ji
      NLPCC    2020      [paper] [code] 

•   Implicit Objective Network for Emotion Detection
     Hao Fei, Yafeng Ren, Donghong Ji
      NLPCC    2019      [paper]  

▶ Journal

•   Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment
     Hao Fei, Shengqiong Wu, Meishan Zhang, Min Zhang, Tat-Seng Chua, Shuicheng Yan
     IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)    2024     [paper] 

•   TKDP: Threefold Knowledge-enriched Deep Prompt Tuning for Few-shot Named Entity Recognition
     Jiang Liu, Hao Fei, Fei Li, Jingye Li, Bobo Li, Liang Zhao, Chong Teng, Donghong Ji
     Transactions on Knowledge and Data Engineering (TKDE)    2024     [paper] 

•   Knowledge-enhanced event relation extraction via event ontology prompt
     Ling Zhuang^#, Hao Fei^#, Po Hu
     Information Fusion    2023     [paper] 

•   Syntax-based Dynamic Latent Graph for Event Relation Extraction
     Ling Zhuang, Hao Fei, Po Hu
     Information Processing & Management (IPM)    2023     [paper] 

•   On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model, Data, and Training
     Hao Fei, Tat-Seng Chua, Chenliang Li, Donghong Ji, Meishan Zhang, Yafeng Ren
     ACM Transactions on Information Systems (TOIS)    2023     [paper] 

•   Nonautoregressive encoder-decoder neural framework for end-to-end aspect-based sentiment triplet extraction
     Hao Fei, Yafeng Ren, Yue Zhang, Donghong Ji
     IEEE Transactions on Neural Networks and Learning Systems (TNNLS)    2023     [paper] 

•   Optimizing Attention for Sequence Modeling via Reinforcement Learning
     Hao Fei, Yue Zhang, Yafeng Ren, Donghong Ji
     IEEE Transactions on Neural Networks and Learning Systems (TNNLS)    2022     [paper] [code]

•   A semantic and syntactic enhanced neural model for financial sentiment analysis
     Chunli Xiang, Junchi Zhang, Fei Li, Hao Fei, Donghong Ji
     Information Processing & Management (IPM)    2022     [paper] 

•   Enriching contextualized language model from knowledge graph for biomedical information extraction
     Hao Fei, Yafeng Ren, Yue Zhang, Donghong Ji, Xiaohui Liang
     Briefings in Bioinformatics (BiB)    2021     [paper] [code]

•   A span-graph neural model for overlapping entity relation extraction in biomedical texts
     Hao Fei, Yafeng Ren, Yue Zhang, Donghong Ji, Xiaohui Liang
     Bioinformatics    2021     [paper] 

•   Second-Order Semantic Role Labeling With Global Structural Refinement
     Hao Fei, Shengqiong Wu, Yafeng Ren, Donghong Ji
     IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)    2021     [paper] 

•   High-Order Pair-Wise Aspect and Opinion Terms Extraction With Edge-Enhanced Syntactic Graph Convolution
     Shengqiong Wu, Hao Fei, Yafeng Ren, Bobo Li, Fei Li, Donghong Ji
     IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)    2021     [paper] [code]

•   Topic-Enhanced Capsule Network for Multi-Label Emotion Classification
     Hao Fei, Donghong Ji, Yue Zhang, Yafeng Ren
     IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)    2020     [paper] 

•   Cross-Lingual Semantic Role Labeling With Model Transfer
     Hao Fei, Meishan Zhang, Fei Li, Donghong Ji
     IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP)    2020     [paper] 

•   Boundaries and edges rethinking: An end-to-end neural model for overlapping entity relation extraction
     Hao Fei, Yafeng Ren, Donghong Ji
     Information Processing & Management (IPM)    2020     [paper] 

•   A tree-based neural network model for biomedical event trigger detection
     Hao Fei, Yafeng Ren, Donghong Ji
     Information Sciences    2020     [paper] 

•   Dispatched attention with multi-task learning for nested mention recognition
     Hao Fei, Yafeng Ren, Donghong Ji
     Information Sciences    2020     [paper] 

•   A deep neural network model for speakers coreference resolution in legal texts
     Donghong Ji, Jun Gao, Hao Fei, Chong Teng, Yafeng Ren
     Information Processing & Management (IPM)    2020     [paper] 

•   An end-to-end joint model for evidence information extraction from court record document
     Donghong Ji, Peng Tao, Hao Fei, Yafeng Ren
     Information Processing & Management (IPM)    2020     [paper] 

•   A hybrid neural network model for predicting kidney disease in hypertension patients based on electronic health records
     Yafeng Ren^#, Hao Fei^#, Xiaohui Liang, Donghong Ji, Ming Cheng
     BMC Medical Informatics & Decision Making (MIDM)    2019     [paper] 

▶ Others

•   Deep Multimodal Learning for Information Retrieval
     Wei Ji, Yinwei Wei, Zhedong Zheng, Hao Fei, Tat-seng Chua
     ACM MM (Workshop Proposal)    2023     [paper] 

•   On the Syntax-oriented Modeling and Optimization of Natural Language
     Hao Fei
     Ph.D Thesis    2021     [paper]