no code implementations • Findings (EMNLP) 2021 • Zhijing Jin, Zeyu Peng, Tejas Vaidhya, Bernhard Schoelkopf, Rada Mihalcea
Mining the causes of political decision-making is an active research area in the field of political science.
no code implementations • ACL (WebNLG, INLG) 2020 • Qipeng Guo, Zhijing Jin, Ning Dai, Xipeng Qiu, xiangyang xue, David Wipf, Zheng Zhang
Text verbalization of knowledge graphs is an important problem with wide application to natural language generation (NLG) systems.
no code implementations • 23 May 2024 • Zhijing Jin, Nils Heil, Jiarui Liu, Shehzaad Dhuliawala, Yahang Qi, Bernhard Schölkopf, Rada Mihalcea, Mrinmaya Sachan
This work systematically studies IP through a rigorous mathematical formulation, a multi-perspective moral reasoning framework, and a set of case studies.
no code implementations • 10 May 2024 • Jiarui Liu, Wenkai Li, Zhijing Jin, Mona Diab
In an era of model and data proliferation in machine learning/AI especially marked by the rapid advancement of open-sourced technologies, there arises a critical need for standardized consistent documentation.
1 code implementation • 2 May 2024 • Zhijing Jin, Yuen Chen, Fernando Gonzalez, Jiarui Liu, Jiayi Zhang, Julian Michael, Bernhard Schölkopf, Mona Diab
We find that it is difficult to predict which input examples AMR may help or hurt on, but errors tend to arise with multi-word expressions, named entities, and in the final inference step where the LLM must connect its reasoning over the AMR to its prediction.
no code implementations • 25 Apr 2024 • Giorgio Piatti, Zhijing Jin, Max Kleiman-Weiner, Bernhard Schölkopf, Mrinmaya Sachan, Rada Mihalcea
Through this simulation environment, we explore the dynamics of resource sharing among AI agents, highlighting the importance of ethical considerations, strategic planning, and negotiation skills.
no code implementations • 18 Apr 2024 • Abhinav Lalwani, Lovish Chopra, Christopher Hahn, Caroline Trippel, Zhijing Jin, Mrinmaya Sachan
We evaluate our model on a mixed dataset of fallacies and valid sentences.
1 code implementation • 17 Apr 2024 • Zhiheng Lyu, Zhijing Jin, Fernando Gonzalez, Rada Mihalcea, Bernhard Schoelkopf, Mrinmaya Sachan
Sentiment analysis (SA) aims to identify the sentiment expressed in a text, such as a product review.
1 code implementation • 18 Feb 2024 • Francesco Ortu, Zhijing Jin, Diego Doimo, Mrinmaya Sachan, Alberto Cazzaniga, Bernhard Schölkopf
Interpretability research aims to bridge the gap between the empirical success and our scientific understanding of the inner workings of large language models (LLMs).
1 code implementation • NeurIPS 2023 • Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng Lyu, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, Bernhard Schölkopf
Much of the existing work in natural language processing (NLP) focuses on evaluating commonsense causal reasoning in LLMs, thus failing to assess whether a model can perform causal inference in accordance with a set of well-defined formal rules.
1 code implementation • 15 Nov 2023 • David F. Jenny, Yann Billeter, Mrinmaya Sachan, Bernhard Schölkopf, Zhijing Jin
To operationalize this framework, we propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the LLM decision process.
1 code implementation • 5 Nov 2023 • Ishan Kumar, Zhijing Jin, Ehsan Mokhtarian, Siyuan Guo, Yuen Chen, Mrinmaya Sachan, Bernhard Schölkopf
Thus, we propose CausalCite, a new way to measure the significance of a paper by assessing the causal impact of the paper on its follow-up papers.
1 code implementation • 9 Jun 2023 • Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona Diab, Bernhard Schölkopf
In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs).
1 code implementation • 29 May 2023 • Justus Mattern, FatemehSadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick
To investigate whether this fragility provides a layer of safety, we propose and evaluate neighbourhood attacks, which compare model scores for a given sample to scores of synthetically generated neighbour texts and therefore eliminate the need for access to the training data distribution.
1 code implementation • 24 May 2023 • Yiwen Ding, Jiarui Liu, Zhiheng Lyu, Kun Zhang, Bernhard Schoelkopf, Zhijing Jin, Rada Mihalcea
While several previous studies have analyzed gender bias in research, we are still missing a comprehensive analysis of gender differences in the AI community, covering diverse topics and different development trends.
2 code implementations • 23 May 2023 • Jingwei Ni, Zhijing Jin, Qian Wang, Mrinmaya Sachan, Markus Leippold
Due to the task difficulty and data scarcity in the Financial NLP domain, we explore when aggregating such diverse skills from multiple datasets with MTL can work.
1 code implementation • 23 May 2023 • Yuxin Ren, Qipeng Guo, Zhijing Jin, Shauli Ravfogel, Mrinmaya Sachan, Bernhard Schölkopf, Ryan Cotterell
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.
no code implementations • 21 May 2023 • Oana Ignat, Zhijing Jin, Artem Abzaliev, Laura Biester, Santiago Castro, Naihao Deng, Xinyi Gao, Aylin Gunal, Jacky He, Ashkan Kazemi, Muhammad Khalifa, Namho Koh, Andrew Lee, Siyang Liu, Do June Min, Shinka Mori, Joan Nwatu, Veronica Perez-Rosas, Siqi Shen, Zekun Wang, Winston Wu, Rada Mihalcea
Not surprisingly, this has, in turn, made many NLP researchers -- especially those at the beginning of their careers -- worry about what NLP research area they should focus on.
no code implementations • 19 May 2023 • Badr AlKhamissi, Siddharth Verma, Ping Yu, Zhijing Jin, Asli Celikyilmaz, Mona Diab
Our study entails finetuning three different sizes of OPT on a carefully curated reasoning corpus, resulting in two sets of finetuned models: OPT-R, finetuned without explanations, and OPT-RE, finetuned with explanations.
1 code implementation • 9 May 2023 • Fernando Gonzalez, Zhijing Jin, Bernhard Schölkopf, Tom Hope, Mrinmaya Sachan, Rada Mihalcea
Using state-of-the-art NLP models, we address each of these tasks and use them on the entire ACL Anthology, resulting in a visualization workspace that gives researchers a comprehensive overview of the field of NLP4SG.
1 code implementation • 2 May 2023 • Zhiheng Lyu, Zhijing Jin, Justus Mattern, Rada Mihalcea, Mrinmaya Sachan, Bernhard Schoelkopf
In this work, we take sentiment classification as an example and look into the causal relations between the review (X) and sentiment (Y).
no code implementations • 7 Feb 2023 • Zhijing Jin, Rada Mihalcea
This text is from Chapter 7 (pages 141-162) of the Handbook of Computational Social Science for Policy (2023).
1 code implementation • 27 Jan 2023 • Flavio Schneider, Ojasv Kamal, Zhijing Jin, Bernhard Schölkopf
Recent years have seen the rapid development of large generative models for text; however, much less research has explored the connection between text and another "language" of communication -- music.
no code implementations • 20 Dec 2022 • Justus Mattern, Zhijing Jin, Mrinmaya Sachan, Rada Mihalcea, Bernhard Schölkopf
Generated texts from large pretrained language models have been shown to exhibit a variety of harmful, human-like biases about various demographics.
1 code implementation • 5 Dec 2022 • Anna Costello, Ekaterina Fedorova, Zhijing Jin, Rada Mihalcea
However, when we trace those early drafts to their published versions, a substantial gender gap in linguistic uncertainty arises.
no code implementations • 25 Oct 2022 • Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, Mrinmaya Sachan
To protect the privacy of individuals whose data is being shared, it is of high importance to develop methods allowing researchers and companies to release textual data while providing formal privacy guarantees to its originators.
1 code implementation • 21 Oct 2022 • Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, Bernhard Schölkopf, Mrinmaya Sachan
By grounding the behavioral analysis in a causal graph describing an intuitive reasoning process, we study the behavior of language models in terms of robustness and sensitivity to direct interventions in the input space.
no code implementations • 6 Oct 2022 • Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske, Ryan Cotterell, Zhijing Jin
We present a taxonomy for characterising and understanding generalisation research in NLP.
1 code implementation • 4 Oct 2022 • Zhijing Jin, Sydney Levine, Fernando Gonzalez, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Josh Tenenbaum, Bernhard Schölkopf
Using a state-of-the-art large language model (LLM) as a basis, we propose a novel moral chain of thought (MORALCOT) prompting strategy that combines the strengths of LLMs with theories of moral reasoning developed in cognitive science to predict human moral judgments.
1 code implementation • NAACL 2022 • Jingwei Ni, Zhijing Jin, Markus Freitag, Mrinmaya Sachan, Bernhard Schölkopf
We show that these two factors have a large causal effect on the MT performance, in addition to the test-model direction mismatch highlighted by existing work on the impact of translationese.
1 code implementation • ACL 2022 • Daphna Keidar, Andreas Opedal, Zhijing Jin, Mrinmaya Sachan
We analyze the semantic change and frequency shift of slang words and compare them to those of standard, nonslang words.
2 code implementations • 28 Feb 2022 • Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea, Bernhard Schölkopf
In this paper, we propose the task of logical fallacy detection, and provide a new dataset (Logic) of logical fallacies generally found in text, together with an additional challenge set for detecting logical fallacies in climate change claims (LogicClimate).
no code implementations • 13 Oct 2021 • Hongru Wang, Zhijing Jin, Jiarun Cao, Gabriel Pui Cheong Fung, Kam-Fai Wong
However, previous works rarely investigate the effects of a different number of classes (i. e., $N$-way) and number of labeled data per class (i. e., $K$-shot) during training vs. testing.
1 code implementation • EMNLP 2021 • Zhijing Jin, Julius von Kügelgen, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan, Bernhard Schölkopf
The principle of independent causal mechanisms (ICM) states that generative processes of real world data consist of independent modules which do not influence or inform each other.
2 code implementations • Findings (ACL) 2021 • Zhijing Jin, Geeticka Chauhan, Brian Tse, Mrinmaya Sachan, Rada Mihalcea
We lay the foundations via the moral philosophy definition of social good, propose a framework to evaluate the direct and indirect real-world impact of NLP tasks, and adopt the methodology of global priorities research to identify priority causes for NLP research.
1 code implementation • 14 Dec 2020 • Qipeng Guo, Zhijing Jin, Ziyu Wang, Xipeng Qiu, Weinan Zhang, Jun Zhu, Zheng Zhang, David Wipf
Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.
1 code implementation • COLING 2020 • Zhijing Jin, Qipeng Guo, Xipeng Qiu, Zheng Zhang
With a human-annotated test set, we provide this new benchmark dataset for future research on unsupervised text generation from knowledge graphs.
Ranked #1 on Unsupervised KG-to-Text Generation on GenWiki (Fine)
2 code implementations • CL (ACL) 2022 • Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, Rada Mihalcea
Text style transfer is an important task in natural language generation, which aims to control certain attributes in the generated text, such as politeness, emotion, humor, and many others.
1 code implementation • EMNLP 2020 • Xiaoyu Xing, Zhijing Jin, Di Jin, Bingning Wang, Qi Zhang, Xuanjing Huang
Based on the SemEval 2014 dataset, we construct the Aspect Robustness Test Set (ARTS) as a comprehensive probe of the aspect robustness of ABSA models.
Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA)
2 code implementations • ACL (WebNLG, INLG) 2020 • Qipeng Guo, Zhijing Jin, Xipeng Qiu, Wei-Nan Zhang, David Wipf, Zheng Zhang
Due to the difficulty and high cost of data collection, the supervised data available in the two fields are usually on the magnitude of tens of thousands, for example, 18K in the WebNLG~2017 dataset after preprocessing, which is far fewer than the millions of data for other tasks such as machine translation.
1 code implementation • 5 Jun 2020 • Zhijing Jin, Yongyi Yang, Xipeng Qiu, Zheng Zhang
In natural language, often multiple entities appear in the same text.
1 code implementation • ACL 2020 • Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, Peter Szolovits
Current summarization systems only produce plain, factual headlines, but do not meet the practical needs of creating memorable titles to increase exposure.
1 code implementation • 22 Jan 2020 • Di Jin, Zhijing Jin, Joey Tianyi Zhou, Peter Szolovits
State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data.
6 code implementations • 27 Jul 2019 • Di Jin, Zhijing Jin, Joey Tianyi Zhou, Peter Szolovits
Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models.
3 code implementations • IJCNLP 2019 • Zhijing Jin, Di Jin, Jonas Mueller, Nicholas Matthews, Enrico Santus
Text attribute transfer aims to automatically rewrite sentences such that they possess certain linguistic attributes, while simultaneously preserving their semantic content.
2 code implementations • NAACL 2019 • Yujie Qian, Enrico Santus, Zhijing Jin, Jiang Guo, Regina Barzilay
Most modern Information Extraction (IE) systems are implemented as sequential taggers and only model local dependencies.
1 code implementation • 30 Oct 2018 • Zhijing Jin, Tristan Swedish, Ramesh Raskar
Over the recent years, there has been an explosion of studies on autonomous vehicles.