Introduction:

Foundation models (e.g., GPT, CLIP, and Stable Diffusion) have demonstrated remarkable zero-shot performance over previous learning paradigms, paving new avenues towards artificial general intelligence (AGI). Despite their impressive performance, current foundation models are still far from perfect in the open world, easily hampered by imperfect data during both training and deployment. These imperfect data may pose real-world threats, motivating this workshop toward robust and reliable foundation models. To be specific, we call for papers and speakers, discussing existing failure cases for current foundation models and suggesting new training, fine-tuning, and post-hoc strategies that alleviate these failures. Here are some examples of imperfect data.

Here are some examples.
1.Noisy Data: Noise exists in either data features (e.g., corrupted images/texts) or data associations (e.g., incorrect image-caption pairs), further attributed to natural noise and crafted toxics (e.g., backdoor and adversarial attack). Preliminary works have revealed the detriments of noisy data for foundation model training, while how to systematically design robust learning paradigms remains an open question.
2.Biased Data: Biasness in foundation models comes from long-tailed distributions and those data violated human ethics/preferences. Addressing the related issues may require complex data collection procedures and tedious human interference, but effective and efficient paradigms for aligning with ethic and real-world cases are yet to be suggested.
3.Out-of-distribution Data: Out-of-distribution (OOD) data refer to those cases where models lack extrapolated knowledge. Foundation models will fail to make proper predictions, misleading downstream applications in making wrong decisions. Therefore, detecting OOD data and/or improve model OOD performance can be crucial for foundation models.

Workshop Chair:

Dr. Bo Han is an Assistant Professor of Computer Science and a Director of Trustworthy Machine Learning and Reasoning Group at Hong Kong Baptist University, and a BAIHO Visiting Scientist at RIKEN Center for Advanced Intelligence Project. He was a Visiting Faculty Researcher at Microsoft Research (2022) and a Postdoc Fellow at RIKEN AIP (2019-2020). He received his Ph.D. degree in Computer Science from University of Technology Sydney (2015-2019). During 2018-2019, he was a Research Intern with the AI Residency Program at RIKEN AIP, working on trustworthy representation learning. He also works on causal representation learning. He has co-authored a machine learning monograph, including Machine Learning with Noisy Labels (MIT Press). He has served as area chairs of NeurIPS, ICML, ICLR, and UAI. He has also served as action editors and editorial board members of JMLR, MLJ, TMLR, and IEEE TNNLS. He received the Outstanding Paper Award at NeurIPS and Outstanding Area Chair at ICLR. His homepage can be found here: https://bhanml.github.io/.

Submission method: Please submit the paper via Microsoft CMT system
Please note: When you submit your papers in CMT system, please choose "Workshop 2: Trustworthy Foundation Models under Imperfect Data" in the Subject Areas.

 

Invited Speakers

Asst. Prof. Kaiyang Zhou
Hong Kong Baptist University, Hong Kong S.A.R., China

Biography: Dr. Kaiyang Zhou is an Assistant Professor at the Department of Computer Science, Hong Kong Baptist University. He was previously a postdoc at Nanyang Technological University, Singapore. He received his PhD in computer science from the University of Surrey, UK. His research lies at the intersection of computer vision and machine learning and has been published at top-tier journals and conferences in relevant fields such as TPAMI, IJCV, and CVPR. He is an associate editor of the International Journal of Computer Vision (IJCV), and the creator of the popular open-source software Torchreid for person re-identification.
Speech Title: Prompt learning for generalizable visual perception
Abstract: Originating from natural language processing, the new paradigm of prompting has recently swept through the computer vision community, bringing disruptive changes to various computer vision applications, such as image recognition and image generation. In comparison to the traditional fixed-once-learned architecture, like a linear classifier trained to recognize a specific set of categories, prompting offers greater flexibility and more opportunities for novel applications. This is because prompting allows the model to perform new tasks, such as recognizing new categories, by tuning textual instructions or modifying a small number of parameters in the model's input space while keeping most of the pre-trained parameters untouched. This paradigm significantly pushes human-AI interaction to unprecedented levels. In this talk, I will discuss how to use prompt learning to adapt large vision-language models for generic visual perception tasks including image classification and object detection.

Asst. Prof. Ying Wei
City University of Hong Kong, Hong Kong S.A.R., China

Biography: Ying Wei is an Assistant Professor in the Department of Computer Science, City University of Hong Kong since Feb 2021. Prior to that, she was a Senior Researcher at the Machine Learning Center of Tencent AI Lab from November 2017 to January 2021. She works on machine learning, and is especially interested in solving challenges in transfer and meta learning by pushing the boundaries of both theories and applications. She received her Ph.D. degree from Hong Kong University of Science and Technology in 2017 with the support of Hong Kong PhD Fellowship. Her work was nominated for the best paper candidates at KDD’14. She has been invited as a senior PC member for AAAI, to give invited talks at NeurIPS MetaLearn 2021 and FTL-IJCAI 2021 workshops, and as a PC member for many other top-tier conferences, such as ICML, NeurIPS, ICLR, and etc.
Speech Title: Promoting the Generalization of a Pre-trained Model
Abstract： Much of the research hitherto has made strides under pre-trained models and fine-tuning of them in downstream tasks, though the generalization of pre-trained models in real-word applications is less guaranteed. In this talk, I explore three perspectives that improve the generalization of pre-trained models. In particular, I will first explore a transferability estimation measure, so as to give a hint on which pre-trained model out of a huge pool to use for a specific downstream task. Then I will discuss how to promote the diversity of tasks pre-trained during meta-training via a task augmentation approach, which also greatly improves the generalization of a pre-trained model via foreseeing test/unseen tasks. Finally, given the chosen pre-trained model and the target downstream task, I will explore an ensemble fine-tuning scheme that automatically determines where to transfer from the pre-trained model.

Asst. Prof. Yifan Chen
University of Illinois Urbana-Champaign, USA

Biography: Yifan Chen is an assistant professor in computer science and math at Hong Kong Baptist University. He is broadly interested in developing efficient algorithms for machine learning, encompassing both statistical and deep learning models. Before joining HKBU, Yifan earned his Ph.D. degree in Statistics from University of Illinois Urbana-Champaign in 2023, fortunately under the guidance of Prof. Yun Yang.
Speech Title: NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning
Abstract: Pre-trained language model (PLM) emerges as the core component in many natural language processing applications. However, even fine-tuning the PLMs and doing inference are expensive, especially on edge devices with low computing power. Despite some general approaches (e.g., quantization and distillation) for efficient fine-tuning, very few one-shot compression techniques are explored. In this paper, we investigate the neural tangent kernel (NTK)--which reveals the gradient descent dynamics of neural networks--of the multilayer perceptrons (MLP) modules in a PLM and propose to coin a lightweight PLM through NTK-approximating MLP fusion.

Introduction:

Reinforcement learning (RL) is a machine learning technique where an agent learns a policy that generates decisions as actions in an environment through trial and error. The agent receives feedback in the form of rewards or penalties based on its actions, allowing it to learn optimal strategies to maximize its cumulative reward over time. RL has achieved tremendous successes in simulated environments such as in video games, MuJoCo benchmarks, and the game of Go, where the environments are well-defined and training samples are sufficient. There have been growing interests in applying RL in the real world, with recent successes in various domains such as robotics, healthcare, finance, autonomous driving, and energy management. However, deploying RL to real-world applications still remains an open challenge. This workshop covers (1) fundamental research questions underlying the real-world RL applications, and (2) exemplary studies that demonstrate successes of real-world RL applications, and lessons learned.

Topics of interest include (but are not limited to):
1. studies addressing technical challenges about real-world RL systems, esp. about deployment/product, such as robustness, interpretability, sim2real gap, etc.
2. building high-fidelity simulators and benchmarks/datasets for real-world RL systems
3. human factors in real-world RL systems

Workshop Chair:

Haipeng Chen is an assistant professor of data science at William & Mary. Previously, he was a postdoctoral fellow at Harvard University and Dartmouth College. He obtained his Ph.D. from Nanyang Technological University (NTU). His primary research interest is in AI for social good. He investigates theoretical and practical research questions in deploying AI techniques (e.g., reinforcement learning, combinatorial optimization, and prediction) to social domains (e.g., health, environment, cybersecurity, networks). His research has been recognized with the best paper nomination at AAMAS-2021, Innovation Demonstration Award runner-up at IJCAI-2019, and Champion of the 2017 Microsoft Malmo Collaborative AI Challenge. He publishes in premier AI/data science conferences such as AAAI, IJCAI, NeurIPS, AAMAS, UAI, KDD, ICDM, and journals (e.g., IEEE/ACM Transactions). He regularly serves as reviewer/meta-reviewer at premier AI conferences such as AAAI, IJCAI, NeurIPS, ICLR, and ICML. He co-chaired several workshops at the theme of AI and society, including two at ICLR and IJCAI, and one at Harvard. Aiming at algorithm deployment, he collaborates with various non-profits and practitioners from the field, such as the Virginia Department of Health, Tufts Medical Center, Olde Towne Medical Center, The Family Van, Mobile Health Map, Safe Place for Youth, and Wadhwani AI.

Workshop Co-Chair:

Dr. Yuxi Li obtained his PhD in CS from the University of Alberta. On reinforcement learning for real life, he is the lead co-chair for NeurIPS 2022 and ICML 2021/2019 Workshops (https://sites.google.com/view/rl4reallife/), and the lead guest editor (twice) for the Machine Learning Journal Special Issue. He authored a 150-page deep RL overview, with 1200+ citations, https://bit.ly/2AidXm1, and Reinforcement Learning in Practice: Opportunities and Challenges, https://arxiv.org/abs/2202.11296. He reviews for AAAI, ACM CSUR, CACM, JAIR, JMLR, NeurIPS, TKDD, etc. Founder of RL4RealLife.org.

Submission method: Please submit the paper via Microsoft CMT system
Please note: When you submit your papers in CMT system, please choose "Workshop 3: Reinforcement Learning in the Real World" in the Subject Areas.
Important Date for Workshop 3
Submission Deadline: July 21, 2023
Notification Date: August 4, 2023
Registration Deadline: August 9, 2023

Invited Speakers

Dr. Ruibo Liu
Google DeepMind

Biography: Dr. Ruibo Liu is currently a Research Scientist at Google DeepMind. He is interested broadly in natural language processing, with a particular focus on considering human factors in building NLP systems.

Dr. Hung Le
Salesforce Research

Biography: Hung Le is currently a Research Scientist at Salesforce Research. Throughout his research career, he has received great guidance from Steven Hoi and Nancy Chen. During his PhD, he was fortunate to collaborate as a research intern with leading research labs, including Facebook Research, with Satwik Kottur and Alborz Geramifard, and Salesforce Research, with Steven Hoi and Richard Socher.

Dr. Zheng Yuan
Alibaba DAMO Academy

Speech Title: RRHF: Rank responses to align language models with human feedback without tears

Asst. Prof. Hua Wei
Arizona State University, USA

Biography: Hua Wei has joined Arizona State University's School of Computing and Augmented Intelligence as an assistant professor, while also affiliating with the Lawrence Berkeley National Laboratory. Prior to this, he served as an Assistant Professor at the New Jersey Institute of Technology and a Staff Researcher at Tencent AI Lab. His research spans Reinforcement Learning, Data Mining, Urban Computing, and Human-in-the-loop Computations.
Speech Title: Towards actionable decision making in the real world

Asst. Prof. Ryan (Zheyuan) Shi
University of Pittsburg, USA

Speech Title: Bandit and large language models for public sector applications: deployed studies