Abstract:
Natural language processing (NLP) algorithms have reached unprecedented milestones in the last few years, primarily due to the introduction of sizeable pre-trained language models. While decent results can be achieved even in zero-shot setups (i.e., when the model is not exposed to labeled data from the task of interest), solid results still depend on fine-tuning with labeled task data from the test distribution (a.k.a, the target domain distribution). Yet, since test data may emanate from many linguistics domains (each with unique distributional properties), NLP algorithms are likely to perform under an out-of-distribution (OOD) scenario. As generalization beyond the training distribution is still a fundamental challenge, NLP algorithms suffer a significant degradation when applied to OOD examples.
This seminar addresses these shortcomings, primarily focusing on domain adaptation definitions, which naturally tackle the OOD challenge. We present two complementary efforts: The evolution of classic domain adaptation methods in the large pretrained language models era; and the development of new fundamental approaches to performing domain adaptation.
https://technion.zoom.us/j/94950420992
