Postdoc Position: Efficiency and Robustness in Language Model Pre-training, IT University of Copenhagen

IT University of Copenhagen
May 31, 2024
Offerd Salary:Negotiation
Working address:N/A
Contract Type:Other
Working Time:Full time
Working type:N/A
Ref info:N/A

Postdoc Position: Efficiency and Robustness in Language Model Pre-training, IT University of Copenhagen

The IT University of Copenhagen invites highly motivated individuals to apply for a postdoc position starting on 1 September 2024 or soon thereafter.

The IT University's vision is to create and share knowledge that is profound and leads to groundbreaking information technology and services for the benefit of humanity. The Computer Science Department seeks to conduct research and education that are both foundational and have impact on industry and society.

Job description

Recent generative systems based on pre-trained language models are remarkably fluent, but this is achieved by extreme volumes of computation and training data. This means not only high energy costs, but also training on data that is problematic in various ways: copyright, harmful social stereotypes, non- representative sampling, misinformation, junk SEO texts, pornography, and contamination with NLP datasets used for evaluation.

This project will create an ambitious resource for research on transfer learning, in which pre-training data is held constant, and evaluation takes into account how much similar data was observed in training, and in what ways it was similar. This resource will encourage the development of more efficient and robust approaches, since it will not be possible to improve benchmark scores by simply training on more data.

The ideal candidate will have a strong background in Computational Linguistics/Natural Language Processing and experience developing NLP resources, as well as core skills in programming in Python and machine learning.

The position is funded for 1 year, and it is our intention to find additional funding to extend this postdoc to a 2- or 3-year position. Besides research, the postdoc will gain experience with organization of an international workshop and shared task and build up their international network. For those interested in pursuing an academic career, it is also possible to:

  • gain experience in applying for external funding with professional support of ITU funding unit (either for the continuation of the postdoc's own position, e.g. Marie Curie postdoctoral fellowship, or by contributing to PI's grant proposals);
  • supervise Master students solo, and/or assist in supervising a PhD student;
  • undertake a formal teacher training program, including teaching guest lectures in the relevant data science courses at the ITU computer science department.
  • The research environment

    The candidate will be a member of the national Pioneer Centre for Artificial Intelligence, a 5-university Danish research endeavor, and of the NLPnorth research group at the IT University's Computer Science Department. Both the centre and research group are highly international and well-funded, working on a broad range of research topics.

    The main supervisor for the project is Associate Professor Anna Rogers. The project will be performed in collaboration with Tal Linzen, Associate Professor at New York University.

    General information

    The IT University of Copenhagen (ITU) is a teaching and research-based tertiary institution concerned with information technology (IT) and the opportunities it offers. The IT University has more than 160 full-time Faculty members. Research and teaching in information technology span all academic activities which involve computers including computer science, information and media sciences, humanities and social sciences, business impact and the commercialization of IT.

    About the Computer Science Department

    The ITU Computer Science Department conducts research in all aspects of computer science and data science, including algorithms, databases, government IT, machine learning, natural language processing, network science, optimization, programming languages, quantum computing, robotics, security, semantics, software engineering, software verification, type theory, and more, as well as a range of applications. The department has 55 permanent faculty, 65 PhDs and postdocs, as well as other staff. It is responsible for the IT University's most technical education programs BSc Software Development, BSc Data Science, MSc Computer Science, MSc Data Science, and MSc Software Design.

    Working at the IT University

    Located in modern architecture, a 6-minute metro ride from the center of Copenhagen, the IT University offers an informal working environment, a varied everyday life, and a highly motivated, ambitious, competent, and diverse international faculty, with PhD degrees from 17 different countries. The IT University is an equal opportunity employer and prides itself in actively and successfully pursuing an improved gender balance among students and among faculty.

    Working in Copenhagen

    Copenhagen has a solid educational system, a rich cultural life, universal healthcare, good childcare, and well-functioning infrastructure. Copenhagen is among the world's most livable cities according to the Economist Global Liveability Index. Living and working in Copenhagen will be a good experience for you and your family. Here you can find some reflections by foreign PhD students in the NLPNorth research group.

    Salary and terms of employment

    The applicant will be employed and enrolled at the ITU University for a period of 12 months. Appointment and salary will be in accordance with the Ministry of Taxation's agreement with the Danish Confederation of Professional Associations (AC) and job structure for academic staff at universities.

    Qualification Requirements

    The applicant must hold or be close to completion of, a PhD-degree with proven research experience in Computational Linguistics or Natural Language Processing with a strong publication record, for example, publications in top conferences such as ACL, LREC, COLING, or similar, or in journals of a comparable quality .

    Application The application and all other documents ought to be in English and must include:

  • A full CV including name, phone number, e-mail, previous and present employment and academic background, a complete list of publications, and the names and addresses of 2-3 professional references.
  • A cover letter discussing the match between this position and your qualifications, as well as your research and career agenda;
  • Copies of up to 3 research publications;
  • A brief statement (up to 1 page) of your contributions to these publications, and the relevance of the corresponding skills or expertise to this project;
  • Documentation of academic degrees (copy of degree certificates etc.)
  • Applications without the above-mentioned required documents will not be assessed.

    The applicant will be accessed according to the Appointment Order from the Ministry of Science, Technology and Innovation of 13 March 2012.

    The IT University may use shortlisting in connection to the recruitment process. In case of shortlisting, the Chair of the hiring committee selects applicants for assessment in consultation with the hiring committee after the application deadline. All applicants are notified whether their application has been passed for assessment. The shortlisting of candidates for assessment is based on the criteria in the job posting.

    Further information

    Questions about the position can be directed to Associate Professor, Anna Rogers, IT University of Copenhagen, [email protected]. Early contact with our faculty staff is encouraged. The list of PI's upcoming talks, where it may be possible to meet in person, can be found in this page.

    Questions related to the application procedure may be directed to HR, [email protected].

    Application procedure

    You can only apply for this position through our e-recruitment system. Apply by pressing the button "Apply for position" in the job announcement on our website: https://

    Please read the guidelines for applicants carefully before filling in the application form. You can see the guidelines through this link: https: //

    The IT University might upon request in the interview process contact 1-2 references.

    Please note that all application material will be destroyed after the assessment.

    Application deadline: 31 May 2024 at 23:59 CEST.

    Applications/enclosures received at ITU after the application deadline will not be taken into consideration. If you submit an application, it is your responsibility to ensure that it arrives before the deadline so please allow sufficient time for upload of publications and other documents.

    The IT University invites all qualified researchers regardless of age, gender, religious affiliation, or ethnic background to apply for the positions.

    IT University of Copenhagen, Rued Langgaards Vej 7, DK-2300 Copenhagen S


    From this employer

    Recent blogs

    Recent news