Doctoral Researcher 16 (CONVERGENCE of HUMANS and MACHINES): Socially-Aware Machine Perception / Väitöskirjatutkija: Sosiaalisesti tietoinen koneaistiminen

Tampere University of Technology
February 15, 2023
Offerd Salary:Negotiation
Working address:N/A
Contract Type:fixed-term period of
Working Time:Full time
Working type:N/A
Ref info:N/A

Tampere University and Tampere University of Applied Sciences create a unique environment for multidisciplinary, inspirational and high-impact research and education. Our universities community has its competitive edges in technology, health and society. www.

CONVERGENCE of HUMANS and MACHINES project at the Faculty of Information Technology and Communication Sciences is looking for a Doctoral Researcher to work on the topic of Socially-Aware Machine Perception.

Convergence is the trailblazing doctoral field, aiming at bringing together expertise of natural sciences & engineering (ENG) and social sciences & humanities (SSH) in multidisciplinary union. As many as 16 doctoral research projects are related to the following developments: affective computing, gamification, augmented reality, cybernetics, ubiquitous connectivity, dispersed computing, AI & machine learning, and robotics & machine perception. Each student will be supervised by two leading senior faculty members (and, sometimes, a PostDoc or company/organization representative). The main supervisor and the co-supervisor are from different research fields: ENG and SSH. Check the end of the position description for the supervisory team details.

Join Convergence … And make a change!

Convergence has 16 funded doctoral positions. The applicant may only apply for up to 3 positions within Convergence. If you are unsure if this (#16) is the correct one – please, check the website with the list of Convergence positions. Make your selection first and only apply the ones that are suitable for your background.

Project background

Convergence is a new research field that brings together academics from Social Sciences & Humanities and Technology & Engineering to train the next generation of Doctoral Expertise and address future challenges and opportunities of multidisciplinary Convergence of Humans and Machines. Convergence is funded by the Jane and Aatos Erkko Foundation.

Convergence is based on the premise that, on the one hand, people are increasingly integrated with technology and our culture and practices are increasingly dictated by information technology. On the other hand, machines are becoming more alive, creative, and dynamic. In other words, human and machine are converging. The project has emerged from the need to redefine and renegotiate the roles of human and machine in society and from the need of cross-disciplinary collaboration understand this rising area.

Job description

We are looking for a Doctoral Researcher to pursue the research towards “Socially-Aware Machine Perception” topic, which operates in the intersection of machine learning, machine perception, and linguistics.

Successful candidates will pursue a doctoral degree at Tampere University. Full-time doctoral studies are expected to be completed in four years.

An applicant may only apply for maximum 3 Convergence positions from this list. This position number is 16.

Short description:

This research will develop methods towards future AI technologies that can learn to understand the world similarly to humans by communicating with them, and represent their knowledge similarly to humans. The focus will be on machine perception methods that can make interpretations about the physical and social world based on sensory signals such as audio and images, and represent their knowledge using natural language. The use of natural language enables representing complex phenomena and communicating knowledge with humans in natural interaction.

In humans, the meaning of audio-visual percepts depends on social roles and context, which cannot be modeled by existing machine perception methods. This doctoral researcher project studies and develops machine perception methods that can model and take into account how social factors affect the interpretation of natural audio-visual scenes. By modeling different perspectives, levels of abstraction, and social context, the developed methods will enable AI technologies that learn from human communication, where the above phenomena are continuously present.

The material will consists of large-scale audio-visual datasets paired with textual descriptions, collected from social media as well from expert- collected databases, to enable covering a wide range of different social contexts. State of the art deep learning methods will be used to obtain context-dependent representations of sensory inputs and text. The developed methods will be evaluated objectively using their predictive capability in a wide range of linguistic audio-visual analysis tasks, such as text-based audio/image/video retrieval, automatic audio-visual captioning, and automatic audio-visual question answering. A new set of audio-visual linguistic analysis tasks will also be developed to evaluate the capability of the models to take into account analyze social factors, for example dialog-based audio-visual question answering.

The social factors will be modeled with linguistic multimodal theories in combination with machine learning methods that can learn from data. The machine learning techniques used for modeling the social context include both unsupervised learning and supervised learning. Unsupervised learning will be used to learn latent social factors automatically from data, by optimizing their predictive capacity in a wide range of linguistic audio-visual analysis tasks. Supervised learning will utilize a set of social roles and contexts identified based on linguistic multimodal data analysis methods. Linguistic frameworks such as cognitive and systemic-functional theories, and corpus and discourse analysis, will be used in meaning analysis models.

The expected results will lead to AI technologies that can learn to understand audio-visual scenes by communicating with humans, thanks to their capability to model social factors in human interpretations of such scenes.

Tentative Research problems/questions:

Development of machine perception methods that can model and take into account how social factors affect the interpretation of natural audiovisual scenes.


The applicant needs to have a Master's degree in computer science, statistics, signal processing, or related field, that involves a significant amount of studies on modern machine learning methods. The applicant has demonstrated (during your studies towards the Master's degree or otherwise) the competence and motivation to pursue postgraduate studies. Please be prepared to provide proof of your merits when requested.

Successful candidates must be pursuing or will be accepted to study towards a doctoral degree in the Doctoral Programme of Humans and Technologies (DPHAT). Please visit the admissions webpage https: // www. with-us/programmes/doctoral-programmes for more information on eligibility requirements. Please, pay special attention to the language requirements.

Preferred applicant's qualifications:

Expertise in deep learning for machine vision, audio or speech analysis, natural language processing, or linguistics will be preferred.

Candidates are selected based on the level of promise that they show, and this will be evaluated on the basis of their study and research interest description, their previous success in studies, and merits in writing the doctoral thesis and / or scientific publishing, as well as their background match to the research topic.

Tampere University is a unique, multidisciplinary and boldly forward-looking, evolving community. Our values are openness, diversity, responsibility, courage, critical thinking, erudition building, and learner-centeredness. We hope that you can embrace these values and promote them in your work.

We offer

An applicant may only apply for maximum 3 Convergence positions from this list. This position number is 16.

The position will be filled for a fixed-term period of maximum four (4) years. The starting date is 1 September 2023 or as mutually agreed. A trial period of six (6) months applies to all our new employees.

The salary will be based on both the job requirements and the employee's personal performance in accordance with the salary system of Finnish universities. According to the criteria applied to teaching and research staff, the position of a Doctoral Researcher is placed on level 2—4 of the job requirements scale. A typical baseline starting salary for Doctoral Researchers is approximately 2,500 EUR/month.

We are inviting you to be a part of a vibrant, active and truly international research community. We value interdisciplinarity, as it allows you to expand your research network and exposes you to new perspectives and ideas to solve complex research problems and pursue novel research findings. We are strongly committed to the highest level of scientific research and the provision of high-quality doctoral education.

As a member of staff at Tampere University, you will enjoy a range of competitive benefits, such as occupational health care services, flexible work schedule, versatile research infrastructure, modern teaching facilities and a safe and inviting campus area as well as a personal fund to spend on sports and cultural activities in your free time. Please read more about working at Tampere University. You can also find more information about us and working and living Tampere by watching our video: Tampere Higher Education Community - our academic playground.

Finland is among the most stable, free and safe countries in the world, based on prominent ratings by various agencies. Tampere is the largest inland city of Finland, and the city is counted among the major academic hubs in the Nordic countries. Tampere region is the most rapidly growing urban area in Finland and home to a vibrant knowledge-intensive entrepreneurial community. The city is an industrial powerhouse that enjoys a rich cultural scene and a reputation as a centre of Finland's information society. Tampere is also surrounded by vivid nature with forests and lakes, providing countless opportunities for easy-to-access outdoor adventures and refreshment throughout the year.

Read more about Finland and Tampere:

  • Visit Finland
  • This is Finland
  • Ministry of Economic Affairs and Employment: Welcome to Finland
  • Visit Tampere
  • How to apply

    To address these rising challenges, Convergence is looking for 16 funded doctoral researchers. Note, the applicant may only apply for 3 positions within Convergence.

    Please submit your application through our online recruitment system. The closing date for applications is 15 February 2023 (23:59 EET /UTC +2). Please write your application and all the accompanying documentation in English and attach them in PDF format.

    Please attach only the following documents (in PDF format) to your application:

  • A letter of motivation and description of your research interests (max. 2 page(s)). Named “SurnameMotivationLetter.pdf”
  • The selected applicant will have to apply and be admitted to the doctoral programme prior to the start of the work. The student needs to write a more detailed research plan at that stage.
  • Curriculum vitae according to the TENK template . Named “SurnameCV.pdf”
  • Pdf copy of your MSc and BSc degree certificates, including transcripts of all university records and their English translations (Finnish and Swedish certificates are also accepted).
  • If you do not have the certificates yet, please, include the letter from your university that provides the expected graduation date as well as your current transcript of records. Named “SurnameDegrees.pdf”
  • Certificate of language proficiency “Named “SurnameLang.pdf”
  • The minimum English language test result requirements : Test name: minimum test result (The previous CAE and CPE have been renamed as C1 Advanced and C2 Proficiency): TOEFL iBT / TOEFL iBT Special Home Edition: 92 overall, with no section below 20; IELTS (academic) / IELTS Indicator: 6.5 overall, with no section below 5.5; PTE (academic): 62 overall, with no section below 54; C1 Advanced: C; C2 Proficiency: C1; Finnish National Certificate of Language Proficiency (English): Proficiency level 5
  • Exemptions to the language requirements : It is possible to be exempted from submitting a language test score if you have completed a higher education degree in English in certain countries . If you apply without a valid language test result, carefully examine the exemptions that your studies fulfil the requirements.
  • Either 2 Recommendation letters preferably OR the contact details of two referees if the first option is not possible, and with explained reasoning, e.g., your M.Sc. supervisors / employers. Named “SurnameRecommendations.pdf”
  • For further information, please contact the supervisors:

    Professor Tuomas Virtanen , [email protected]

    Associate Professor (tenure track) Maija Hirvonen , [email protected]

    Associate Professor (tenure track) Esa Rahtu , [email protected]

    Associate Professor (tenure track) Okko Räsänen , [email protected]

    For general question about Convergence , contact: Coordinator, Dr. Aleksandr Ometov , [email protected]

    For questions about the doctoral studies enrollment, check DPHAT doctoral programme contacts.

    Summary in Finnish

    Tampereen yliopisto ja Tampereen ammattikorkeakoulu muodostavat yhdessä Suomen toiseksi suurimman monitieteisen, innostavan ja vaikuttavan tutkimus- ja oppimisyhteisön. Korkeakouluyhteisömme osaamiskärjet ovat tekniikka, terveys ja yhteiskunta. Lue lisää: www.

    ITC on käynnistänyt uuden monitieteisen tutkimussuunnan Ihmiset ja teknologiat -tohtoriohjelmaan (DPHAT) sijoittuvan CONVERGENCE – Ihmisen ja koneen konvergenssi -tutkimusalan puitteissa.

    Maailma on täyttymässä erilaisista koneista, autonomisista ajoneuvoista, reaaliaikaisista käännöstyökaluista, puettavasta teknologiasta ja muista laitteista, joissa ihmisen ja koneen välinen raja on katoamassa. Ihmisen ja koneen konvergenssi -ohjelman päätavoitteena onkin tutkia tulevaisuutta, jossa tietoa, asiantuntemusta ja taitoja voidaan siirtää reaaliajassa ja jossa tekoälyyn perustuvat toimijat ymmärtävät ja aistivat ihmistä reagoiden tähän ennakoiden ja eettisesti. Kehitys näyttää johtavan modernin ihmisen ja koneiden yhteistoimintaan ja rinnakkaiseloon eli konvergenssiin, joka tuo mukanaan täysin uusia tutkimusongelmia.

    Näiden haasteiden ratkaisemiseksi ITC-tiedekunta rekrytoi 16 palkkapaikallista väitöskirjatutkijaa. Jokaista opiskelijaa ohjaa pääohjaaja ja sivuohjaaja, jotka ovat eri aloilta (toinen yhteiskunta- ja humanistisista tieteistä ja toinen tekniikan alalta).

    Luodaan uutta eri alueet yhdistävää osaamista yhdessä!


    Väitöskirjatutkija (Sosiaalisesti tietoinen koneaistiminen)

    Hankkeessa kehitetään menetelmiä jotka pystyvät tekemään tulkintoja äänesta ja kuvasta, ja esittämään niitä luonnollisen kielen avulla. Projekti kehittää koneaistimismenetelmiä jotka voivat huomioida miten sosiaaliset tekijät vaikuttavat audio-visuaalisten aineistojen tulkintaan.

    Hakuohjeet ja lisätiedot

    Jätäthän hakemuksesi yliopiston sähköisellä hakulomakkeella (linkki löytyy tämän ilmoituksen alta). Katso tarkemmat ohjeet ja yhteystiedot englanninkielisestä ilmoitukselta.

    Hakuaika tehtävään päättyy 15.2.2023 klo 23.59.

    Application period starts: 2022-12-30 13:00Application period ends: 2023-02-15 23:59

    From this employer

    Recent blogs

    Recent news