Post-Doctoral Research Visit F/M [Campagne Postdoctorant DRI 2021] Learning Adaptive Communication Graphs for Decentralized Federated Learning

Inria

France

July 10, 2021

Description

2021-03792 - Post-Doctoral Research Visit F/M Campagne Postdoctorant DRI 2021 Learning Adaptive Communication Graphs for Decentralized Federated Learning

Contract type : Fixed-term contract

Level of qualifications required : PhD or equivalent

Fonction : Post-Doctoral Research Visit

About the research centre or Inria department

The Inria Lille - Nord Europe Research Centre was founded in 2008 and employs a staff of 320, including 280 scientists working in fourteen research teams. Recognised for its outstanding contribution to the socio-economic development of the Hauts-De-France région, the Inria Lille - Nord Europe Research Centre undertakes research in the field of computer science in collaboration with a range of academic, institutional and industrial partners.

The strategy of the Centre is to develop an internationally renowned centre of excellence with a significant impact on the City of Lille and its surrounding area. It works to achieve this by pursuing a range of ambitious research projects in such fields of computer science as the intelligence of data and adaptive software systems. Building on the synergies between research and industry, Inria is a major contributor to skills and technology transfer in the field of computer science.

Context

Magnet (Machine Learning in Information Networks) is an Inria project-team located in Inria's Lille Nord Europe research center and is also part of CRIStAL (UMR CNRS 9189). In recent years, Magnet has developed a strong expertise in federated/decentralized and privacy-preserving machine learning algorithms. Aurélien Bellet (researcher at Inria) and Marc Tommasi (Professor at the University of Lille), who will supervise the postdoctoral researcher, have published several papers at top ML conferences and co- organized several national and international workshops on these topics.

This project will stimulate existing and emerging collaborations with the SaCS (Scalable Computing Systems Laboratory) group of Anne-Marie Kermarrec at École Polytechnique Fédérale de Lausanne (EPFL).

This post-doctoral fellowship will be funded by the EPFL-Inria International Lab:

https: // project.inria.fr/epfl-Inria/post-doctoral-fellowship-opportunity-2021/

Applications will need to be submitted to postdoc-dri@inria.fr before July 10, 2021 with all of the following documents:

  • the completed summary sheet
  • Research project including subject title, research program, work plan and planned visits during the post-doc, the duration of the post-doc (between 12 and 24 months) and the desired starting date (default start date is November 1st, 2021 and not later than January, 1st 2022)
  • Detailed CV with a description of the PhD and a complete list of publications with the two most significant ones highlighted Motivation letter from the candidate

  • 2 letters of recommendations

  • Letters of support from the host Inria research team and from the host international partner
  • Passport copy
  • Assignment

    The postdoctoral researcher will conduct research in the area of federated/decentralized machine learning.

    Federated Learning (FL) allows a set of data owners to collaboratively train machine learning models while keeping their datasets decentralized. Decentralized FL algorithms, in which participants communicate in a peer-to- peer fashion along the edges of a network graph, are popular choices thanks to their scalability and privacy properties. However, how to construct the communication graph so as to optimize or balance certain criteria (convergence speed, communication cost, generalization performance, privacy guarantees) remains an open question. In this postdoc, we will develop approaches for learning communication graphs in a data-dependent fashion, as well as new decentralized FL algorithms that are able to efficiently adapt the communication graph during training without exchanging raw data.

    Main activities

    The topic of this postdoctoral position is to study, both theoretically and empirically, the role of the network topology in decentralized FL and to develop approaches for learning such communication graphs in a data-dependent fashion. Ultimately, the goal is to design new decentralized FL algorithms that are able to efficiently adapt the communication graph during training so as to optimize or balance certain criteria (convergence speed, communication cost, generalization performance, privacy guarantees) without exchanging raw data. We plan to illustrate the relevance of the developed approaches on real- world data from standard benchmark datasets but also concrete applications (e.g., from the medical domain).

    In particular, we would like to investigate some of the following questions:

  • How to choose good neighbors adaptively in decentralized SGD, e.g., based on importance sampling strategies;
  • How to infer an optimal communication graph in a decentralized manner based on assumptions on the data distribution across nodes;
  • How to incorporate formal differential privacy constraints, characterizing the role of topology and the associated privacy-utility-efficiency trade- offs;
  • How to extend the above techniques to learning personalized models, with provable generalization guarantees;
  • How the choice of topology can affect the robustness (or lack thereof) of the algorithms to malicious participants.
  • Skills

    The applicant must hold a PhD in machine learning or related fields. She/he is expected to have strong mathematical skills (e.g., probability, statistics, linear algebra, numerical optimization). Some knowledge in federated learning or distributed algorithms is a plus.

    Benefits package
  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage
  • Remuneration

    Gross monthly salary (before taxes) : 2 653 €

    General Information
  • Theme/Domain : Optimization, machine learning and statistical methods Statistics (Big data) (BAP E)

  • Town/city : Villeneuve d'Ascq

  • Inria Center : CRI Lille - Nord Europe
  • Starting date : 2021-11-01
  • Duration of contract : 2 years
  • Deadline to apply : 2021-07-10
  • Contacts
  • Inria Team : MAGNET
  • Recruiter : Bellet Aurelien / aurelien.bellet@inria.fr
  • About Inria

    Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.

    Instruction to apply

    CV + application letter + recommendation letters + List of publications

    Defence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.

    Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities.

    Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.

    Similar Jobs

    Add to favorites Read more...

    Post-Doc : Development Of Algorithms For Decentralized, Resilient Federated Learning.

    Location: France › GIF-SUR-YVETTE together researchers on distributed ledger technology, privacy and machine privacy-preserving and resilient federated learning. keeping the training data decentralized (communicating only the model Hence, in traditional federated learning, a central server...

    Technische Universität München

    Germany Jun 3, 2021

    Add to favorites Read more...

    PhD in Deep Federated Learning with Medical Imaging (Data Heterogeneity)

    PhD in Deep Federated Learning with Medical Imaging (Data Heterogeneity) The Albarqouni lab develops innovative deep Federated Learning (FL) to, i) handling distributed DL models with data heterogeneity including non of data heterogeneity including non-iid...

    Technische Universität München

    Germany Jun 3, 2021

    Add to favorites Read more...

    Postdoc in Deep Federated Learning with Medical Imaging

    Postdoc in Deep Federated Learning with Medical Imaging The Albarqouni lab develops innovative deep Federated Learning algorithms i) handling distributed DL models with data heterogeneity including non robustness in Federated Learning algorithms. Developing state-of-the-art (SOTA)...