Senior Data Engineer Research Data Platform
Position
Non academic staff (supporting staff)
Irène Curie Fellowship
No
Service(s)
Library and Information Services
FTE
1,0
Date off
08/05/2024
Reference number
V81.7394
Job description
The goal of the Research Data Infrastructure Lab (RDI Lab) is to support our
university's academics throughout the research life-cycle, with means and
expertise required for them to design and materialize their research by using
modern data platforms and tooling. As research matures, the accompanying
complexity of the technology stack supporting the work also increases. The RDI
Lab aims to offer the right technical abstraction for various research
profiles by leveraging industry-hardened, but also experimental, solutions.
The RDI Lab covers a broad landscape where it is futile to ‘do it all'.
Therefore, we are looking for people who can smartly navigate the domain and
are capable to discuss the reasoning behind several approaches with our
researchers to support them in their decision-making and our platform design.
The RDI Lab is a young team, with a lot of room for personal development and
growth. We are looking for people who are not afraid to take initiative and
are willing to take on a pioneering role in the development of our services.
The RDI Lab covers the entire research data lifecycle ranging from planning
phase to data collection, analysis, publication and sharing of data with FAIR
principles in mind. Over the past years, the team has focused on establishing
the foundational components such as version control systems and CI/CD
(GitLab, GitHub), infrastructure for deploying applications (Azure), lab
support systems (ClusterMarket), data platforms (Databricks, Microsoft
Fabric), Atlassian and other essential tooling for researchers.
We are starting to develop a Research Data Lakehouse to ingest and make
available data from various sources. We also want to offer a Trusted Research
Environment to process sensitive data. We are looking for a
Data Engineer with a proactive mindset who can collaborate with our
multidisciplinary team and contribute to the development and acquisition of
these and other systems together with our researchers, institutes, and
engineers within the team.
What does the job entail?
As Data Engineer, you will be co-creating solutions for the Research Data
Lakehouse and Trusted Research Environment developments.
You will collect, load, prepare, clean and deliver access to data to a
wide range of research stakeholders.
You will ensure that proper infrastructure is implemented to guarantee
data quality and security practices are in place.
You will become an essential part of our RDI Lab team to develop and
further operationalize new tooling and services for our university.
What will you be doing?
Working on a data lakehouse implementation based on Azure Databrick,
together with our team and occasionally with third-party vendors.
Designing, implementing, and maintaining data pipelines for data
ingestion, processing, and transformation - regarding both realtime and
batch data.
Implementing required cloud infrastructure based on Terraform and using
CI/CD.
Propose and implement cloud-based strategy focused on reducing maintenance
and costs.
Work with our cloud platform team to harden security.
Job requirements
Proficiency in Databricks: This includes understanding Databricks
clusters, notebooks, jobs, libraries, and integrating Databricks with
other services.
Experience with Azure services: A strong understanding of Azure data
services such as Azure SQL Database, Azure Data Factory, Azure Event Hub,
or equivalent services with other cloud providers.
Proficiency in Terraform: Ability to write Terraform scripts for
provisioning and managing infrastructure as code (IaC) on cloud
platforms.
Strong programming skills in Python: Python is used for data manipulation,
processing, and automation tasks.
Solid understanding of infrastructure concepts: Knowledge of networking,
security, virtual machines, containers, storage solutions, and other
infrastructure components relevant to cloud environments.
Conditions of employment
A challenging job in a dynamic setting, where you can develop both your
technical skills and communication competences;
A unique mix between academic and industrial environment, due to our close
collaboration with research groups at the university and with our
industrial partners;
A competitive salary scheme (salary scale 10/11 with max € 5.929,- gross
p/m) in accordance with the Collective Labour Agreement for Dutch
Universities The salary level depends on your experience, education and
knowledge. On top of this you also receive extra annual holiday allowance
(8% in May), an end-of-year allowance (8.3% in November, a so-called
13th check.)
An attractive package of fringe benefits, including an excellent technical
infrastructure, excellent sports facilities, savings schemes, and 41 days
of holiday based on 40 hours a week. Foreign experts may benefit from the
tax ruling facility in order to get a higher net salary, the TU/e
facilitates you in the application to our local tax authority.
Information and application
Do you recognize yourself in this profile and would you like to have more
information about the function, please contact Mart Wetzels – Lead RDI Lab
at telephone 040-2473883 or per e-mail: m.h.wetzelsattue.nl
For other questions about conditions of employment contact internal
Recruiter Rob van der Linden via [email protected].
Application
If you are interested in this vacancy you can apply by uploading your
motivation and your curriculum vitae via the “Apply for this job” button.
In case of equal suitability, internal candidates will have preference over
external candidates.