Multimodal AI Framework for Quality Monitoring and Nuanced Understanding
About the Project
The College of Business, Technology and Engineering draws on talents, expertise and facilities across Sheffield Hallam University. The vision is to be the leading provider of applied research excellence delivering computing, science and engineering innovations meeting the development needs of industry.
PhD Research Topic
This project aims to develop a Multimodal AI Framework to work with different modalities of data, explore the complex interactions between them to support reliable decision-making and improved prediction. This project will be conducted in collaboration with Advanced Food and Innovation Centre.
Unimodal AI typically handles single form of data and is the dominant method in research. However, due to its failure to effectively handle the complex interactions, recently a gradual shift towards multimodal AI frameworks is seen. Different sensing practices have complementary strengths and weaknesses, multimodal approaches leverage this fact across its implementations. For example, in agriculture quality monitoring, to identify any foreign object or visible defects we use images, whereas to identify minute changes that are invisible to naked eyes like molecular composition change, spectroscopy is best suited. Any spoilage signs missed by these can be detected using electronic nose sniffs. By combining different modalities, vital signs missed by one sensor can be caught by another, thereby reducing the chances of error. Monitoring can also significantly reduce agriculture waste by identifying quality issues early, enabling better harvesting, storage transportation, informed decision making and beyond. Currently limited work exists that explores quality monitoring utilising multimodal AI framework.
The PhD project shall work towards bridging this gap by exploring multimodal AI framework, enhance decision-making experience and identify different data modalities for quality monitoring. It will investigate how data modalities can be integrated to capture more context, navigate complexity, reduce ambiguities and maintain performance. Moving beyond usual unimodal techniques it intends to achieve following objectives:
- Identify data modalities best suited for quality monitoring, perform cross modalities synchronization to translate the complex interactions into quality cues.
- Develop a richer context-aware specialised multimodal AI framework for handling heterogeneous data for improved generalization.
- Conduct rigorous experiments to investigate the effectiveness of multimodal data fusion framework.
Areas of theoretical exploration
- Multimodal data Selection: Utilising multimodal data to exploit complementary information from different resources for reliable decision making.
- Representation Learning: Embedding different modalities into one shared representation allows model to learn relationships between them.
- Cross Modal Representation: Knowledge from one modality used to improve another.
- Modality Fusion: Define strategies for how/when information from different modalities is combined.
This project can explore other domains of applications beyond agriculture, such as environment monitoring. Please, contact the lead academic to discuss it further.
Eligibility
Applicants should hold a 1st or 2:1 Honours degree in a Computing related discipline. A Master’s degree in a related area is desirable. We welcome applications from all candidates irrespective of age, pregnancy and maternity, disability, gender, gender identity, sexual orientation, race, religion or belief, or marital or civil partnership status.
English language requirements of IELTS 7 with a minimum score of 6.5 in all test areas (or equivalent) are mandatory if English is not your first language. Qualifications should have been taken within the last two years.
How to apply
All applications must be submitted using the online application form.
As part of your application, please upload:
- A research proposal (max. 1500 words) in your own words, briefly outlining the proposed research, the current knowledge and context referencing key background literature; a proposed methodology or approach to answer the key questions, and any potential significance or impact of the research
- A two-page (maximum) CV (submitted as a single PDF file with the research proposal)
- Copy of your highest degree certificate and transcript.
- Non-UK applicants must submit IELTs results (or equivalent) taken in the last two years and a copy of their passport.
We strongly recommend you contact the lead academic (Dr. Pratikshya Sharma - Pratikshya.Sharma@shu.ac.uk) to discuss your application.
Find Your Best Opportunity
Tell them AcademicJobs.com sent you!
