IEEE TMM SI on Food Data Analysis and Applications

IEEE TMM  Special Issue on 

When Multimedia Meets Food: Multimedia Computing for Food Data Analysis and Applications

Food is central in our life for its fundamental role in human health. The deployment of various networks (e.g., Internet of Things (IoT) and mobile networks), databases (e.g., nutrition tables and food compositional databases), recipe-sharing websites (e.g., Yummly and Meishijie) and social media (e.g., Facebook, Instagram and Twitter) has accumulated unprecedented amounts of food-relevant data, providing new perspectives for the food science and industry to understand food consumption. Compared with other types of multimedia data, there are unique characteristics for food data. For example, food has more diverse modality. Besides common multi-modality information, such as food images, recipe text, cooking videos and other rich attributes, there are some unique modalities, such as smell and touch. In addition, the sensors that generate the data modality is various. For example, we can use wearable and mobile sensors to log food intake through image analysis, speech recognition and chewing pattern classification. The advances in optical imaging techniques can also generate various food modalities for food quality and safety evaluation in the food industry, e.g., RGB, RGB-D, near-infrared spectroscopy, multispectral/hyperspectral signals and structured-illumination reflectance imaging data. Another important characteristic is multi-scale. There are not only macro-food data, such as dishes and ingredients, but also the micro-food data, which is at the molecular level or less, such as various nutrients (e.g., protein, fat, carbohydrate, minerals and vitamins). Multimedia, riding on the wave of artificial intelligence, offers powerful tools to analyze and understand such multi-source multi-modality multi-scale food data. Multimedia-based food analysis has high potential in addressing the long-standing issues in healthcare sector, such as nutrient intake assessment, precision nutrition and health. On the other hand, it also brings new challenges for unique characteristics of food data. For example, due to the heterogeneous food data and inconsistent standardization formats, a new way of linking, associating and organizing these data in a sensible way for deep exploration is highly demanded. The generation mechanisms of smell and touch are different from food images and cooking videos and require new ways of processing and understanding. Therefore, one key technical challenge is how to conduct effective multisensory data representation, fusion and learning. Considering distinctive difference between macro-food and micro-food data, another challenge is how to effectively integrate both macro-food data and micro-food data to enable personalized nutrition and health. Solving these problems calls for new and evolving multimedia computing technologies for food data analysis and applications. The interdisciplinary nature requires us to consider relevant theories and domain knowledge from different fields like food science, nutri-informatics, gastronomy, agricultural science, health, medicine and social science. The collision of food and multimedia technologies would contribute to new scientific findings to food relevant fields like food science, nutrition and industry, and in turn promote the development of multimedia technologies.

The goal of this special issue is to provide a premier forum for researchers and engineers to present their recent studies on multimedia computing for food data organization, analysis, understanding and applications. The list of possible topics includes, but not limited to: