Does our social life influence our nutritional behaviour? Understanding nutritional habits from egocentric photo-streams

Andreea Glavan, Alina Matei, Petia Radeva, Estefania Talavera*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review


Nutrition and social interactions are both key aspects of the daily lives of humans. In this work, we propose a system to evaluate the influence of social interaction in the nutritional habits of a person from a first-person perspective. In order to detect the routine of an individual, we construct a nutritional behaviour pattern discovery model, which outputs routines over a number of days. Our method evaluates similarity of routines with respect to visited food-related scenes over the collected days, making use of Dynamic Time Warping, as well as considering social engagement and its correlation with food-related activities. The nutritional and social descriptors of the collected days are evaluated and encoded using an LSTM Autoencoder. Later, the obtained latent space is clustered to find similar days unaffected by outliers using the Isolation Forest method. Moreover, we introduce a new score metric to evaluate the performance of the proposed algorithm. We validate our method on 104 days and more than 100 k egocentric images gathered by 7 users. Several different visualizations are evaluated for the understanding of the findings. Our results demonstrate good performance and applicability of our proposed model for social-related nutritional behaviour understanding. At the end, relevant applications of the model are discussed by analysing the discovered routine of particular individuals.

Original languageEnglish
Article number114506
JournalExpert systems with applications
Publication statusPublished - 1 Jun 2021
Externally publishedYes


  • Pattern discovery
  • Egocentric vision
  • Nutrition
  • Behaviour understanding
  • Lifelogging
  • Deep learning


Dive into the research topics of 'Does our social life influence our nutritional behaviour? Understanding nutritional habits from egocentric photo-streams'. Together they form a unique fingerprint.

Cite this