Received November 27, 2020, accepted December 16, 2020, date of publication December 21, 2020,

date of current version December 31, 2020.

Digital Object Identifier 10.1109/ACCESS.2020.3046031

FoodScan: Food Monitoring App by Scanning the

Groceries Receipts
BEATRIZ SAINZ-DE-ABAJO 1 , JOSÉ MANUEL GARCÍA-ALONSO 2 , (Member, IEEE),
JOSÉ JAVIER BERROCAL-OLMEDA 2 , (Member, IEEE), SERGIO LASO-MANGAS 2 ,
AND ISABEL DE LA TORRE-DÍEZ 1
1 Department of Signal Theory, Communications and Telematics Engineering, University of Valladolid, 47011 Valladolid, Spain
2 Department of Computer Science and Telematic Systems Engineering, University of Extremadura, 10003 Caceres, Spain
Corresponding author: Beatriz Sainz-de-Abajo ([email protected])
This work was supported by the Interreg V-A España-Portugal 2014-2020, under Project 0499_4IE_PLUS_4_E.

ABSTRACT In the mobile device market there is a large number of applications to help people monitor
intake or provide suggestions to lose weight and manage a healthy diet. However, the vast majority of
these apps consume a lot of time by having to introduce food one by one. This paper presents the work
to develop and pilot test a new Android application, FoodScan, aimed at people over 70, specially those
from rural environments or with limited technical knowledge, to manage their food from the items that
appear on their grocery receipts, avoiding the obligation to introduce one by one those foods, and generating
recommendations. To achieve this final objective, specific objectives have been completed as indicated in
the methods section. We conducted a review of current calorie control applications to learn about their
weaknesses and strengths. Different algorithms were tested to expedite the introduction of food into the
application and the most suitable for the FoodScan application was selected. Likewise, several options were
taken into account to create the knowledge base of food, taking into account dietary recommendations
for people over 70 years. Once developed, a pilot evaluation was carried out with a convenience sample
of 109 volunteers in rural areas of Caceres and Valladolid (Spain) and Alentejo (Portugal). They tested
FoodScan for a month after which they completed a user satisfaction survey. 93 % (101/109) believed that
the app was easy to download and install, 66 % (72/109) thought that it was easy to use, 47 % (51/109) noted
that the charts with the recommendations helped them with diet control and 49 % (53/109) indicated that
FoodScan helped them improve healthy eating habits. One-month pilot evaluation data suggested that most
users found the app somewhat helpful for monitoring food intake, easy to download and easy to use.

INDEX TERMS MHealth, android, food intake monitoring, elderly, automatic dietary assessment, Web
scraping, optical character recognition algorithm (OCR), user evaluation.

I. INTRODUCTION Overweight, which affects all age groups, has increased

The guidelines to follow in any healthy diet are summarized to alarming levels as a consequence not only of excessive
in: (1) variety in foods; (2) reduce fat consumption; (3) calorie intake but also of being more sedentary [3], [4].
increase the consumption of complex carbohydrates such as The increase in the production of processed foods, the rapid
fruits and vitamins; (4) increase the consumption of fiber- urbanization and the change in lifestyles have led to a change
rich foods; (5) reduce sugar and salt; (6) maintain the intake in eating habits. More and more people are far from their ideal
of vitamin D and calcium; (7) hydrate; (8) moderate alcohol weight, which is a serious health problem due to related heart
consumption; (9) make 4 or 5 meals a day reducing the diseases, diabetes, stroke, etc. Eating a healthy diet helps
amounts in each one; and (10) exercise regularly. Technology protect us from malnutrition in all its forms, as well as the
is a highly effective tool for controlling diet and physical diseases mentioned above. It is necessary to change people’s
activity [1], [2]. lifestyles, controlling the amount of food we eat and start
eating healthier.
The associate editor coordinating the review of this manuscript and In rural areas, with an increasingly aging population and
approving it for publication was Shen Yin. with medical services distant from their homes, it is observed

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 8, 2020 227915
 B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

that many elderly people who live alone are suffering from
nutritional deficiencies that, in some cases, can cause signif-
icant damage to their physical condition. To determine the
health of the elderly population and avoid these problems,
we should have accurate information about their nutritional
patterns to predict their quality of life [5]–[7].
The objective the FoodScan application is to facilitate the
control of the elder diet, specially of those living in rural
areas. These people have limited technical knowledge and
prefer simple devices. In the development of the application,
it was taken into account that connectivity can also be a
problem. Sometimes the Internet connection does not arrive
in optimal conditions to certain rural areas or they do not
even have any signal. FoodScan is a simple application that
collects the information by scanning the purchase groceries
receipts and that does not require an Internet connection for
its management. We can store food in an integrated way with-
out having to do it one by one. It has been developed for the
FIGURE 1. Flow chart.
Android Operating System, since it has a higher penetration
rate than other operating systems.
The steps that we have carried out throughout the work
have been: (1) to review existing Calorie Control Apps; (2) III. METHODS
to analyse algorithms registering the purchased groceries A. WORK FLOW
and select a food collection to compare; (3) to develop the As a starting point, we carry out a review of mobile appli-
FoodScan App; and (4) to conduct a user evaluation. cations that offer characteristics similar to those we try to
implement in our app. After this comparative review, several
II. RELATED WORK algorithms were analysed to speed up the introduction of food
Technology can help people to maintain a healthy diet [8]. into the application. It was also very important to choose the
A large number of applications have been designed for this most appropriate food knowledge base, taking into account
purpose [9], [10]. Some manage sports activities or physical the food recommendations for people over 70 years old living
exercise plans [11]–[14]. Others focus on food control [15]. in rural areas. After achieving these objectives, we developed
Nevertheless, most of those applications unify both aspects: the application and finally carried out a satisfaction survey
control of food, caloric intake and sports activities. among end users. Figure 1 shows the flow chart followed in
Generally, applications for food control have the main this work.
function of registering and storing meals throughout the day.
The user must enter, one by one, the type of food ingested. B. REVIEW OF CURRENTLY AVAILABLE MOBILE APPS TO
This controls the calories ingested in a day and the specific CONTROL CALORIES
nutrients (proteins, carbohydrates, fats, vitamins), and allows A search was performed through Google, Scopus and Web of
comparisons with the data recorded from previous days. Science. The terms used were: ‘‘best calorie counter apps’’,
Applications can incorporate other functions such as: (1) set ‘‘apps to control calorie intake’’, ‘‘apps to control diet’’, and
goals to lose weight over a period of time; (2) store healthy ‘‘apps to control feeding’’. After an exhaustive examination
food recipes; or (3) include a step counter to complement the of mobile applications, we focused on the three that had
record of calories ingested with those that have been burned the higher number of downloads in order to analyse their
throughout the day doing physical exercises. functionalities and features: Lose It! [20], Fat Secret [21] and
Mobile applications have been developed to scan purchase My Fitness Pal [22]. Five people downloaded these three free
receipts [16], [17]. We found a study to measure the effects applications and tested them for a period of approximately
of front-of-pack interpretive nutrition labels on grocery shop- two months. They tested them simultaneously by entering the
ping using label scanning[18]. In another article, the authors same data in all three applications. It was decided that the
tried to know, by registering receipts, the behavior of buying evaluators would analyse characteristics of the applications
food in households [19]. In addition, that will be effective such as ease of downloading, user registration in the applica-
if there is a commitment by the user to scan the tickets. tion, graphical interface, simplicity of the food registration
However, we have not found any mobile application that process, possibility of scanning food barcodes, and using
photographs and records the groceries receipts to compare graphs and charts for providing the results. After this trial
them over a period of time, and based on that comparison period, opinions on the functionality of the applications were
show recommendations for a balanced diet. shared.

227916 VOLUME 8, 2020

B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

C. ANALYSIS OF ALGORITHMS REGISTERING THE

PURCHASED GROCERIES AND SELECTION OF A FOOD
COLLECTION TO COMPARE
Three options were tested: (1) Web Scraping; (2) object
recognition and; (3) Optical Character Recognition (OCR).
Web Scraping allows one to extract large amounts of infor-
mation from web pages in an automated and dynamic way.
In order to prove its effectiveness, online shopping websites
were crawled with the aim of analysing and comparing the
user’s different purchases over a period of time. The tech-
nique is not efficient for our purpose because pages have
different structures. In addition, data protection problems
must be taken into account [23]–[25]. Finally, people over
70 years old living in rural areas do not use online shopping
websites.
Image recognition identifies objects in an image or video
FIGURE 2. FoodScan use-cases diagram.
sequence and facilitates their subsequent classification
[26]–[28]. To test the algorithm, photographs of the prod-
ucts of the refrigerator were taken several days apart to
see the food that the user consumed during that time inter- higher. Nevertheless, the proposed application has been tested
val. Comparing the photographs, and from a healthy diet in new and more powerful devices with similar results. The
scheme or template, it can be concluded if the user followed functions allowed by the application are:
a balanced diet or needs to buy or consume certain foods.
1) Scan grocery receipts for food registration.
Although it is a good strategy to compare images, the amount
2) Manually add a food to an already registered grocery
of comparisons is very large. The vectorization process con-
receipt, or create a new one.
sumes a large amount of resources, and the methodology
3) View the record of stored grocery receipts, and search
to recognize the similarity between these objects is very
and filter by date.
complex. If we add difficulty in taking a clear and clean
4) See summaries of statistics, by groups of food con-
photograph of the items in the refrigerator, it is concluded
sumed in the week or month and those that remain to
that this technique does not turn out to be the most efficient
be consumed, through graphs.
for our target population.
The Optical Character Recognition (OCR) algorithm con- The domain of the FoodScan ontology would be "Food"
sists of the digitalization of texts that are automatically iden- as the main class. Different food groups are established as
tified from an image, symbols or characters that belong subclasses, namely: Legumes, Vegetables, Fruits, Cereals and
to a certain alphabet [29], [30]. There is a wide range of derivates, Dry fruits, Milk and derivates, Eggs, Meats, and
tools or libraries that make the use of OCR algorithms easier Fish. In the case of meat and fish, new subclasses are estab-
for developers. Asprise OCR SDK provides an API to recog- lished based on the proportion of fat: fatty and lean. Finally,
nize text and image barcodes. The problem is that the demo more than 100 individuals, specific products that can be auto-
has very limited functions [31], [32]. matically recognized by the application, has been included
As a search method to select a food collection to compare, in the ontology, distributed along the different classes. The
we analysed online bibliographic references. It is important relationships that appear in this ontology are only belonging.
to know the recommendations of the specialists, and differen- A food belongs to a certain class, regardless of relationships
tiate the types and groups of food that are classified according between classes. We save and export the ontology in a file of
to the distribution of the food pyramid for people with the age the RDF / XML type. The file uses different brands or labels
range above 70 years [33]. And we must keep it in mind to to indicate classes, individuals and relationships. It is possible
avoid the nutritional disorders of the elderly. to modify this file, to add new individuals to some class,
because it is not static. For the implementation of the ontology
we used the editor ‘‘Protégé’’[34].
D. APP STRUCTURE AND DEVELOPMENT The use-cases diagram shows the structure of the applica-
FoodScan has been developed for a minimum Android 6.0 tion, the actors involved and the different functions performed
version, equivalent to the API level 23, and the version with by those actors. These types of diagrams are designed to show
which the project has been compiled is Oreo 8.1, equivalent the interaction between the system and the user, or another
to the API level 27. Support for older APIs is provided since system. User interactions with the application allows them
the target users of the application tend to own low-end devices to: (1) scan a grocery receipt; (2) register a grocery receipt
that are not frequently updated and the latest data shows that manually; (3) see records; and (4) see recommendations. See
more than 84 % of Android devices run a version 6.0 or in Figure 2.

VOLUME 8, 2020 227917

 B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

Since the application data is personal and unique for that

application it is not necessary to upload it to a remote
server. That is why we have chosen to use SQLite as the inter-
nal database for the Android application. Data management
is done locally and, therefore, personal data is not sent to a
remote server to be analysed, improving the privacy of the
proposed solution.
To generate the graphics, we use MPAndroidChart [35].
We established the recommended weekly quantities for each
food group, and they were all stored together in a numerical
array. Each bar of the graph is composed of two sections. The
first section indicates the food bought. The second section of
the bar presents the food that remains to be ingested. This
amount is the result of the operation: Recommended amount
- Ingested amount.

E. END USER EVALUATION

A small group of volunteers visited rural areas from the
provinces of Caceres and Valladolid (Spain) and Alentejo
(Portugal) to show FoodScan. The population was notified
about the investigation by means of posters placed in the civic
FIGURE 3. FoodScan Activity Diagram.
centers and the town hall. The number of people who attended
the first demonstration meeting exceeded 270 among all the
rural areas visited. Volunteers gave a practical demonstration
of how to download and enter food data into the FoodScan
app to get graphs of results.
About a month after the first meeting, volunteers returned
to conduct the web based survey among those who used
the FoodScan application. A considerably smaller number of
users attended this second meeting: 109 Spanish and Por-
tuguese users. All of them tested the application and were
interested in answering the satisfaction survey. Please, note
that no incentive were offered by the volunteers.
The questions were created by the researchers. It is not
based on an established user satisfaction survey. A person
FIGURE 4. FoodScan architecture. in charge of the Privacy Policy Service of the University of
Valladolid reviewed the survey and recommended to elimi-
nate the questions that could identify the respondents. After
The activity diagram (see Figure 3) shows the workflow following the recommendations, the survey was approved.
that is performed and details the different routes between the In addition, participants were not asked to provide demo-
events of the application. graphic characteristics. We did not consider requesting these
Two tables are created for the design of the relational data because we did not aim to know the health of the partic-
database and implemented using the SQLite database. The ipants. We were only interested in the age of the respondents
‘‘grocery receipts’’ table contains the grocery receipt data in because the application is aimed to people over 70 years old.
the app, storing an automatically generated grocery receipt All the people recruited for the study were over 50 years old
code that identifies it, the date on which this receipt was added with medium-low technological knowledge.
to the database and a text with the address where the image Although the data collected is anonymous and therefore
is stored of the grocery receipt. The ‘‘food’’ table contains in it is not necessary to include transparency clauses on data
each of its records the name of the recognized food and the protection in the survey, an informed consent was also col-
code of the grocery receipt to which it belongs. lected from those who completed the survey. Participants
In Figure 4, one can observe the relationships between were informed by letter of the content of the survey and asked
the modules that integrate FoodScan: (1) OCR algorithm; about their implicit consent. Participants who gave ethical
(2) food ontology; (3) groceries receipt database; and (4) approval were offered the possibility to use tablets to conduct
functions of the result graph. the survey.

227918 VOLUME 8, 2020

B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

TABLE 1. User satisfaction survey. due to the high burden related to entering foods into apps.
Therefore, in order to increase the adherence, users need
applications registering the food intake without requiring
introducing these foods one by one.

B. ANALYSIS OF ALGORITHMS REGISTERING THE

PURCHASED GROCERIES AND SELECTION OF A FOOD
COLLECTION TO COMPARE
The one that was finally used in FoodScan is the OCR algo-
rithm. The OCR character recognition services of web pages
were tested. Some of those consulted were Newocr.com,
Onlineocr.net, Ocr.space, and Tess4j together with tesseract
[36]. In the Android Studio development environment, there
were difficulties in making the Tess4j library compatible.
Finally, Google Vision was used, which allows us to under-
The tool for the development of the survey was Google stand the content of an image by encapsulating powerful
Forms due to its simplicity and because a license is not machine learning models in a simple and easy-to-use way
required for its use. [37]–[39].
A survey was carried out with 7 questions (see Table 1), Foods are categorized into different groups. It is a data
accessible through a link provided to the users of the applica- store that allows us to compare the scanned and recognized
tion. We were especially interested in the answers to ques- foods of the grocery receipts using the OCR algorithm. The
tions regarding the usability and ease of downloading the considered alternatives were: (1) UNESCO Thesaurus [40];
application. We decided that the number of questions in the (2) Database; and (3) Ontologies. For FoodScan we use an
survey should be small, so that people would be encouraged to ontology, because it offers several functions that allow us to
complete it. Thanks to the simplicity of the questionnaire, all build a simple and complete collection. It can be adjusted to
the people who carried out the survey answered all the ques- the needs of FoodScan, to help and advise people who use
tions. These results were transferred to an Excel spreadsheet it in the diet, offering recommendations based on purchases
in order to better analyse the results and generate graphs. made within a certain period of time.

IV. RESULTS C. FOODSCAN APPLICATION

A. SUMMARY OF THE REVIEW OF CURRENTLY AVAILABLE The FoodScan home screen is shown in Figure 5. It offers
APPS three options to add or register a grocery receipt in the app’s
All three applications offer similar features and interfaces. database: (1) take a picture; (2) choose an image from the
The user can set the ideal weight (s)he wants to achieve and gallery; and (3) add a new groceries receipt by selecting
how quickly (s)he hopes to achieve it. Depending on their foods manually. The 3 horizontal lines icon, in the upper
age, weight, height and sex, the application indicates a daily left of the screen, takes us to the menu with the functions
calorie regime. The user has to record the daily food and of the app: (1) back to the beginning; (2) open the record of
physical exercises, can visualize in a graph the calorie and the groceries receipts stored in the database; and (3) initiate
nutrient intake, follow the progress in their marked objec- recommendations or statistics.
tives, modify the quantities or portions of the foods, and see Figure 6 shows the grocery receipts registration screen.
suggestions of healthy recipes. The success of these three It allows one to insert the ‘‘start date’’ and the ‘‘end date’’. The
applications compared to others is that it is possible to scan "Filter" button shows only the grocery receipts between these
the food barcode with the camera of the mobile device and dates. If the dates are not selected correctly, an error message
register the food. The scanned item is recognized if it is will be displayed on the screen. If the dates are not selected,
stored in the database. If the food is not stored, it can be all the grocery receipts registered in the database are shown.
manually included. They are visual applications and have By clicking on a grocery receipt, a pop-up menu will appear
different sections. Graphs show the values of calories and with 3 options: (1) see details, which offers groceries receipt
nutrients in quantities and percentages. It also offers a simple details, the food it contains and an image if it exists; (2)
and navigable interface, with the possibility of configuring delete, which allows the grocery receipt to be deleted; or (3)
the app in several languages. cancel, to close the menu without changes. The user can
But not everything are advantages. The main problem of manually add a food to the grocery receipt among those stored
these applications is that the user must introduce food and in the application. This is done sequentially as can be seen
physical activities one by one. This can be a tedious and in Figure 7.
time-consuming task, since they have to provide the food, Regarding the recommendations of FoodScan based on
the quantity and the time of day at which it is ingested. In the the recorded data, we decided to provide them using visual
long run, there is an increased likelihood of abandonment techniques such as graphs. FoodScan has the advantage of

VOLUME 8, 2020 227919

 B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

FIGURE 7. Sequence to manually add a food to the groceries receipt.

FIGURE 5. FoodScan home screen.

FIGURE 8. Graph with recommendations.

food bought, not the one consumed. As a criterion of colors,

we consider the "traffic light system", whose colors range
FIGURE 6. Menu of the groceries receipt list. from green to dark red, and which mark the different levels
of risk. A Color Code has been established based on the
percentage of food consumed: (1) between 0-25 %, red color;
offering weekly or monthly recommendations. In the case of (2) between 25-50 %, orange color; (3) between 50-75 %,
the weekly recommendation, the user must select the day on yellow color; and (4) between 75-100 %, green color. If the
which that week begins, and the application will take that color of the bar for a certain food group is not green, the user
day and the next six. If it is monthly, the user must select should adjust their diet to meet the recommended amounts (in
the month and year. In both cases, the app will search the grey color). In Figure 8, we see that only three food groups
database for grocery receipts included in this period of time, were added to the application database: (1) vegetables; (2)
and if there are grocery receipts on those dates, it will show cereals and derivatives; and (3) milk and derivatives. All in
the graph with the results of the analysis. If there is not any insufficient quantities and that is why the bars are in orange
grocery receipt in the database, the process is canceled and and red. In a balanced diet, Figure 8 would show all the bars in
the app displays a message to the user warning of the error. green. Therefore, from the visualization of this graph, the user
FoodScan offers a visual analysis of the foods of each knows that the consumption of food should be increased,
group consumed in a given time, comparing them with the if (s)he wants to follow a healthy diet, according to the
amounts recommended by specialists, to follow a balanced recommendations of the dietitians.
diet. If there are groceries receipts stored for that period of
time, a bar graph is shown for each food group with the D. RESULTS OF THE END USER EVALUATION
amount of food consumed, and those that remain to be con- The opinion of the users allows us to know if it has a friendly
sumed according to the established recommended quantities. interface. In addition, the information collected will help us
A possible limitation of FoodScan is that it only tracks the to improve the application in the future.

227920 VOLUME 8, 2020

B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

FIGURE 9. Weight Loss Apps usage.

FIGURE 10. Problems downloading and installing.

Regarding the use of applications for weight loss, more

than half users answered that they did not use any before try-
ing FoodScan. Those who had used weight loss applications
indicated that the most used were My Fitness Pal, Fat Secret
and Lost It! Few users reported having used more than one
application. See Figure 9.
Regarding the download and installation of FoodScan,
65 out of 109 participants (60 %) answered that it was easy
and 36 out of 109 (33 %) very easy (see Figure 10). As can be
seen in Figure 11, regarding ease of use, 4 out of 109 partic-
ipants (4 %) found it difficult, 33 out of 109 (30 %) average
difficulty, 53 out of 109 (49 %) easy and 19 out of 109 (17
%) very easy.
As can be seen in Figure 12, with the exception of 2 of
the total of 109 polled (2 %) who do not consider it useful,
the public appreciates the recommendations shown in the FIGURE 11. Ease of use.
charts. As shown in Figure 13, users generally agree with
the fact that the app helps them improve their healthy eating
habits. In Figure 14, one can see the price users would be
willing to pay if it were marketed. It is an application that,
in general, has liked. 101 out of 109 participants (93 %) would
recommend it to other users (see Figure 15).

V. DISCUSSION AND CONCLUSION

A. PRINCIPAL RESULTS
In this work, we started from the analysis of mobile applica-
tions that offer similar characteristics to those that we try to
implement in FoodScan. After this comparative review, var-
ious algorithms were analysed to accelerate the introduction
of food and the most suitable food knowledge base for people
over 70 years old. Next, we developed the application and
finally we carried out a satisfaction survey among users who FIGURE 12. Do the charts with recommendations help control the diet?.
used the application for a month. Below we summarize the
objectives achieved:
1) Review the currently available apps. The analysis tified is that the information of the food intake has
allowed us to better understand its characteristics, to be manually provide to the analysed apps. Table 2
weaknesses and strengths. The main weakness iden- compares the features of FoodScan with other apps.

VOLUME 8, 2020 227921

 B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

TABLE 2. Comparison of the features of FoodScan and other commercial

apps.

(OCR). OCR was finally used because of the number

of services available and its accuracy. Also, we selected
a food collection to compare. The following alterna-
tives that were considered: (1) UNESCO Thesaurus;
FIGURE 13. Does it help improve your healthy diet habits?.
(2) Database; and (3) Ontologies. We used ontologies
because it allowed us to easily build a complete collec-
tion.
3) Development of the FoodScan application. We based
on the premise that our target audience is people over
70 years old, with limited technical knowledge and
living in rural areas. This population group is generally
reluctant to use technologies and, therefore, the need
to create a simple, intuitive, friendly application that
offers visual results that are very easy to understand
is crucial to increase the user experience. In addition,
FoodScan works without being connected to the Inter-
net, so it does not require any communication for using
it. FoodScan is an Android application that meets the
characteristics and functionalities necessary for these
users to control their food, in an easy and intuitive
way. It allows users to add the food products they
FIGURE 14. What amount of money would you be willing to pay?. purchased by scanning grocery receipts. By means of
an OCR algorithm, the grocery receipts are analysed
and the food that matches to those stored is added to
a grocery receipts register. With these grocery receipts
stored, the app offers users a visual and understandable
analysis of the product bought in each group during
a period of time, comparing them with the amounts
recommended by the specialists.
4) End user evaluation. The results of the satisfaction
survey, completed by users who tried the FoodScan
tool, highlight its ease of installation and handling.

B. LIMITATIONS
The implemented application and this study have some limi-
tation that should be highlighted:
1) The satisfaction survey do not ask for the demographics
FIGURE 15. Would you recommend it?. information of the participants. We did not consider
their inclusion because the only requirement for partic-
ipating in this study was to be over 50 years old. They
Only our application allows you to register the food all met the demand.
included in the ticket by scanning. 2) FoodScan tracks what is purchased, not what is con-
2) Analysis of algorithms registering purchased foods. sumed. Therefore, it cannot be used to estimate the
Three options were tested: (1) Web Scraping; (2) object daily nutritional intake, since some of the purchased
recognition and; (3) Optical Character Recognition groceries could be shared or thrown away. These

227922 VOLUME 8, 2020

B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

drawbacks were explained during the demonstration of and/or speed of the groceries receipt analysis, so that the user
the application and the users were given correct guide- does not have to wait a long time. Another improvement to be
lines. Also, consideration should be given to whether developed is the possibility of making the food analysis more
the person lives alone or not. complete by offering, a more specific study of nutrients and
3) The user should follow a good practice in using the calories ingested.
application, but if (s)he shares, throws away and buys
more than necessary, the results will not be realistic. REFERENCES
We are not able to specify what a person has eaten at a [1] S.-W. Chen, D.-L. Chiang, T.-S. Chen, H.-Y. Lin, Y.-F. Chung, and F. Lai,
certain time, but we get a general idea in a period of a ‘‘An implementation of interactive healthy eating index and healthcare
system on mobile platform in college student samples,’’ IEEE Access,
week or a month. vol. 6, pp. 71651–71661, 2018.
4) Further research is required to determine if users are [2] M. Chukwu, ‘‘Personalized mobile monitor for assisted healthy-living,’’ in
able to persist in app use over the long term and if Proc. IEEE Consum. Commun. Netw. Conf. (CCNC), Jan. 2011, pp. 18–22.
[3] Obesity: Preventing and Managing the Global Epidemic Report of a WHO
the app leads to improvement in dietary intake. We are Consultation., World Heal. Organ., Geneva, Switzerland, 2000.
also interested in knowing the lack of adherence to the [4] M. A. Subhi, S. H. Ali, and M. A. Mohammed, ‘‘Vision-based approaches
application. The responses obtained in a short period of for automatic food recognition and dietary assessment: A survey,’’ IEEE
use of the application cannot offer reliable data on the Access, vol. 7, pp. 35370–35381, 2019.
[5] E. Moguel, J. Berrocal, and J. García-Alonso, ‘‘Systematic literature
improvement in diet habits or weight loss. review of food-intake monitoring in an aging population,’’ Sensors, vol. 19,
no. 15, p. 3265, Jul. 2019.
C. CONCLUSION [6] A. A. Sayer and C. Cooper, ‘‘Early diet and growth: Impact on ageing,’’
Proc. Nutrition Soc., vol. 6, no. 1, pp. 79–85, 2002.
The elderly who are not used to new technologies may find it [7] I. Darnton-Hill, C. Nishida, and W. James, ‘‘A life course approach to diet,
difficult to use mobile applications to control their diet. The nutrition and the prevention of chronic diseases,’’ Public Health Nutrition,
disadvantage of many of these applications is that they require vol. 7, no. 1a, pp. 101–121, Feb. 2004.
[8] S. Pagoto, K. Schneider, M. Jojic, M. DeBiasse, and D. Mann, ‘‘Evidence-
a lot of time to introduce the food consumed throughout the based strategies in weight-loss mobile apps,’’ Amer. J. Preventive Med.,
day. From the user’s point of view, it is not practical and, vol. 45, no. 5, pp. 576–582, Nov. 2013.
in the long run, most of them will abandon this control. Our [9] B. Y. Laing, C. M. Mangione, C.-H. Tseng, M. Leng, E. Vaisberg,
M. Mahida, M. Bholat, E. Glazier, D. E. Morisky, and D. S. Bell, ‘‘Effec-
app is useful and easily manageable and avoids the obligation tiveness of a smartphone application for weight loss compared with usual
to introduce one by one those foods. The analysis of the care in overweight primary care patients,’’ Ann. Internal Med., vol. 161,
purchased food is offered through a bar graph with different no. 10, p. 5, Nov. 2014.
[10] J. Tang, C. Abraham, E. Stamp, and C. Greaves, ‘‘How can weight-loss
colors, so it is simple and intuitive. The elderly, or the people app designers’ best engage and support users? A qualitative investigation,’’
in their care, can control how they are eating or what food Brit. J. Health Psychol., vol. 20, no. 1, pp. 151–171, Feb. 2015.
deficiencies they may have. [11] J. Stephens and J. Allen, ‘‘Mobile phone interventions to increase phys-
ical activity and reduce weight: A systematic review,’’ J. Cardiovascular
It can be used by all kinds of public, regardless of their
Nursing, vol. 28, no. 4, pp. 320–329, 2013.
technical knowledge. Although it is geared towards serving [12] G. Flores Mateo, E. Granado-Font, C. Ferré-Grau, and
people over the age of 70, we hope that it will also be used X. Montaña-Carreras, ‘‘Mobile phone apps to promote weight loss
by anyone in their daily lives and that they will be loyal in the and increase physical activity: A systematic review and meta-analysis,’’
J. Med. Internet Res., vol. 17, no. 11, p. e253, Nov. 2015.
long term. Only by adopting healthy habits we can improve [13] G. Bleser, D. Steffen, M. Weber, G. Hendeby, D. Stricker, L. Fradet,
our quality of life. F. Marin, N. Ville, and F. Carré, ‘‘A personalized exercise trainer for the
FoodScan is an application that we have just developed and elderly,’’ J. Ambient Intell. Smart Environ., vol. 5, no. 6, pp. 547–562, 2013.
[14] P. K. Diwakar, Y. Keun Oh, S.-H. Park, and Y.-R. Yoon, ‘‘Personal digital
will continue to improve thanks to the contributions of users. exercise trainer for managing, monitoring and recording the exercise,’’ in
We will regularly monitor the 109 people who completed the Proc. IEEE Eng. Med. Biol. 27th Annu. Conf., 2005, pp. 3720–3723.
survey on the regular use of FoodScan to control their diet. [15] C. M. Wharton, C. S. Johnston, B. K. Cunningham, and D. Sterner,
‘‘Dietary self-monitoring, but not dietary quality, improves with use of
smartphone app technology in an 8-Week weight loss trial,’’ J. Nutrition
D. FUTURE LINES OF WORK Edu. Behav., vol. 46, no. 5, pp. 440–444, Sep. 2014.
One of the aspects of the application to consider in the future [16] A. Jäckle, J. Burton, M. P. Couper, and C. Lessof, ‘‘Participation in a
mobile app survey to collect expenditure data as part of a large-scale
is to extend it and implement a more complete ontology and probability household panel: Coverage and participation rates and biases,’’
a bigger food collection by adding more quantity and variety. Surv. Res. Methods, vol. 13, no. 1, 2019.
Furthermore, we are already working to expand the food [17] B. Read, ‘‘Respondent burden in a mobile app: Evidence from a shopping
receipt scanning study,’’ Surv. Res. Methods, vol. 13, no. 1, pp. 45–71 2019.
ontology and include users of a wider age range. Another [18] E. Volkova, B. Neal, M. Rayner, B. Swinburn, H. Eyles, Y. Jiang, J. Michie,
important point would be to improve the design of the appli- and C. N. Mhurchu, ‘‘Effects of interpretive front-of-pack nutrition labels
cation based on user ratings. This application is configured in on food purchases: Starlight randomised controlled trial,’’ Obesity Res.
Clin. Pract., vol. 8, pp. 110–111, Dec. 2014.
Spanish, Portuguese and English. It is planned to adapt it to
[19] S. A. French, S. T. Shimotsu, M. Wall, and A. F. Gerlach, ‘‘Capturing the
other languages to reach a greater number of users. spectrum of household food and beverage purchasing behavior: A review,’’
To make the application more powerful, artificial intelli- J. Amer. Dietetic Assoc., vol. 108, no. 12, pp. 2051–2058, Dec. 2008.
gence should be added, capable of learning what items refer to [20] Lose It. Accessed: Oct. 27, 2020. [Online]. Available: https://www.
loseit.com/
a specific food, since a direct comparison is currently made. [21] Fat Secret. Accessed: Oct. 27, 2020. [Online]. Available: https://www.
We will have to weigh that it does not affect the performance fatsecret.com/

VOLUME 8, 2020 227923

 B. Sainz-de-Abajo et al.: FoodScan: Food Monitoring App by Scanning the Groceries Receipts

[22] Myfitnesspal. Accessed: Oct. 27, 2020. [Online]. Available: JOSÉ MANUEL GARCÍA-ALONSO (Member,
https://www.myfitnesspal.com/ IEEE) received the Ph.D. degree in software
[23] O. Castrillo-Fernández, ‘‘Web scraping: Applications and tools,’’ Eur. engineering from the University of Extremadura,
Public Sector Inf. Platform (EPSI platform), Tech. Rep. 10, 2015, in 2014. He is currently an Associate Profes-
pp. 1–31. [Online]. Available: https://pdfslide.net/documents/web- sor with the University of Extremadura, and a
scrapingapplications-and-tools.html Co-Founder of Gloin, Software-Consulting Com-
[24] F. Polidoro, R. Giannini, R. Lo Conte, S. Mosca, and F. Rossetti, ‘‘Web pany. His research interests include software engi-
scraping techniques to collect data on consumer electronics and airfares
neering, mobile computing, pervasive computing,
for Italian HICP compilation,’’ Stat. J., vol. 31, no. 2, pp. 165–176, 2015.
eHealth, and gerontechnology.
[25] R. Penman, T. Baldwin, and D. Martinez, ‘‘Web scraping made simple
with SiteScraper,’’ Penman Web Scraping, pp. 1–10, May 2009. [Online].
Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.654.
745&rep=rep1&type=pdf
[26] J. R. R. Uijlings, K. E. A. Van De Sande, T. Gevers, and
A. W. M. Smeulders, ‘‘Selective search for object recognition,’’ Int.
J. Comput. Vis., vol. 104, no. 2, pp. 154–171, 2013.
[27] P. F. Felzenszwalb and D. P. Huttenlocher, ‘‘Pictorial structures for object
recognition,’’ Int. J. Comput. Vis., vol. 61, no. 1, pp. 55–79, 2005.
[28] D. G. Lowe, ‘‘Object recognition from local scale-invariant features,’’ in
Proc. 7th IEEE Int. Conf. Comput. Vis., vol. 2, Sep. 1999, pp. 1150–1157.
[29] A. Chaudhuri, K. Mandaviya, P. Badelia, and S. K. Ghosh, ‘‘Optical
JOSÉ JAVIER BERROCAL-OLMEDA (Member,
character recognition systems,’’ Stud. Fuzziness Soft Comput., vol. 352,
IEEE) received the Ph.D. degree in computer sci-
pp. 9–41, Dec. 2017.
[30] L. Converso and S. Hocek, ‘‘Optical character recognition,’’ J. Vis. Impair. ence from the University of Extremadura, Spain,
Blind., vol. 84, no. 10, pp. 507–509, Dec. 1990. in 2014. In 2016, he obtained an Associate posi-
[31] M. K. Ugale and M. S. Joshi, ‘‘Improving optical character recognition for tion at the University of Extremadura. He is cur-
low resolution images,’’ IJCSN Int. J. Comput. Sci. Netw., vol. 6, no. 25, rently a Co-Founder of company Gloin, which
pp. 18–20, 2017. is a Software-Consulting Company. His main
[32] A. Farooq, A. K. Khan, and G. Raja, ‘‘Implementation of a speech based research interests include mobile computing, con-
interface system for visually impaired persons,’’ Life Sci. J., vol. 10, no. text awareness, pervasive systems, crowd sensing,
SPEC. no. 9, pp. 398–400, 2013. the Internet of Things, and fog computing.
[33] R. M. Russell, H. Rasmussen, and A. H. Lichtenstein, ‘‘Modified Food
Guide Pyramid for People over Seventy Years of Age,’’ J. Nutr., vol. 129,
no. 3, pp. 751–753, Mar. 1999.
[34] Protégé. Accessed: Oct. 27, 2020. [Online]. Available: https://protege.
stanford.edu/
[35] P. Jahoda. (2016). MPAndroidChart. Accessed: Oct. 27, 2020. [Online].
Available: https://www.webci.tation.org/5mt5TH6pg
[36] C. Patel, A. Patel, and D. Patel, ‘‘Optical character recognition by open
source OCR tool tesseract: A case study,’’ Int. J. Comput. Appl., vol. 55,
no. 10, pp. 50–56, Oct. 2012.
[37] R. Hyam, ‘‘Automated image sampling and classification can be used to SERGIO LASO-MANGAS is currently pursu-
explore perceived naturalness of urban spaces,’’ PLoS ONE, vol. 12, no. 1, ing the Ph.D. degree with the University of
Jan. 2017, Art. no. e0169357. Extremadura, Spain. He is also working with the
[38] A. Bechmann, ‘‘Keeping it real: From faces and features to social Computing and Telematics Systems Department.
values in deep learning algorithms on social media images,’’ in Proc. His research interests include mobile computing,
50th Hawaii Int. Conf. Syst. Sci., 2017, pp. 1–9. [Online]. Available: pervasive systems, context-awareness, edge com-
https://scholarspace.manoa.hawaii.edu/bitstream/10125/41372/1/ puting, and the Internet of Things.
paper0223.pdf
[39] R. Arief, A. Benny, T. Maulana, and H., ‘‘Automated extraction of large
scale scanned document images using Google vision OCR in apache
Hadoop environment,’’ Int. J. Adv. Comput. Sci. Appl., vol. 9, no. 11,
pp. 112–116, 2018.
[40] UNESCO Thesaurus. Accessed: Oct. 27, 2020. [Online]. Available:
http://vocabularies.unesco.org/browser/thesaurus/es/?clang=en

BEATRIZ SAINZ-DE-ABAJO received the Ph.D.

degree (summa cum laude) from the University ISABEL DE LA TORRE-DÍEZ is currently a Pro-
of Cordoba, in 2009. She is currently an Asso- fessor with the Department of Signal Theory and
ciate Professor in Telecommunications Engineer- Communications, University of Valladolid, Spain.
ing with the University of Valladolid, Spain. Her She is also a Leader of the GTe Research Group.
fields of action are the development and evalua- She is the author or coauthor of more than 200 arti-
tion of e-health systems, m-health, medicine 2.0., cles in SCI journals, peer-reviewed conferences
and cloud computing, focuses on topics related to proceedings, books, and international book chap-
electronic services for the information society. She ters. She has coauthored 21 registered innovative
belongs to the GTe Research Group, integrated software. She has been involved in more than
within the UVa Recognized Research Group Information Society. Among 65 Program committees of international confer-
the lines of research, the group works to develop innovative solutions in the ences until 2020. She has participated/coordinated in 37 funded European,
field of health that help patients improve their quality of life and facilitate national, and regional research projects.
the work of health professionals.

227924 VOLUME 8, 2020