Android Application For Food Label Recognition To Ensure Safe Food Consumption Based On User Allergen Information Leveraging OCR
Android Application For Food Label Recognition To Ensure Safe Food Consumption Based On User Allergen Information Leveraging OCR
2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT) | 979-8-3503-3509-5/23/$31.00 ©2023 IEEE | DOI: 10.1109/ICCCNT56998.2023.10307054
Abstract— The increase in the prevalence of food allergies Keywords—Allergen, Optical Character Recognition(OCR),
is a concerning issue in today’s world. According to some Firebase, Application Programming Interface(API)
estimates, 3 percent of Indians may already have food
allergies, the majority under 40 years of age. Food allergies
cause roughly 30,000 emergency treatments and 100 to 200
I. INTRODUCTION
deaths per year in the nation. Additionally, there is a lack of
awareness and understanding of food allergies in India, which In the present day, the majority of people are unaware of
can lead to delayed diagnosis and treatment. This brings in a the allergens used in packaged food items. Food allergies
need for technology-based solutions to help individuals who affect millions of people worldwide and can be
have food allergies to identify allergens in their food and to life-threatening. For people with food allergies, it can be a
make conscious and informed decisions in their day-to-day challenge to know what foods to avoid and how to manage
consumption of food. This research paper presents an their condition. While there are existing solutions available
application that helps people with food allergies and to help people with food allergies, such as reading food
intolerances identify potential allergens in packaged foods. The labels and carrying epinephrine auto-injectors, these
scope of the project involves the integration of multiple solutions are not always effective or convenient. Due to the
technologies wherein the app utilizes optical character increasing amount of additives in packaged food products, it
recognition (OCR) from the Firebase API to extract the name
is essential to promote nutrition literacy amongst
of the food from a picture taken by the user, and then
consumers. Nutrition literacy can be generally described as
cross-references the name with a dataset of allergen
information. The application provides a user-friendly interface
the capacity an individual has for gathering and synthesising
that is easy to navigate and provides clear alerts, Our nutrition information to make healthy decisions in their
application is specifically designed to ensure that the user is daily lives. One important measure of nutritional literacy is
thoroughly informed of the contents of their food. This brings using the food product label, or the frequency with which
a revolutionary aspect to dietary consumption by allowing one uses a food label. Therefore, it is critically important for
users to have the freedom of consuming their food of choice food labels to be readable at the point of purchase so that
without having to worry about their allergies and other side consumers will be encouraged to use them. In recent years,
effects. In conclusion, this paper demonstrates the successful technology has emerged as a potential solution to help
development and implementation of an Android application people with food allergies manage their condition more
that can help in improving the lives of millions of people effectively.
worldwide who suffer from food allergies and intolerances and
can be further developed and customized to meet specific user In this paper, we present a one-stop android application
needs. called “Nutricate” to help people with food allergies avoid
foods that might cause an allergic reaction. The objective of
the Nutricate application is to facilitate the consumer in behaviours and reduce the risk of chronic diseases[7]. The
making informed decisions keeping their allergies in mind. user perception of mobile applications for diet and physical
This in turn improves public health by making consumers activity tracking can be seen to be very positive and
aware. This simplifies the tedious process of going through provides a new support system in their daily activities[8].
the entire ingredient list in order to avoid consuming The use of Firebase has its benefits and challenges in
specific allergens. The application was developed in android Android app development, a few such advantages are its
as the majority of the Indian population uses android as their real-time database and cloud messaging features[9]. While
primary handheld operating system, hence allowing us to researching the technical aspect of the application, we came
reach a wider audience. across the use of branded food databases as a tool to support
nutrition research and monitoring of the food supply, and
The application allows users to take a picture of how the introduction of databases in the particular field
packaged foods with their names in the picture. The name is could enhance food labelling and monitoring practices[10].
then read from the picture using OCR from the firebase API On the comparison of keyword searches in information
and cross-referenced with a dataset of allergen information. retrieval and relational database system, it can be seen that a
The user's allergen information is already stored in the app, combination of both can be used to achieve better
so the app can convey to the user whether the food they results[11]. The evaluation of the Firebase Realtime
scanned will be allergic to them or not. This paper presents databases’s performance in terms of read/write speed,
the methodology used to develop the application, an concurrency, and scalability with respect to MySQL
evaluation of its effectiveness, and the implications of this Databse[12]. The application that utilizes Firebase Machine
technology for people with food allergies. Learning Kit to recognize text in images, is an efficient tool
for text recognition in mobile applications[13]. The
elaboration of a comprehensive Document Image retrieval
II. LITERATURE SURVEY and its advantages such as keyword search while taking up
The literature review mainly focuses on the use of the least amount of space[14]. The system that utilizes OCR
technology in food labeling and monitoring practices. One technology to extract information about packaged foods acts
of the most important objectives is to identify the as a safety check system for food inconvenience patients
advancements in food labeling and monitoring technologies and such a system can behave as a game-changer in
and their importance in improving consumer health and ensuring the safety in consumption of packaged foods[15].
nutrition. The papers reviewed in the literature survey The study to develop a practical and accessible solution
majorly discuss the different technologies, such as mobile for consumers to detect artificially ripened fruits[16]. Using
apps, databases, and machine learning tools, used to monitor image processing techniques on smartphone-captured
food additives, etc. photographs of the fruits, the suggested method discovered
Current allergen labeling practices are emphasizing the the features that distinguish artificially ripened fruits from
importance of clear and concise labeling in the food industry naturally ripened fruits. This was accomplished with a high
to improve the accuracy and accessibility of food labeling accuracy rate of 91%, demonstrating the effectiveness of the
information[1]. The study highlights the need for accurate method in identifying the presence of calcium carbide in
food labeling, especially for allergen detection, to reduce the mangoes. The utilization of Tesseract OCR library to
risk of adverse reactions. The use of ontologies helps to recognize numbers on food labels and packaging[17]. This
support the integration of heterogeneous food-related data method is a low-cost and efficient way to ensure precise and
on the IoT platform[2]. A mobile advisor application can reliable product identification in the food business. These
enhance the consumer’s ability to make informed food studies provide significant insights and approaches that can
choices, and mainly highlights the importance of providing help to improve food safety and consumer protection
accurate and accessible food labeling information to solutions.
promote healthy eating practices[3]. Similarly, a mobile A few other related apps that we have come across are
application called “FoodScan”, allows consumers to scan “Foodmaestro Food Finder”, which is an app that allows
their grocery receipts to track the nutritional value of their users to scan barcodes or search if they contain potential
food which indeed promotes healthy eating habits by allergens or meet dietary requirements. “Is it Vegan?” is also
providing personalized nutrition recommendations[4]. A a similar mobile application that scans barcodes and
Smart Food Scanner system using a mobile edge computing provides information on whether the product contains any
to analyze label information in real-time can improve the animal-derived ingredients. “ShopWell” as an app can assist
accuracy and speed of food monitoring and promote healthy users to make healthier food choices based on dietary
eating practices[5]. preferences and goals by providing nutrition information for
Investigations on consumer awareness of food additives scanned or searched products. These applications are reliant
and their effects on health highlights the need for accurate on pre-existing barcodes that may be a hassle for users to
food labelling and education on food additives[6]. The look for and scan. Our aim was always to make the user
outcome of a systematic review and meta-analysis to experience palatable and so just scanning the food label is
examine the effectiveness of app-based mobile interventions enough in order to obtain the right information on the
on nutrition behaviours and health outcomes highlights the product that they wish to know more about. Hence, taking
potential of mobile applications to improve nutrition into account all the developments that have taken place in
E. Testing
To ensure that the android application works correctly Fig 2(a) Fig 2(b)
and provides accurate allergen information, we conducted Fig 2: Startup and Allergen Input Screens in the Application
some tests on the application. We conducted functional
testing to ensure that all the features of the application work The welcome section is shown in fig 2, which would
as intended. The ability to take a photo of a packaged food greet the user as soon as they open the application. Then the
and have the application extract the name of the food using page for the input of allergen can be accessed and this is
OCR, check the extracted food name against the allergen shown in fig 3. These will be the first few pages the user
information database and display a message to the user sees and retrieves allergen information from the users.
indicating whether the food is safe for them to eat or not
were tested. During functional testing, we ensured that all
features worked as expected and that the application
provided accurate allergen information for a wide variety of
packaged foods.
We conducted performance testing to ensure that the
application was fast and responsive. We tested the
application on a range of Android devices and monitored its
performance under various conditions, such as low memory
or slow internet connectivity. Finally, we conducted testing
using various packaged foods. Most of the foods were
detected and gave the expected outputs. Some foods when
scanned under low lighting or at oblique angles were not Fig 3(a) Fig 3(b)
detected as accurately.
Fig 3: Image Capture and OCR Screens in the Application
IV. EXPERIMENTAL RESULTS AND DISCUSSIONS
The next activity, as represented in fig 4 and 5, lets the
This section deals with the results we obtained from the user take a picture of their food product and scan the picture
development of the application and its testing. The figures for the product name, using OCR. Then the name is cross
represent the various activities or pages of the application referenced with the products present in the database, and
and also represent the flow of processes in the way a typical checks if the ingredients contain the specific allergen of the
user may use the application. user.
present in it or not. If the allergen is present then the food, V. PROPOSED IMPROVEMENTS
the user is prompted with the indication that it is safe to Our application may be working as intended and give
consume, or is prompted with the information that it is not the desired output, but there are certainly many ways in
safe to consume if allergen is present. These are represented which we can make improvements to this system. The
in fig 6 and 7. dataset needs to definitely be expanded to include a lot more
The results obtained show that the application is foods. At the moment, the database used was manually
working properly and provides the user with accurate created by us and thus lacks the inclusion of a lot of
allergen information. The user can use this information to different kinds of foods. Hence, adding more food will make
make more informed decisions about the foods that they eat. the application more robust and usable. We could also
incorporate machine learning in order to increase the
We tested the application on multiple devices to ensure accuracy and improve the allergen detection process.
that it works properly on them. Varying levels of network Training a model on a large database of food images and
connectivity and memory spaces were also taken into allergen information, we could potentially create a better
account when conducting these tests. Most devices ran our and more useful application.
application with ease and hence can be widely used by
multiple users. Our User Interface may already be functional, but there
is always room for improvement in terms of usability and
We also tested the application on various packaged food aesthetics. By obtaining feedback from users, we can make
items to check if the text recognition and allergen detection the application more user-friendly and intuitive. The
works well in all circumstances. The application performed application could also allow users to scan barcodes on
fairly well in almost all cases. But there were still issues in packaged foods to extract allergen information, or it could
recognising the text when there was low lighting or if the incorporate social sharing features to allow users to share
name was slightly blurred due to low resolution and poor their allergen profiles with friends and family. This could
focus in the picture. Sometimes, different names were being help their friends and family also know the user’s
recognised, like hyphens or slashes were added when they consumption information. The application can further be
weren’t present and removed when they were. developed for other operating systems like IOS, so that
The database was made manually by us and hence lacks users on those systems could also benefit from allergy
a lot of food items that we would normally find in detection.
supermarkets. Hence, not all food items will be recognised.
This can easily be improved by adding a new exhaustive
database and no changes to the code will be needed, making VI. CONCLUSION
it flexible and robust.
In this study work, we provided an Android app that can
Though these problems persist, for most of the tests, the assist persons who have food allergies in making informed
application runs flawlessly and is accurately able to detect decisions about what they eat. To evaluate if a packaged
the text from images and is able to tell if the allergen is food is safe for a user to ingest, the application uses OCR to
present in the food or not. Hence making it very useful for a extract the name of the food from a photo and compares it to
consumer who suffers from allergies. a dataset of allergen information. We discussed the many
parts of the application, such as how OCR was
There are other applications on the market that help implemented, how Firebase Realtime Database was
customers in making informed decisions about their diet. integrated, and how the user interface was created. We also
For example, the Monash University’s FODMAP went over the procedures we used to test and release the
application, provides users, who suffer from irritable bowel application, making sure that it offered users accurate
syndrome, low FODMAP recipes. About 15% of the information on allergens and was simple to use.
population or 1 in every 7th person suffers from this and
hence it is very useful to such individuals, as the symptoms Our study's findings show that the android app is a
can be reduced this way. practical choice for those with food allergies who want a
quick and easy way to find out whether a packaged product
The OCR has an accuracy rate of 91%, where in some is suitable for consumption. By giving users essential
occasions, it can read the wrong text, like missing an information and guiding them away from potentially fatal
apostrophe or a hyphen. The allergen detection rate is fairly allergens, the application has the potential to save lives. The
high, which is about 99%, as it mostly has to exchange data accomplishment of this study work, in our opinion, lays a
with the database, and the only errors can be due to strong platform for subsequent investigations into the
incorrect information present in the database, or an incorrect detection and management of food allergies. There is much
reading of the packaging text. From previous methods such room for additional study and development in this field,
as, using barcodes to find nutritional value and checking for including the application of machine learning and other
allergens, this application is 60% faster and when compared cutting-edge technologies to boost allergy detection's
to manual methods, it is 80% faster and more efficient for precision and effectiveness. Overall, we are happy with the
the user. results of our study project, and we anticipate that those who
have food allergies would benefit from our android
application.