IRJET-Augmented Reality based Platform to Share Virtual Worlds
0 likes58 views
This document discusses using augmented reality and virtual reality technologies to create a shared virtual world platform. It proposes combining Google Cardboard and TensorFlow Lite APIs to allow multiple users to simultaneously connect and interact with virtual content overlaid on the real world. The platform would use cloud computing and real-time databases to securely share information and dynamic virtual objects between users in augmented reality.
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072
© 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 887
Augmented Reality based Platform to share virtual worlds
Prof. Dhananjay Gaikwad1, Akash Chikane2, Shrikrishna Kulkarni3, Aishwarya Nhavkar4
1 Professor, Computer Engineering, NBN Sinhgad School of Engineering, Pune
2,3,4 Computer Engineering, NBN Sinhgad School of Engineering, Pune
-------------------------------------------------------------------------------***---------------------------------------------------------------------------------
Abstract: Augmented reality is new emerging technology,
where things in the physical world are mixed with digital
content to increase user experience and simplicity. As
augmented reality technology allows us to create a virtual
world on top of the physical world, this can be used to share
same physical resources with multiple users. The project
scope will be to provide a platform for users to share physical
resources along with dynamic information to provide
enhanced mixed reality experience. Cloud and the real-time
database will be used to create a simultaneous connection
between multiple users and share, creating a secure
connection and provide data security, also users will be able
to set triggers related to physical world i.e. silent mode in
office.
Keywords: augmented reality, cloud, virtual worlds.
I. INTRODUCTION
Computers have become an inseparable part of our life the
way we interact without a computer is changing day by day.
Often the term virtual world is used with computer gaming
and open world application these applications provide 3d
space to user in the computer where he can explore the
virtual world using input devices like keyboard or gaming
console. The term Augmented Reality (AR) is used to
describe a combination of technologies that enable real-
time mixing of computer-generated content with a live
video display. AR is based on techniques developed in VR
and interacts not only with a virtual world but has a degree
of interdependence with the real world. As stated in
hugues11, “augmenting” reality is meaningless in itself.
However, this term makes sense as soon as we refocus on
the human being and on his perception of the world. Reality
cannot be increased but its perceptions can be. We will,
however, keep the term of Augmented Reality even if we
understand it as an “increased perception of reality”.
Traditionally we use papers, notebooks, and files to store
information.Digitization of information has not
transformed fully we still rely on papers i.e. instruction
manuals datasheets.There are many limitations to using
papers as they can be used to visualize static data but in
todays era where information is updating every day user
has to get updated using devices like mobiles computer
with internet.but the way we visualize the data has been
always concern as viewing data on computer doesent feel
real or relevant as many of the information is connected to
the real world locations but when it comes to viewing that
information with devices we cannot find refrence well as
images can only show 2d space of the place but we cant
explore or get idea how realy the place is.
Augmented Reality addresses such problems by
reintegrating electronic information back into the real
world. The various approaches share a common goal: to
enable people to take advantage of their existing skills in
interacting with objects the everyday world while
benefiting from the power of networked computing [1].
This paper focuses on implementation of the mixed reality
technology by using augmented reality and virtual reality
and its various aspects
2. 2.1. Definition Augmented reality technology has its roots
in the field of computer science interface research [3]. Many
of the basic concepts of AR have been used in movies and
science fiction at least as far back as movies like The
Terminator (1984) and RoboCop (1987). These movies
feature cyborg characters whose views of the physical
world are augmented by a steady stream of annotation and
graphical overlays in their vision systems.
II. AUGMENTED REALITY
Augmented reality technology has its roots in the field of
computer science interface research [3]. Many of the basic
concepts of AR have been used in movies and science fiction
at least as far back as movies like The Terminator (1984)
and Robocop (1987). These movies feature cyborg
characters whose views of the physical world are
augmented by a steady stream of annotation and graphical
overlays in their vision systems.
The term ”augmented reality” was first coined by
researcher Tom Caudell, at Boeing in 1990, who was asked
to improve the expensive diagrams and marking devices
used to guide workers on the factory floor[4]. He proposed
replacing the large plywood boards, which contained
individually designed wiring instructions for each plane,
with a headmounted apparatus that displays a plane’s
specific schematics through high-tech eyeware and project
them onto multipurpose, reusable boards. Many authors
agree to define AR in a way that requires the use of Head-
Mounted Displays (HMDs) [5]. However, in order to avoid
limiting AR to specific technologies, we propose to define
AR as systems that have the following characteristics: 1)
combines real and virtual; 2) interactive in real time; and 3)
registered in 3-D. This definition aims to allow other](https://image.slidesharecdn.com/irjet-v5i1190-180317111722/85/IRJET-Augmented-Reality-based-Platform-to-Share-Virtual-Worlds-1-320.jpg)
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072
© 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 888
technologies, such as mobile technology, besides HMDs
while preserving the essential components of AR [6]. 2-D
virtual overlays on top of live video can be done at
interactive rates, but the overlays are not combined with
the real world in 3-D [7]. However, this definition does
allow monitor based interfaces, monocular systems, see-
through HMDs or mobile devices.
III.AUGMENTED REALITY AND VIRTUAL REALITY.
The term virtual reality is commonly used by the popular
media to describe imaginary worlds that only exist in
computers and our minds. However, let us more precisely
define the term. Virtual is defined to be being in essence or
effect but not in fact. Reality is defined to be something that
constitutes a real or actual thing as distinguished from
something that is merely apparent; something that exists
independently of ideas conceiving it. more recently defined
the full term virtual reality is to be an artificial environment
which is experienced through sensory stimuli (as sights and
sounds) provided by a computer and in which one’s actions
partially determine what happens in the environment. [13]
further defines a virtual reality to be a computer-generated
environment that can be interacted with as if that
environment was real. A good virtual reality system will
allow users to physically walk around objects and touch
those objects as if they were real. Ivan Sutherland, the
creator of one of the world’s first virtual reality systems
stated ”The ultimate display would, of course, be a room
within which the computer can control the existence of
matter. A chair displayed in such a room would be good
enough to sit in. Handcuffs displayed in such a room would
be confining, and a bullet displayed in such a room would
be fatal” sutherland68.
IV.PROPOSED METHODOLOGY
Combining both technologies like ar and ve we need to
combine two leading apis from the google.
1) Google Cardboard
2) Tensorflow Lite
Google Cardboard
Google Cardboard is a virtual reality (VR) platform
developed by Google for use with a head mount for a
smartphone. Named for its fold-out cardboard viewer, the
platform is intended as a low-cost system to encourage
interest and development in VR applications.[1][2] Users
can either build their own viewer from simple, low-cost
components using specifications published by Google, or
purchase a pre-manufactured one. To use the platform,
users run Cardboard-compatible applications on their
phone, place the phone into the back of the viewer, and
view content through the lenses.
Google provides three software development kits for
developing Cardboard applications: one for the Android
operating system using Java, one for the game engine Unity
using C#, and one for the iOS operating system. After
initially supporting only Android, Google announced iOS
support for the Unity plugin in May 2015 at the Google I/O
2015 conference. Third-party apps with Cardboard support
are available on the Google Play store and App Store for iOS.
In addition to native Cardboard apps, there are Google
Chrome VR Experiments implemented using WebGL;
phones, including Apple's, that support WebGL can run
Google's web experiments. A port of the Google Cardboard
demonstration app to iOS was released at Google I/O
2015.[3] In January 2016, Google announced that the
software development kits would support spatial audio, a
virtual reality effect intended to simulate audio coming
from outside of the listener's head located anywhere in 3D
space.
TensorFlow Lite
TensorFlow is an open-source software library for dataflow
programming across a range of tasks. It is a symbolic math
library, and also used for machine learning applications
such as neural networks.[3] It is used for both research and
production at Google, often replacing its closed-source
predecessor,
TensorFlow was developed by the Google Brain team for
internal Google use. It was released under the Apache 2.0
open source.
TensorFlow Lite is an evolution of TensorFlow Mobile. In
most cases, apps developed with TensorFlow Lite will have
a smaller binary size, fewer dependencies, and better
performance. TensorFlow Lite is in developer preview, so
not all use cases are covered yet. We expect you to use
TensorFlow Mobile to cover production cases. TensorFlow
Lite supports only a limited set of operators, so not all
models will work on it by default. TensorFlow for Mobile
has a fuller set of supported functionality. TensorFlow Lite
provides better performance and a small binary size on
mobile platforms as well as the ability to leverage hardware
acceleration if available on their platforms. In addition, it
has many fewer dependencies so it can be built and hosted
on simpler, more constrained device scenarios. TensorFlow
Lite also allows targeting accelerators through the Neural
Networks API.](https://image.slidesharecdn.com/irjet-v5i1190-180317111722/85/IRJET-Augmented-Reality-based-Platform-to-Share-Virtual-Worlds-2-320.jpg)
![International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072
© 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 889
V. ADDING NEURAL NETWORKS
We have a very large number of state-action pairs in mixed
reality as feature extraction is important part as we have to
know where and which place has been shared by the user
and how they differ from each other using multilayer neural
network for such applications is efficient choice.
Tensorflow Lite handles all neural networking tasks
efficiently and without any hustle we can integrate it in our
model
Fig 3 Multilayer Neural Net
Neurons are used for fitting linear forms, e.g., y = a + bi
where i is the input (the frames in our case). Also called
adrenaline rule or Widrow-Hoff rule.
Neurons can also be used by fitting the Q-function in a
piecewise manner, where a linear fit is introduced in every
piece.
Algorithm
Step 1: Initialize the weights of the neural network.
Step 2: Compute the output o using
∑
where
w(j) is the jth weight of neuron and x(j) is the jth input.
Step 3: Update each w(i) for i = 0,1,...,k using:
w(i) ← w(i) + µ[target − output]x(i)
Step 4: Increment iteration while I < iter_max
VI.CONCLUSION
This research presented an approach to implement
augmented reality and virtual reality using neural networks
by tensor flow and google cardboard which is popularly
used for virtual reality technology we solved two issues
regarding mixed reality
1) Using neural network to find features and recognize
objects and sprit placement combined with gps co-
ordinates.
2) Placing view generated with AR on Google
Cardboard platform.
This paper so far shows approach to implementation
sharing of virtual worlds. We tend to create social network
where using this approach we can share related to physical
places with others using same resources
VII. REFERENCES
[1]Jens Naber,Christian Krupitzer,Christian
Becker ”Transferring an Interactive Display Service to the
Virtual Reality”©2016 IEEE
[2]Lynda Joy Gerry University of Copenhagen”Paint with
Me: Stimulating Creativity and Empathy While Painting
with a Painter in Virtual Reality”.
[3]. Martin Duggan,Kieran Flesk,Jim Duggan,Enda
Howley,Enda Barrett
A Reinforcement Learning Approach for Dynamic Selection
of Virtual Machines in Cloud Data Centres
National University Of National University Of Ireland,
Galway,pp. 178-183, 2016.
[4]. X. Dutreilh, S. Kirgizov, O. Melekhova, J. Malenfant, N.
Rivierre, and I. Truck. Using reinforcement learning for
autonomic resource allocation in clouds: Towards a fully
automated workflow. In ICAS 2011, The Seventh
International Conference on Autonomic and Autonomous
Systems, pages 67–74, 2011.](https://image.slidesharecdn.com/irjet-v5i1190-180317111722/85/IRJET-Augmented-Reality-based-Platform-to-Share-Virtual-Worlds-3-320.jpg)
IRJET-Augmented Reality based Platform to Share Virtual Worlds
- 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072 © 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 887 Augmented Reality based Platform to share virtual worlds Prof. Dhananjay Gaikwad1, Akash Chikane2, Shrikrishna Kulkarni3, Aishwarya Nhavkar4 1 Professor, Computer Engineering, NBN Sinhgad School of Engineering, Pune 2,3,4 Computer Engineering, NBN Sinhgad School of Engineering, Pune -------------------------------------------------------------------------------***--------------------------------------------------------------------------------- Abstract: Augmented reality is new emerging technology, where things in the physical world are mixed with digital content to increase user experience and simplicity. As augmented reality technology allows us to create a virtual world on top of the physical world, this can be used to share same physical resources with multiple users. The project scope will be to provide a platform for users to share physical resources along with dynamic information to provide enhanced mixed reality experience. Cloud and the real-time database will be used to create a simultaneous connection between multiple users and share, creating a secure connection and provide data security, also users will be able to set triggers related to physical world i.e. silent mode in office. Keywords: augmented reality, cloud, virtual worlds. I. INTRODUCTION Computers have become an inseparable part of our life the way we interact without a computer is changing day by day. Often the term virtual world is used with computer gaming and open world application these applications provide 3d space to user in the computer where he can explore the virtual world using input devices like keyboard or gaming console. The term Augmented Reality (AR) is used to describe a combination of technologies that enable real- time mixing of computer-generated content with a live video display. AR is based on techniques developed in VR and interacts not only with a virtual world but has a degree of interdependence with the real world. As stated in hugues11, “augmenting” reality is meaningless in itself. However, this term makes sense as soon as we refocus on the human being and on his perception of the world. Reality cannot be increased but its perceptions can be. We will, however, keep the term of Augmented Reality even if we understand it as an “increased perception of reality”. Traditionally we use papers, notebooks, and files to store information.Digitization of information has not transformed fully we still rely on papers i.e. instruction manuals datasheets.There are many limitations to using papers as they can be used to visualize static data but in todays era where information is updating every day user has to get updated using devices like mobiles computer with internet.but the way we visualize the data has been always concern as viewing data on computer doesent feel real or relevant as many of the information is connected to the real world locations but when it comes to viewing that information with devices we cannot find refrence well as images can only show 2d space of the place but we cant explore or get idea how realy the place is. Augmented Reality addresses such problems by reintegrating electronic information back into the real world. The various approaches share a common goal: to enable people to take advantage of their existing skills in interacting with objects the everyday world while benefiting from the power of networked computing [1]. This paper focuses on implementation of the mixed reality technology by using augmented reality and virtual reality and its various aspects 2. 2.1. Definition Augmented reality technology has its roots in the field of computer science interface research [3]. Many of the basic concepts of AR have been used in movies and science fiction at least as far back as movies like The Terminator (1984) and RoboCop (1987). These movies feature cyborg characters whose views of the physical world are augmented by a steady stream of annotation and graphical overlays in their vision systems. II. AUGMENTED REALITY Augmented reality technology has its roots in the field of computer science interface research [3]. Many of the basic concepts of AR have been used in movies and science fiction at least as far back as movies like The Terminator (1984) and Robocop (1987). These movies feature cyborg characters whose views of the physical world are augmented by a steady stream of annotation and graphical overlays in their vision systems. The term ”augmented reality” was first coined by researcher Tom Caudell, at Boeing in 1990, who was asked to improve the expensive diagrams and marking devices used to guide workers on the factory floor[4]. He proposed replacing the large plywood boards, which contained individually designed wiring instructions for each plane, with a headmounted apparatus that displays a plane’s specific schematics through high-tech eyeware and project them onto multipurpose, reusable boards. Many authors agree to define AR in a way that requires the use of Head- Mounted Displays (HMDs) [5]. However, in order to avoid limiting AR to specific technologies, we propose to define AR as systems that have the following characteristics: 1) combines real and virtual; 2) interactive in real time; and 3) registered in 3-D. This definition aims to allow other
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072 © 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 888 technologies, such as mobile technology, besides HMDs while preserving the essential components of AR [6]. 2-D virtual overlays on top of live video can be done at interactive rates, but the overlays are not combined with the real world in 3-D [7]. However, this definition does allow monitor based interfaces, monocular systems, see- through HMDs or mobile devices. III.AUGMENTED REALITY AND VIRTUAL REALITY. The term virtual reality is commonly used by the popular media to describe imaginary worlds that only exist in computers and our minds. However, let us more precisely define the term. Virtual is defined to be being in essence or effect but not in fact. Reality is defined to be something that constitutes a real or actual thing as distinguished from something that is merely apparent; something that exists independently of ideas conceiving it. more recently defined the full term virtual reality is to be an artificial environment which is experienced through sensory stimuli (as sights and sounds) provided by a computer and in which one’s actions partially determine what happens in the environment. [13] further defines a virtual reality to be a computer-generated environment that can be interacted with as if that environment was real. A good virtual reality system will allow users to physically walk around objects and touch those objects as if they were real. Ivan Sutherland, the creator of one of the world’s first virtual reality systems stated ”The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal” sutherland68. IV.PROPOSED METHODOLOGY Combining both technologies like ar and ve we need to combine two leading apis from the google. 1) Google Cardboard 2) Tensorflow Lite Google Cardboard Google Cardboard is a virtual reality (VR) platform developed by Google for use with a head mount for a smartphone. Named for its fold-out cardboard viewer, the platform is intended as a low-cost system to encourage interest and development in VR applications.[1][2] Users can either build their own viewer from simple, low-cost components using specifications published by Google, or purchase a pre-manufactured one. To use the platform, users run Cardboard-compatible applications on their phone, place the phone into the back of the viewer, and view content through the lenses. Google provides three software development kits for developing Cardboard applications: one for the Android operating system using Java, one for the game engine Unity using C#, and one for the iOS operating system. After initially supporting only Android, Google announced iOS support for the Unity plugin in May 2015 at the Google I/O 2015 conference. Third-party apps with Cardboard support are available on the Google Play store and App Store for iOS. In addition to native Cardboard apps, there are Google Chrome VR Experiments implemented using WebGL; phones, including Apple's, that support WebGL can run Google's web experiments. A port of the Google Cardboard demonstration app to iOS was released at Google I/O 2015.[3] In January 2016, Google announced that the software development kits would support spatial audio, a virtual reality effect intended to simulate audio coming from outside of the listener's head located anywhere in 3D space. TensorFlow Lite TensorFlow is an open-source software library for dataflow programming across a range of tasks. It is a symbolic math library, and also used for machine learning applications such as neural networks.[3] It is used for both research and production at Google, often replacing its closed-source predecessor, TensorFlow was developed by the Google Brain team for internal Google use. It was released under the Apache 2.0 open source. TensorFlow Lite is an evolution of TensorFlow Mobile. In most cases, apps developed with TensorFlow Lite will have a smaller binary size, fewer dependencies, and better performance. TensorFlow Lite is in developer preview, so not all use cases are covered yet. We expect you to use TensorFlow Mobile to cover production cases. TensorFlow Lite supports only a limited set of operators, so not all models will work on it by default. TensorFlow for Mobile has a fuller set of supported functionality. TensorFlow Lite provides better performance and a small binary size on mobile platforms as well as the ability to leverage hardware acceleration if available on their platforms. In addition, it has many fewer dependencies so it can be built and hosted on simpler, more constrained device scenarios. TensorFlow Lite also allows targeting accelerators through the Neural Networks API.
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072 © 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 889 V. ADDING NEURAL NETWORKS We have a very large number of state-action pairs in mixed reality as feature extraction is important part as we have to know where and which place has been shared by the user and how they differ from each other using multilayer neural network for such applications is efficient choice. Tensorflow Lite handles all neural networking tasks efficiently and without any hustle we can integrate it in our model Fig 3 Multilayer Neural Net Neurons are used for fitting linear forms, e.g., y = a + bi where i is the input (the frames in our case). Also called adrenaline rule or Widrow-Hoff rule. Neurons can also be used by fitting the Q-function in a piecewise manner, where a linear fit is introduced in every piece. Algorithm Step 1: Initialize the weights of the neural network. Step 2: Compute the output o using ∑ where w(j) is the jth weight of neuron and x(j) is the jth input. Step 3: Update each w(i) for i = 0,1,...,k using: w(i) ← w(i) + µ[target − output]x(i) Step 4: Increment iteration while I < iter_max VI.CONCLUSION This research presented an approach to implement augmented reality and virtual reality using neural networks by tensor flow and google cardboard which is popularly used for virtual reality technology we solved two issues regarding mixed reality 1) Using neural network to find features and recognize objects and sprit placement combined with gps co- ordinates. 2) Placing view generated with AR on Google Cardboard platform. This paper so far shows approach to implementation sharing of virtual worlds. We tend to create social network where using this approach we can share related to physical places with others using same resources VII. REFERENCES [1]Jens Naber,Christian Krupitzer,Christian Becker ”Transferring an Interactive Display Service to the Virtual Reality”©2016 IEEE [2]Lynda Joy Gerry University of Copenhagen”Paint with Me: Stimulating Creativity and Empathy While Painting with a Painter in Virtual Reality”. [3]. Martin Duggan,Kieran Flesk,Jim Duggan,Enda Howley,Enda Barrett A Reinforcement Learning Approach for Dynamic Selection of Virtual Machines in Cloud Data Centres National University Of National University Of Ireland, Galway,pp. 178-183, 2016. [4]. X. Dutreilh, S. Kirgizov, O. Melekhova, J. Malenfant, N. Rivierre, and I. Truck. Using reinforcement learning for autonomic resource allocation in clouds: Towards a fully automated workflow. In ICAS 2011, The Seventh International Conference on Autonomic and Autonomous Systems, pages 67–74, 2011.