Collecting big data in cinemas to improve recommendation systems - a model with three types of motion sensors
- 1. Collecting big data in cinemas to improve recommendation systems - a model with three types of motion sensors Kristian Dokic, Domagoj Sulc, Dubravka Mandusic
- 2. Introduction • recommendation system helps consumers to identify attractive content and most video-on-demand services are supported by some RS • they can be divided into two types: personalised and non- personalised systems • Personalised systems use consumer profile data to suggest video content to consumers, • non-personalised systems propose video content based on other data types. • What about cinemas ?
- 3. RS in cinemas ? • cinema operators have less opportunity to use recommendation systems • non-selective methods is to show advertisements before the screening itself • use a loyalty system that is typically available online or by using a mobile application, where cinema operators can have much higher precision from RS • when cinema operators use loyalty systems, they also know the place in the cinema where the user sat while watching the movie • this allows them to measure the user's body movements while watching the film
- 4. Body movements when watching... ? • The association of emotions and unconscious body and face movements • unconscious body movements of cinema viewers could be recorded while watching a movie • analyze and draw certain conclusions about cinema viewer preferences • collect a certain amount of data and then try to find correlations • Model for data collecting is proposed
- 5. Recommendation systems based on conscious choices • The authors try to achieve better results of recommendation systems, but the data source they use is based on the conscious choices of system users • In this paper, a model is set up that assumes that certain knowledge is also stored in the unconscious movements of the user's body while watching a movie. • How to get this data and how to process it is the essence of this paper.
- 6. Involuntary body movement research • Involuntary body movements in response to a stimulus have been studied in psychology for quite some time. • Montepare et al. studied the possibility of recognising emotions based on body movements. • Deal et al. found that there is relatively little research dealing with the detection of emotions based on body movements. • Sogon and Izard explored gender differences in emotion recognition based on body movements.
- 7. Body movement sensing and measuring • Face is the most interesting to researchers • Camera is a very commonly used device for measuring body movement • there are several other sensors for the same purpose that have been described in the literature (accelerometer, gyroscope, global positioning system, magnetometer, electromyography, resistive flex and pressure sensing textile and environmental context sensors )
- 8. Acceptable sensors for body movement measuring in a cinema • Camera ? NO. • Body attached ? NO. • WHICH SENSORS ? a) microwave radar sensor based on the Doppler effect b) passive infrared receiver sensor c) ultrasonic distance sensor
- 9. Microwave radar sensors • emit microwave signal and compare the echo with the transmitted signal and calculate the object's distance based on the Doppler effect • device serves as a motion detector and the output is a logical zero or one, depending on whether the movement is registered
- 10. Passive infrared receiver sensor • electronic sensor that measures infrared light emitted from objects around so it detects the movement of people
- 11. Ultrasonic distance sensor • ultrasonic distance sensor is based on the principle of echolocation • The sensor emits a short ultrasound signal, and registers the echo of the emited signal • distance of the object can be calculated based on the elapsed time.
- 12. Device development • Sensors are connected to the Arduino Uno development board, to which the button is also attached • A sampling time of one second was defined, and testing could begin • board collects data from all three sensors and forward it over a serial connection to a computer
- 13. Location of the sensors and device • the best place for the sensors is on the left or right side of the armchair in the cinema hall • sensors are sufficiently protected
- 14. Data collecting • development board has the function of a signal concentrator with three different sensors • collected data were not always harmonized • it was decided to generate a neural network that could, based on the collected data, determine whether the cinema spectator is moving on an armchair or not • a button is connected to the Arduino Uno board, the values of which are also sent to the computer • neural networks belong to supervised learning, the button is used to collect data on a person's movement by pressing a button each time he or she moves. In this way, data were collected for neural network training.
- 15. Data processing and results • One thousand samples were collected in just over 16 minutes and computer received four numbers every second • These 1000 samples were used to train the neural network • Edge impulse service was used to train the network
- 16. Trained neural network • The neural network consists of input, output and two fully connected layers with 30 neurons • model is only 3 kB in size • very high accuracy was achieved in the viewer idle mode (98%) • accuracy of predicting the viewer's movement is 72%, while the neural network will register 28% of situations when the viewer moves in an armchair as idle
- 17. Discussion and conclusion • application of the proposed system in cinemas requires first of all the installation of the system in cinema armchairs • cinema spectators must have the choice of whether or not they want data about their unconscious movements to be stored, and it is up to cinemas to argue to cinema visitors the benefits of collecting data using the system described • proposed model represents only the first step in the use of big data in cinemas • we have reached a technological level in recent years that allows cheap processing of large amounts of data and finding knowledge in them