Ενσωμάτωση συναισθημάτων σε παιχνίδια σοβαρού σκοπού Εβδομάδα 6 Εισαγωγή στη Συναισθηματική νευροεπιστήμη

Summary The need for ubiquitous contemporary controllers in smart homes Beyond the interfaces, applications and controllers. The Controller Application Communication (CAC) framework Facilitating developers in adopting new techniques independently of hardware device, OS and programming language. Introducing proposed architecture in lab environment Moving on in smart home environment

…σύγχρονα χειριστήρια… Χειριστήρια που έκαναν την εμφάνισή τους πρόσφατα: inertial sensors (Wii remote controller) pressure sensors (Wii balance board) camera systems (Kinect sensor) biosensors (Neurosky’s Mindwave - one channel EEG) multisensory devices (smart phones, smart watches)

…σύγχρονες συσκευές αλληλεπίδρασης… Computers smartTV tablet Smart phones

Η ανάγκη για διάχυτη πληροφορία χειριστηρίων σε έξυπνα σπίτια Παρέμβαση και πρόληψη (e.g. Wii suite of controllers has been used for enabling elderly people to increase their physical activity levels by engaging in exergames) Διακριτική παρακολούθηση υγείας (e.g. Kinect has been utilized assess probability of elderly fall) Εφαρμογές διαισθητικής αλληλεπίδρασης (χειρονομίες)

Πλαίσιο επικοινωνίας συσκευών και εφαρμογών (CAC) framework Ενδιάμεση εφαρμογή για την διασύνδεση χειριστηρίων και εφαρμογών. Συνεχής ροή πληροφορίας από τα χειριστήρια στις εφαρμογές (streaming) (e.g. skeleton, RGB image, center of pressure) to the server. Ομαδοποίηση πελατών λογισμικού.

Πλαίσιο επικοινωνίας συσκευών και εφαρμογών (CAC) framework Πρώτόκολλο επικοινωνίας: Web Sockets (full-duplex communication channels,  designed to be implemented in web browsers and web servers Μορφή μηνυμάτων: JSON Συμβατά χειριστήρια: Microsoft Kinect, Wii remote control, Wii balance board, Mindwave, Emotiv, Android Sensors

Αρχιτεκτονική πλαισίου επικοινωνίας συσκευών και εφαρμογών Konstantinidis, E.I., Antoniou, P.E., Bamparopoulos, G., Bamidis, P.D.: A lightweight framework for transparent cross platform communication of controller data in ambient assisted living environments. (2013), (under review at Information Sciences)

Technology and Active Aging - Well being Technology intervenes non-pharmacologically Serious games of physical and cognitive training are used against cognitive decline Elderly people improve their health and well being crucial for prolonged independent living How can we tailor “silvergames” environment and conditions to suit “grown up kids” capabilities and needs? Which are the biological and behavioral signals we can combine with gaming? How can emotion, a function which remains stable or even improves across adult lifespan, be employed? How can the fusion of the collected information modify the game environment so as to motivate the user to interact with the game on a daily basis? Me maura osa exoyn apantithei apo proigoumenes meletes Me kokkina osa skopeuoyme na apadisoume emeis me ti mellontiki mas ereyna.edo apla ta episimainoume!!!!

Prevention of Cognitive Decline Cognitive decline is responsible for much of the Quality of Life deterioration in seniors Non-pharmacological arsenal: Physical exercise Cognitive exercise Proper training  neural plasticity changes  slow down or invert cognitive decline (Smith et al., 2009). auto einai to background gia ta 3 prota points tou slide 2

Serious Games Motion capturing is an integral part of the game design employment of sensors within a gaming environment projected on mobiles devices (phones, tablets) and SmartTVs body movement gesture and facial recognition Bla-bla gia slides 4-6 Serious games for elderly healthcare provide a promising and novel way to promote the well-being of senior citizens. The gaming environment, originally designed for a younger target population, benefits from the increasing power of personal computers, mobiles devices (phones, tablets) and SmartTVs, as well as the recent emergence of motion capture technology developed for videogame consoles via worn physical sensors and controller free sensors. Measuring self-efficacy by elderly individuals on such gaming environments has characterized them in terms of their effectiveness to motivate an audience reluctant to undertake more conventional forms of activities. However, the relationship between the features making the game challenging and the senior user’s behaviour in becoming motivated so as to interact with the system independently and effectively is still elusive. For instance, failure in understanding the game’s instructions or adapting to the speed and complexity may be thought of as barriers that affect the seniors’ satisfactory interaction within the gaming environment.

Serious Games for Elderly Healthcare Current State Of The Art: A cohort of studies on serious games for seniors has only been recently introduced. Objective: Identify and meet senior users’ needs within the gaming environment Seniors have to be motivated so as to retain their engagement to the game on a frequent basis. Approach: This is a game design issue that may be answered via Affective Computing develop computer interfaces so as to explore the underlying relation between the user’s emotions and human health, both mental and physical. Bla-bla gia slides 4-6 Serious games for elderly healthcare provide a promising and novel way to promote the well-being of senior citizens. The gaming environment, originally designed for a younger target population, benefits from the increasing power of personal computers, mobiles devices (phones, tablets) and SmartTVs, as well as the recent emergence of motion capture technology developed for videogame consoles via worn physical sensors and controller free sensors. Measuring self-efficacy by elderly individuals on such gaming environments has characterized them in terms of their effectiveness to motivate an audience reluctant to undertake more conventional forms of activities. However, the relationship between the features making the game challenging and the senior user’s behaviour in becoming motivated so as to interact with the system independently and effectively is still elusive. For instance, failure in understanding the game’s instructions or adapting to the speed and complexity may be thought of as barriers that affect the seniors’ satisfactory interaction within the gaming environment.

Literature Guidelines for “Silvergames” address usability issues related to physical and mental (cognitive and/or emotional) changes in elderly users. the game control should be easy in adaptation provide a pleasant experience by being responsive (findings based on video observations) the game control should be accurate provide a realistic experience that will fit the user’s expectations (findings based on interviews) the game control should employ appropriate gestures, visual and auditory feedback the game control should employ clear user interface, ability to adjust the difficulty Beyond guidelines, we aim in using emotions in the midst of the gaming environment so as to allow for the embodiment of real-time mental (cognitive and emotional), and physical data. The main goal is to focus on assessing the features that encourage the elderly individuals to interact with the gaming environment on a daily basis. We propose to further enhance the gaming environment through the use of personal biosensors (i.e. wireless EEG) and cameras (i.e. fisheye camera) so as to collect the user’s mental and physical changes over time and fuse them in a decision support system. This information will eventually provide feedback on the gaming experience so as to modify it according to the user’s affective state. Beyond Guidelines ?

Affection across Adult Lifespan Positivity effect=put more priority on positive information than on negative information. Cognitive decline, potential health problems, physical limitations. Aging Paradox Auto einai to backround tou point 6 tou slide 2. As people grow older, they lose some of their ability to process information and show decline to a wide range of processes, except for the ability to regulate emotion. Older adults have greater motivation than young adults to regulate emotional states, and may alter their environmental circumstances and expectations to support positive experiences.

Aging paradox & functions Presence or absence of a positivity effect may depend on valence, arousal as well as the type of measurements, availability of cognitive resources Emotional Regulation (which requires self-directed attention and memory) More efficient in older adults, e.g. in order to not disrupt performance on a memory-intensive task Decreased negative affect / More stable positive affect Motivation Older adults selectively focus on positive material when they’re faced with choices Older adults are more patient and perform emotional processing strategies with clear benefits for emotional well-being Attention (Mather & Carstensen, 2003; Knight et al., 2007) Older adults spend a larger proportion of time viewing positive items and a smaller proportion viewing negative items than do younger adults Have the ability to remember positive images over negative ones. / Younger adults, conversely, were better at remembering the negative images. Auto to slide tha to vgaleis apo tin parousiasi. Diavase to omos gia na katalaveis pos sxetizetai to aging paradox Older adults increase positive affect by reducing the discrepancies between current states and goal states, either by lowering standards, or by shifting goals away from unmanageable areas and toward manageable ones (Lacey, 2006).

Aging paradox & memory interact with arousal and valence Memory (Kensinger, 2008) a positivity effect emerged for non-arousing words Older adults remembered positive non-arousing words better than negative non-arousing words and showed no mnemonic benefit for negative non-arousing words as compared with neutral words / Young adults remembered negative non-arousing words better than positive non-arousing items Aging preserves responses to arousing information while altering the processing of non-arousing information Memory (Mather & Carstensen, 2003; Mather & Knight., 2005; Charles et al., 2003) a positivity effect emerged for age-valence interaction Older adults recall the emotional content of pictures Have the ability to remember positive images over negative ones. / Younger adults, conversely, were better at remembering the negative images Auto to slide tha to vgaleis apo tin parousiasi. Diavase to omos gia na katalaveis pos sxetizetai to aging paradox

Different modalities Brain Activity via EEG Facial characteristics via cameras Voice via microphones Me kokkino auta pou skopeuoyme na metrisoume Heart Rate via ECG Posture, gestures via fisheye camera

Proposed Methodology Edo anafere sta grigora osa grafeis stin diki sou parousiasi sta slides 3, 5, 8, 9 kai einai i vasi tou paper…

DSS schematic overview 4 Re-adjustment of gaming environment Long term analysis 5 Gaming interaction Central processing 2 Sensor – Level processing Short term analysis – Context analysis Sensor analyzers 1 3 1. Sensors analysers collect vital signs and behavioral cues 2. Process them and transform them to low-level information cues e.g high arousal 3. Fuse multimodal information cues to provide contextual information, e.g. happy emotion 4. Create a senior’s profile based on historical values, e.g. emotion trends, behavior trends, senior attitude towards gaming 5. Contextual information and long term trends provide useful hints on senior’s attitude towards the gaming environment. 5. Evaluation and adaptation of gaming environment based on previous findings. Sensor Data Fusion Billis et al., “A Decision-Support Framework for promoting Independent Living and Ageing Well”, J Biomed. Health Inf. (under minor revision)