Sketching and Bio-sensing for "Wellness Entertainment"August 2020 - Kinki University, Japan
Yoichi Nagashima (SUAC/ASL)
Description of tutorialThis is a lecture intended for the designer/artist/therapist of the interactive system for "wellness entertainment". In recent years, the fields of serious games and medical/welfare/rehabilitation applications are gaining attention in the field of entertainment computing. The presenter has been promoting R & D activities in this area for 30 years, and recently targeting "Wellness" or "Well-being" entertainment using bio-sensing technology. This lecture contains four parts of practical techniques/ideas to realize effective interactive system for "wellness entertainment" - environment & interface, concept of biofeedback, special techniques using commercial bio-sensors and cutting-edge topics.
The first part is short lecture of "Max8"(cycling'74) - famous and powerful platform/environments to develop realtime multimedia system with sensors. This "Sketching"(physical computing) lecture with many effective examples will help future development. The popular interface "Gainer" had retired few years ago, so this lecture also contains three types of "post-Gainer" new sensing/interface techniques using Arduino.
The second part is a lecture about "biofeedback and human wellness" related to "Wellness Entertainment" System Design. The idea of biofeedback is well-known in the rehabilitation and welfare field, and recent bio-sensing technology and multimedia technology enable to realize effective biofeedback system easily. Generally, human mental power for homeostasis works in emotion/feeling channel - which leads us "Wellness" or "Well-being" status. Especially, reaction information from unconscious muscles is gaining attention as "interoceptive sense" leading to emotion/feeling - this is the reason the organizer pays particular attention to the EMG sensing.
The third part is the special techniques using bio-sensing systems which are commercially available. (1) The "Myo" armband - three original techniques will be opened - how to use 2 or 3 Myo at the same time, how to avoid Myo's sleeping, and how to get directly data such as 8ch-EMG and 9-D data via Bluetooth. (2) The "MUSE" headband - how to get directly data such as 4ch-brain waves and 3D-direction via Bluetooth. (3) The combination application using "double Myo" and "MUSE" will be shown with the detail of Max8 programming - as a realtime interactive multimedia performance work in media art.
The fourth part is two cutting-edge bio-sensing topics. (1) The presenter and collaborator developed a novel 4ch EMG sensor system called "VPP-SUAC". The whole development data is open-source in the Web. While Myo is an "exclusive for arm" EMG sensor, our sensor focuses on sensing legs and lower body EMG which are important in welfare and rehabilitation area. (2) The other is a new special tactile/rubbing sensor called "PAW sensor", which is a small PCB with a cylinder of urethane foam on it. The output information of this sensor is four channel voltages which is time-shared conversion, which means the nuances of rubbing/touching the urethane foam with fingers. In this lecture, the attendees will experience the special eight sensors and its experimental application which generates 3-D realtime sounds/graphics for healing entertainment or prevention of dementia.
Background/relevance of tutorial topic
- The sensing technology is the key part in entertainment computing, and platform/environment/interfaces of interactive system is the base of the researches/development.
- This workshop/lecture is "hands-on" which means that the attendees will experience the developing process and many demonstrations.
- With attending this workshop/lecture, many specialists will appear in this field.
Outline of tutorial workshopPart 1 Short/dense Introducing Lecture of "Max8" and "post-Gainer" Techniques (1) Max/MSP/jitter Introduction Demos (2) "post-Gainer" Techniques - Firmata+maxuino, Arduino2Max, USB-MIDI Part 2 Lecture/discussion about "Biofeedback and Human Wellness" (1) Biofeedback and rehabilitation/habilitation (2) interoceptive sense Part 3 Special Techniques using Bio-sensing Systems which are commercially available (1) Myo and DoubleMyo (2) Muse and MuseOSC Part 4 two cutting-edge bio-sensing topics (1) PAW sensor (tactile sensor) - single test, "MRTI2015", "PAW-eight", "PAW-double" (2) Introduction for original EMG/EEG sensor "VPP-SUAC"
Duration3 hours ?
LanguageEnglish (if requested, Japanese a little)
Target participants, including level of experience
- “intermediate" - "advanced"
- The designer/researcher/specialist who experiences an interactive system that uses Max8, sensors, and Arduino is preferable.
Expected tutorial workshop outcomes
- Sample programs, special libraries for bio-sensing
- “take-home” skills development
- Understanding/knowledge for biofeedback, Wellness Entertainment, application for medical/welfare/rehabilitation fields.
Sample materialsExamples of interactive works are online - for example: http://nagasm.org/1106/PhysiCom/
CV information of the tutorial presenter
Yoichi Nagashima, composer/researcher/PE, was born in 1958 in Japan. He learned and played some instruments: violin, recorder, guitar, keyboards, electric bass, drums, and vocal/choral music for 45 years. He was the conductor of Kyoto University Choir and composed over 100 choral music, and studied nuclear physics there. As the engineer of Kawai Musical Instruments, he developed some sound generator LSIs, and designed some electronic musical instruments, and produced musical softwares. From 1991, He has been the director of "Art & Science Laboratory" in Japan Hamamatsu, produces many interactive tools of realtime music performance with sensor/MIDI, cooperates some researchers and composers, and composes experimentally pieces. He is also the key-member of Japanese computer music community. From 2000, he has been also the associate professor of SUAC(Shizuoka University of Art and Culture), Faculty of Design, Department of Art and Science, and teaches multimedia, computer music and media art. As a composer of computer music, he collaborates many musicians in his composition: Piano, Organ, Percussion, Vocal, Flute, Sho, Koto, Shakuhachi, Dance, Percussion, etc. He organized and was the General Chair of NIME04. He became a professor on April 2007. He organized a tutorial “Parallel Processing Platform for Interactive Systems Design” at ICEC2009 (Paris).
Lecture/Workshop of Yoichi Nagashima (in English)
- Lecture "Technology for Computer Music", Seoul, Korea, 1997
- Workshop "Sensors for Interactive Music Performance", ICMC2000, Berlin, Germany, 2000
- Workshop "Sensors for Interactive Music Performance", Studio CCMIX, Paris, France, 2001
- Lecture "Composition of [Visional Legend]", International Workshop on "Human Supervision and Control in Engineering and Music", Kassel, Germany, 2001
- Lecture "Interactive Media Art with Biological Interfaces", STEIM Open Lecture Concert Series, Amsterdam, Holland, 2004
- Lecture "Interfaces for Interactive Media Art", IRCAM Open Lecture, Paris, France, 2004
- Lecture "Interactive Art with Bio-Interfaces", La Kitchen Open Lecture, Paris, France, 2004
- Lecture "Interactive Multi-Media Performance with New Interfaces", International Workshop on Interactive Media Art, Kai-Nan Univ., Taiwan, 2007
- Lecture "Interactive Multi-Media Art with New Technology", Tsukuba Univ., Japan, 2007
- Lecture "SUAC Installation - Case Studies as Physical Computing -", Sketching in Hardware 2008, Rhode Island School of Design, USA, 2008
- Lecture "Some interactive works ('08-'09) - case study of my sketching”, Sketching in Hardware 2009, University College of London, UK, 2009
- Tutorial "Parallel Processing Platform for Interactive Systems Design”, International Conference on Entertainment Computing, Conservatoire National des Arts et Metiers, Paris, 2009
- Lecture "Technology for Computer Music / Interactive Multi-Media Performance with New Interfaces", International Festival/Competition SYNC. 2010 Tutorial(1), The Ural State Conservatory, Yekaterinburg, Russia, 2010
- Lecture "SUAC Installation - Case Studies as "Physical Computing", International Festival/Competition SYNC.2010 Tutorial(2), The Ural State Conservatory, Yekaterinburg, Russia, 2010
- Lecture "Interactive Art with Bio-Interfaces", International Festival/Competition SYNC.2010 Tutorial(3), The Ural State Conservatory, Yekaterinburg, Russia, 2010
- Lecture "Sketching as entertainment of design”, Sketching in Hardware 2012, University of Oregon, USA, 2012
- Lecture "New Platform for Design Entertainment”, Sketching in Hardware 2013, PARC, California, USA, 2013
- Lecture "Against the boom of image recognition sensors”, Sketching in Hardware 2015, Biosphare 2, Arizona, USA, 2015
- Lecture "Biofeedback Instruments and Improvisation”, 4th tempora International Meeting, Bordeaux, France, 2016
- Lecture "Interactive Media Arts - New Ideas and New Technologies", General Lecture for Ekaterinburg's Contemporary Art Academy, Gorky library, Yekaterinburg, Russia, 2016
- Lecture "Interactive Media Arts - New Ideas and New Technologies", General Lecture - open public, Ural branch of the National Center for Contemporary Arts, Yekaterinburg, Russia, 2016
- Workshop "Introduction of Programming - Creating Art Objects", Practical Workshop for Art Academy's Designer Students, Center of Culture “Ordzhonikidzevsky”, Yekaterinburg, Russia, 2016
- Workshop "Interactive System Design - Creating Media Arts", Practical Workshop for Art Academy's Programmers Students, Center of Culture “Ordzhonikidzevsky”, Yekaterinburg, Russia, 2016
- Lecture "Human-Computer Interaction and Media Arts (part1/2)”, Lecture Workshop at MARS Gallery, MARS Gallery, Moscow, Russia, 2016
- Lecture "Bio-Sensing and Bio-Feedback Interfaces”, Conference “Musical Interfaces and Robotics”, Lecture Workshop at Tokyo University of Arts, Japan, 2017
- Workshop/Tutorial "Bio-Sensing Platforms for [Wellness Entertainment] System Design", International Conference on Entertainment Computing, Poznan University of Technology, Poland, 2018
YouTube Demo movies
- OpenBCI Muse DoubleMyo
- Rehearsal of "Bordeaux Power"
- VPP-SUAC (1)
- VPP-SUAC (2)
- single PAWsensor
- single PAWsensor
- MRTI2015 (10 PAWsensors)
- PAW-eight (8 PAWsensors)
- PAW-double (2 PAWsensors)
Presentation and papers
VPP-SUAC (sorry, currently in Japanese only - please use "Google Translate")
PAW-double (sorry, currently in Japanese only - please use "Google Translate")