ICEC2018 Tutorial Workshop

Bio-sensing Platforms for "Wellness Entertainment" System Design

Monday, 17th September 2018 - Poznan, Poland

Yoichi Nagashima (SUAC/ASL)

Call for Participants

Description of workshop

This is a "hands-on" lecture/workshop intended for the designer/artist/therapist of the interactive system for "wellness entertainment". In recent years, the fields of serious games and medical/welfare/rehabilitation applications are gaining attention in the field of entertainment computing. The organizer of this workshop has been promoting R & D activities in this area for over 20 years, and recently targeting "Wellness" or "Well-being" entertainment using bio-sensing technology. This lecture/workshop contains four parts of practical techniques/ideas to realize effective interactive system for "wellness entertainment" - environment & interface, concept of biofeedback, special techniques using commercial bio-sensors and cutting-edge topics.

The first part is short/dense introducing lecture of "Max7"(cycling'74) - famous and powerful platform/environments to develop realtime multimedia system with sensors. This "physical computing" lecture with many effective examples will help future development. The popular interface "Gainer" had retired two years ago, so this lecture also contains three types of "post-Gainer" new sensing/interface techniques using Arduino.

The second part is a lecture/discussion about "biofeedback and human wellness" related to "Wellness Entertainment" System Design. The idea of biofeedback is well-known in the rehabilitation and welfare field, and recent bio-sensing technology and multimedia technology enable to realize effective biofeedback system easily. Generally, human mental power for homeostasis works in emotion/feeling channel - which leads us "Wellness" or "Well-being" status. Especially, reaction information from unconscious muscles is gaining attention as "interoceptive sense" leading to emotion/feeling - this is the reason the organizer pays particular attention to the EMG sensing.

The third part is the special techniques using bio-sensing systems which are commercially available. (1) The "Myo" armband is very smart/powerful gesture sensor, however it is a waste in default usage that its high performance is not utilized. In this lecture, three original techniques will be opened - how to use 2 or 3 Myo at the same time, how to avoid Myo's sleeping, and how to get directly data such as 8ch-EMG and 9-D (3D accel, 3D direction, 3D gyro) via Bluetooth in Max7 environment. (2) The "MUSE" headband is very smart brain sensor, however its default application is limited only meditation training. In this lecture, the original technique will be opened - how to get directly data such as 4ch-brain waves and 3D-direction via Bluetooth in Max7 environment. (3) The combination application using "double Myo" and "MUSE" will be shown with the detail of Max7 programming - as a realtime interactive multimedia performance work in media art. As related information, I will also report about Max7 acquiring sensor data of brain wave sensor "OpenBCI" directly via Bluetooth.

The fourth part is two cutting-edge bio-sensing topics. (1) The organizer and the collaborator has developed a novel 4ch EMG sensor system called "VPP-SUAC" in 2017. The whole development data is open-source in the Web (sorry, currently in Japanese only), and we are welcome to develop the second source. While Myo is an "exclusive for arm" EMG sensor, our sensor focuses on sensing legs and lower body EMG which are important in welfare and rehabilitation area, and we are promoting collaboration with experts - medical/welfare/nursing/care. In this lecture, the participants will experience this system individually with special platform "NucleoF401RE"(ARM), XBee WiFi and special Max7 program with effective EMG-analyzing algorithm. (2) The other is a new special tactile/rubbing sensor called "PAW sensor" from Japan. The "PAW sensor" is a small PCB with a large cylinder of urethane foam on it. The output information of this sensor is four channel voltages which is time-shared conversion, which means the nuances of rubbing/touching the urethane foam with ones fingers. In this lecture, the attendees will experience the special sensor "PAW-eight" with eight sensors on it and its experimental application which generates 3-D realtime sounds/graphics for healing entertainment or prevention of dementia.

Background/relevance of tutorial workshop topic

Outline of tutorial workshop

Part 1 Short/dense Introducing Lecture of "Max7" and "post-Gainer" Techniques
	(1) Max/MSP/jitter Introduction Demos
	(2) "post-Gainer" Techniques - Firmata+maxuino, Arduino2Max, USB-MIDI

Part 2 Lecture/discussion about "Biofeedback and Human Wellness"
	(1) Biofeedback and rehabilitation/habilitation
	(2) interoceptive sense

Part 3 Special Techniques using Bio-sensing Systems which are commercially available
	(1) Myo and DoubleMyo
	(2) Muse and MuseOSC
	(3) OpenBCI (topic only)

Part 4 two cutting-edge bio-sensing topics
	(1) PAW sensor (tactile sensor) - single test, "MRTI2015", "PAW-eight", "PAW-double"
	(2) Introduction for original EMG/EEG sensor "VPP-SUAC"

Duration

full-day (2+5 hours)

Expected number of participants

5 - 10 people(group). The 1 group can contain 2-3 people at maximum (same desk).

Language

English

Target participants, including level of experience

Expected tutorial workshop outcomes

Registration

This tutorial workshop is free for ICEC2018 "full pass" attendees. This "Workshop only" registration is "daily pass".
For details, see ICEC2018 home page: http://www.ifip-icec.org/registration/
If the registration request will be over the limit, the rule is "first come, first served".

Sample materials

The participants may bring her/his MacBook and Max7 to develop own programs. The hardware (Arduino, sensors, Myo, Muse, VPP-SUAC, PAW sensor, etc) will be brought and lent by the organizer. Of course, if the participant wants to use Arduino or Myo or Muse for future project, it will be better to get it before the workshop and bring it. Examples of interactive works are online - for example: http://nagasm.org/1106/PhysiCom/

CV information of the tutorial workshop organizer

Yoichi Nagashima, composer/researcher/PE, was born in 1958 in Japan. He learned and played some instruments: violin, recorder, guitar, keyboards, electric bass, drums, and vocal/choral music for 43 years. He was the conductor of Kyoto University Choir and composed over 100 choral music, and studied nuclear physics there. As the engineer of Kawai Musical Instruments, he developed some sound generator LSIs, and designed some electronic musical instruments, and produced musical softwares. From 1991, He has been the director of "Art & Science Laboratory" in Japan Hamamatsu, produces many interactive tools of realtime music performance with sensor/MIDI, cooperates some researchers and composers, and composes experimentally pieces. He is also the key-member of Japanese computer music community. From 2000, he has been also the associate professor of SUAC(Shizuoka University of Art and Culture), Faculty of Design, Department of Art and Science, and teaches multimedia, computer music and media art. As a composer of computer music, he collaborates many musicians in his composition: Piano, Organ, Percussion, Vocal, Flute, Sho, Koto, Shakuhachi, Dance, Percussion, etc. He organized and was the General Chair of NIME04. He became a professor on April 2007. He organized a tutorial “Parallel Processing Platform for Interactive Systems Design” at ICEC2009 (Paris).

http://nagasm.org/ASL/profile/

Lecture/Workshop of Yoichi Nagashima (in English)

  1. Lecture "Technology for Computer Music", Seoul, Korea, 1997
  2. Workshop "Sensors for Interactive Music Performance", ICMC2000, Berlin, Germany, 2000
  3. Workshop "Sensors for Interactive Music Performance", Studio CCMIX, Paris, France, 2001
  4. Lecture "Composition of [Visional Legend]", International Workshop on "Human Supervision and Control in Engineering and Music", Kassel, Germany, 2001
  5. Lecture "Interactive Media Art with Biological Interfaces", STEIM Open Lecture Concert Series, Amsterdam, Holland, 2004
  6. Lecture "Interfaces for Interactive Media Art", IRCAM Open Lecture, Paris, France, 2004
  7. Lecture "Interactive Art with Bio-Interfaces", La Kitchen Open Lecture, Paris, France, 2004
  8. Lecture "Interactive Multi-Media Performance with New Interfaces", International Workshop on Interactive Media Art, Kai-Nan Univ., Taiwan, 2007
  9. Lecture "Interactive Multi-Media Art with New Technology", Tsukuba Univ., Japan, 2007
  10. Lecture "SUAC Installation - Case Studies as Physical Computing -", Sketching in Hardware 2008, Rhode Island School of Design, USA, 2008
  11. Lecture "Some interactive works ('08-'09) - case study of my sketching”, Sketching in Hardware 2009, University College of London, UK, 2009
  12. Tutorial "Parallel Processing Platform for Interactive Systems Design”, International Conference on Entertainment Computing, Conservatoire National des Arts et Metiers, Paris, 2009
  13. Lecture "Technology for Computer Music / Interactive Multi-Media Performance with New Interfaces", International Festival/Competition SYNC. 2010 Tutorial(1), The Ural State Conservatory, Yekaterinburg, Russia, 2010
  14. Lecture "SUAC Installation - Case Studies as "Physical Computing", International Festival/Competition SYNC.2010 Tutorial(2), The Ural State Conservatory, Yekaterinburg, Russia, 2010
  15. Lecture "Interactive Art with Bio-Interfaces", International Festival/Competition SYNC.2010 Tutorial(3), The Ural State Conservatory, Yekaterinburg, Russia, 2010
  16. Lecture "Sketching as entertainment of design”, Sketching in Hardware 2012, University of Oregon, USA, 2012
  17. Lecture "New Platform for Design Entertainment”, Sketching in Hardware 2013, PARC, California, USA, 2013
  18. Lecture "Against the boom of image recognition sensors”, Sketching in Hardware 2015, Biosphare 2, Arizona, USA, 2015
  19. Lecture "Biofeedback Instruments and Improvisation”, 4th tempora International Meeting, Bordeaux, France, 2016
  20. Lecture "Interactive Media Arts - New Ideas and New Technologies", General Lecture for Ekaterinburg's Contemporary Art Academy, Gorky library, Yekaterinburg, Russia, 2016
  21. Lecture "Interactive Media Arts - New Ideas and New Technologies", General Lecture - open public, Ural branch of the National Center for Contemporary Arts, Yekaterinburg, Russia, 2016
  22. Workshop "Introduction of Programming - Creating Art Objects", Practical Workshop for Art Academy's Designer Students, Center of Culture “Ordzhonikidzevsky”, Yekaterinburg, Russia, 2016
  23. Workshop "Interactive System Design - Creating Media Arts", Practical Workshop for Art Academy's Programmers Students, Center of Culture “Ordzhonikidzevsky”, Yekaterinburg, Russia, 2016
  24. Lecture "Human-Computer Interaction and Media Arts (part1/2)”, Lecture Workshop at MARS Gallery, MARS Gallery, Moscow, Russia, 2016
  25. Lecture "Bio-Sensing and Bio-Feedback Interfaces”, Conference “Musical Interfaces and Robotics”, Lecture Workshop at Tokyo University of Arts, Japan, 2017

YouTube Demo movies

Presentation and papers

VPP-SUAC (sorry, currently in Japanese only - please use "Google Translate")

PAW-double (sorry, currently in Japanese only - please use "Google Translate")

Question to the tutorial workshop organizer

email (please use "Subject: about ICEC2018 tutorial workshop")