the body and the brain
as controller	

Talk to Digital Culture students 
@University of Jyvaskyla, Finland 	

15 April 2013
	

erik champion	

	

http://erikchampion.wordpress.com 	

nzerik@gmail.com
Civilization comprises the laws that allow
people to live close together, in a city, civitas. 

Culture is what is cultivated or allows one to
cultivate a setting, a local domain. 	

Osvald Spengler
virtual heritage	

• 

• 

…the use of computer-based interactive technologies to
record, preserve, or recreate artefacts, sites and actors of
historic, artistic, religious, of cultural significance and to
deliver the results openly to a global audience in such a way
as to provide formative educational experiences through
electronic manipulations of time and space.
Stone, Robert, and Takeo Ojika. 2000. Virtual heritage:
what next? Multimedia, IEEE no. 7 (2):73-74.
al meetings no more interesting tha
ones so why recreate mundality?
digital-interactivity sometimes better than the dig
Goodbye body	

•  Spengler wrote

This machine technics will
end with the Faustian civilization and one
day will lie in fragments, forgotten -- our
railways and steamships as dead as the
Roman roads and the Chinese wall, our
giant cities and skyscrapers in ruins like old
Memphis and Babylon.
scale is also important
the brain	

•  space and memory are inextricably linked	

•  memory is more a jigsaw than filing cabinet	

•  memory decay can be challenged	

•  games help multi-tasking	

•  biofeedback can help evaluation
Caption: The flow of object information in a monkey brain (left) and a human brain.
Credit: Sabine Kastner, Princeton University

Humans See Tools Differently Than
RNA Game (Wired magazine) EteRNA 	


•  Computers don t have flashes of insight. But th
human brain can..for gamifying RNA. 	


•  One proposal they kicked around was to have

snippets of RNA fight each other to the death,
in the style of a Japanese combat game called
Senshuken; another was to create a first-perso
experience in which players navigated the worl
as RNA molecules of their own design.
http://publicvr.org/html/pro_oracle.html
prototyping for ownership	

Klaus Birk Roman Grasy

media architecture biennale
http://www.mediaarchitecture.org/biennale-2012-workshops/
or http://moritzbehrens.com/2013/mab12/ 
15 November 2012, Aarhus
sketchy sketches
Xbox Kinect+Projector+VVV
dance projection	


• imagine...	

•  public squares	

•  flashmob dance mixes	

•  crowdsourcing VJs
machinima tools
MOODLE-unity
kinect, jibe, drawing
peripheral projection
warped screen
peripheral fear	


our peripheral vision is not
olour, it reacts to movement	

how to best utilize this?
biofeedback
biofeedback
m game can host virtual recreations (of
c stories or any other), the player can
l the avatar, and issue voice
ands recognised by the game).
tants can be easily reprogrammed to
stories. Trading, praying, conversing
g etc are possible, not just violence.!

Skyrim + body
Oblivion

Eric Fassbender PhD thesis project, s
tour guide of lighthouse, Macquarie
mixed reality 


Second Life and VR tracker and goo
Georgia Tech Project!
2006 calligraphic

University of Queensland: project by
multimedia students, Amy and Isaac
Journey to the West
touch screen taoism
“new media…offers enormous possibilities
for the enhancement and enrichment of
heritage experience and interpretation”
http://chinablog.cc/2009/10/siyi-four-arts-of-the-chinese-scholar/

	

Qin literally refers to a unique seven-string Chinese music
instrument Guqin , which was invented 3,000 years ago in
http://chinablog.cc/2009/10/siyi-four-arts-of-the-chinese-scholar/
	


Daoism is an ancient Chinese combination of religion, philosophy, and folk
beliefs, including ritual healing. Its different strands of belief date far back in
history. Daoism is deeply entwined with Chinese culture and history.

Arts .	


•  The

Four Arts are Music ( Qin ), the
board game ( Qi ), calligraphy ( Shu ), and
brush painting ( Hua ). 	


•  Helped

perceive the ultimate doctrine of
the heavens , make themselves [be]
enlightened , express their emotions/their
understanding of the doctrine , and inspire
others so that their lives achieve peace

and harmony. 	

Z. Dainian, Key Concepts in Chinese Philosophy. Beijing: Foreign Languages Press, 2002.
youtube links	

•  Opening http://youtu.be/gFYG4zTn4Js	

•  Game Hua http://youtu.be/DiGDezTM8hY	

•  Game Qi1 http://youtu.be/jP9nfdUFDTU	

•  Game Qi2 http://youtu.be/orCga2CQBjs	

•  Game Qin http://youtu.be/iC2BGT5IbDE	

•  Game Shu http://youtu.be/dv_TOnl_sbc
• 
• 
• 
• 
• 

What are the Five Elements in the traditional Chinese
culture?	

What are the Five Basic Tones in the traditional Chinese
music?	

What are the traditional Chinese philosophical concepts
revealed by Go?	

Which one of the following features is one of the main
features of Chinese character writing system? ]
Cuneiform / [ ] Alphabet / [ ] Pictography / [ ]
Phonology?	

What are the tools for Chinese traditional painting?
•  Very difficult to recreate original action scenes and
moments of discovery as game devices. 	


•  Chinese players, familiar with a distorted version of

the original, not aware their cultural knowledge was
not accurate, did not appreciate being told this.	


•  Recreating linear narrative via game design is
torturous. 	


•  OR: simulate the procedural knowledge of rituals

and symbol-making via thematically-akin interaction..
• 
• 
• 
• 
• 
• 
• 
• 
• 

what is needed: more comprehensive pre-test and post-test questionnaire.	

The general questions are too vague. 	

Consider changing from rating games to ranking them.	

Test extrapolated knowledge rather than memory of simple facts. 	

Examine how tacit knowledge can be learnt and evaluated.	

Compare tests between touch-screen and non-touch screen games. 	

A more 3D interface: sculptures, HD projection on rice paper or liquid
media, with 3D audio effects and ambient movies projection.	

Ambient movies react to the player s physiological changes -by biosensors.	

Can interactive digital media convey and evaluate tacit knowledge?
crazy ideas
•  Dutch thriller App sends
to your phone additional
storyline information..	


•  why not have

biofeedback sent back to
the movie..
fps...	

•  when your character dies, why not have

the death spasms relate to the heartbeatGSR?	


•  or only respawn when you calm down..	

•  reactions relate to heartbeat	

•  enemies attack peripherally sensing fear
RPGs-chameleon effect	

•  when you start shaking you lose your

character or change between ethnic profile	


•  Cinematic camera views=biofeedback	

•  you can only convince NPCs when calm
sew some characters	


•  http://lilypadarduino.org/
reflective games	


Kuthodaw Paya: must walk around Buddhist
http://www.dreamingmethods.com/	


IF+3D model+body

Jyväskylä 15/04/2013 talk: the body and the brain…

  • 1.
    the body andthe brain as controller Talk to Digital Culture students @University of Jyvaskyla, Finland 15 April 2013 erik champion http://erikchampion.wordpress.com [email protected]
  • 2.
    Civilization comprises thelaws that allow people to live close together, in a city, civitas. Culture is what is cultivated or allows one to cultivate a setting, a local domain. Osvald Spengler
  • 3.
    virtual heritage •  •  …the useof computer-based interactive technologies to record, preserve, or recreate artefacts, sites and actors of historic, artistic, religious, of cultural significance and to deliver the results openly to a global audience in such a way as to provide formative educational experiences through electronic manipulations of time and space. Stone, Robert, and Takeo Ojika. 2000. Virtual heritage: what next? Multimedia, IEEE no. 7 (2):73-74.
  • 4.
    al meetings nomore interesting tha ones so why recreate mundality?
  • 5.
  • 6.
    Goodbye body •  Spenglerwrote This machine technics will end with the Faustian civilization and one day will lie in fragments, forgotten -- our railways and steamships as dead as the Roman roads and the Chinese wall, our giant cities and skyscrapers in ruins like old Memphis and Babylon.
  • 7.
    scale is alsoimportant
  • 8.
    the brain •  spaceand memory are inextricably linked •  memory is more a jigsaw than filing cabinet •  memory decay can be challenged •  games help multi-tasking •  biofeedback can help evaluation
  • 9.
    Caption: The flowof object information in a monkey brain (left) and a human brain. Credit: Sabine Kastner, Princeton University Humans See Tools Differently Than
  • 10.
    RNA Game (Wiredmagazine) EteRNA •  Computers don t have flashes of insight. But th human brain can..for gamifying RNA. •  One proposal they kicked around was to have snippets of RNA fight each other to the death, in the style of a Japanese combat game called Senshuken; another was to create a first-perso experience in which players navigated the worl as RNA molecules of their own design.
  • 12.
  • 13.
    prototyping for ownership KlausBirk Roman Grasy media architecture biennale http://www.mediaarchitecture.org/biennale-2012-workshops/ or http://moritzbehrens.com/2013/mab12/ 15 November 2012, Aarhus
  • 14.
  • 15.
  • 16.
    dance projection • imagine... •  publicsquares •  flashmob dance mixes •  crowdsourcing VJs
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
    warped screen peripheral fear ourperipheral vision is not olour, it reacts to movement how to best utilize this?
  • 22.
  • 24.
  • 25.
    m game canhost virtual recreations (of c stories or any other), the player can l the avatar, and issue voice ands recognised by the game). tants can be easily reprogrammed to stories. Trading, praying, conversing g etc are possible, not just violence.! Skyrim + body
  • 26.
    Oblivion Eric Fassbender PhDthesis project, s tour guide of lighthouse, Macquarie
  • 27.
    mixed reality SecondLife and VR tracker and goo Georgia Tech Project!
  • 28.
    2006 calligraphic University ofQueensland: project by multimedia students, Amy and Isaac
  • 29.
  • 30.
  • 31.
    “new media…offers enormouspossibilities for the enhancement and enrichment of heritage experience and interpretation”
  • 32.
    http://chinablog.cc/2009/10/siyi-four-arts-of-the-chinese-scholar/ Qin literally refersto a unique seven-string Chinese music instrument Guqin , which was invented 3,000 years ago in
  • 33.
    http://chinablog.cc/2009/10/siyi-four-arts-of-the-chinese-scholar/ Daoism is anancient Chinese combination of religion, philosophy, and folk beliefs, including ritual healing. Its different strands of belief date far back in history. Daoism is deeply entwined with Chinese culture and history.

  • 34.
    Arts . •  The FourArts are Music ( Qin ), the board game ( Qi ), calligraphy ( Shu ), and brush painting ( Hua ). •  Helped perceive the ultimate doctrine of the heavens , make themselves [be] enlightened , express their emotions/their understanding of the doctrine , and inspire others so that their lives achieve peace and harmony. Z. Dainian, Key Concepts in Chinese Philosophy. Beijing: Foreign Languages Press, 2002.
  • 35.
    youtube links •  Openinghttp://youtu.be/gFYG4zTn4Js •  Game Hua http://youtu.be/DiGDezTM8hY •  Game Qi1 http://youtu.be/jP9nfdUFDTU •  Game Qi2 http://youtu.be/orCga2CQBjs •  Game Qin http://youtu.be/iC2BGT5IbDE •  Game Shu http://youtu.be/dv_TOnl_sbc
  • 36.
    •  •  •  •  •  What are theFive Elements in the traditional Chinese culture? What are the Five Basic Tones in the traditional Chinese music? What are the traditional Chinese philosophical concepts revealed by Go? Which one of the following features is one of the main features of Chinese character writing system? ] Cuneiform / [ ] Alphabet / [ ] Pictography / [ ] Phonology? What are the tools for Chinese traditional painting?
  • 37.
    •  Very difficultto recreate original action scenes and moments of discovery as game devices. •  Chinese players, familiar with a distorted version of the original, not aware their cultural knowledge was not accurate, did not appreciate being told this. •  Recreating linear narrative via game design is torturous. •  OR: simulate the procedural knowledge of rituals and symbol-making via thematically-akin interaction..
  • 38.
    •  •  •  •  •  •  •  •  •  what is needed:more comprehensive pre-test and post-test questionnaire. The general questions are too vague. Consider changing from rating games to ranking them. Test extrapolated knowledge rather than memory of simple facts. Examine how tacit knowledge can be learnt and evaluated. Compare tests between touch-screen and non-touch screen games. A more 3D interface: sculptures, HD projection on rice paper or liquid media, with 3D audio effects and ambient movies projection. Ambient movies react to the player s physiological changes -by biosensors. Can interactive digital media convey and evaluate tacit knowledge?
  • 39.
  • 40.
    •  Dutch thrillerApp sends to your phone additional storyline information.. •  why not have biofeedback sent back to the movie..
  • 41.
    fps... •  when yourcharacter dies, why not have the death spasms relate to the heartbeatGSR? •  or only respawn when you calm down.. •  reactions relate to heartbeat •  enemies attack peripherally sensing fear
  • 42.
    RPGs-chameleon effect •  whenyou start shaking you lose your character or change between ethnic profile •  Cinematic camera views=biofeedback •  you can only convince NPCs when calm
  • 43.
    sew some characters • http://lilypadarduino.org/
  • 44.
    reflective games Kuthodaw Paya:must walk around Buddhist
  • 45.