1. Name of Game
Fuze
A gestural music game prototype
From
SonicViz
http://www.essentialreality.com/p5_glove.asp +
Logitech Wingman RumblePad (Vibration Feedback)
2. Description of Game (300 words or less).
Welcome to the worlds of Fuze!
Fuze is very simple to play. Use two P5 gloves ( http://www.essentialreality.com/) to
control two glove Picons (play icons) in 3d space. In front of you,
floating high in the air of an open environment, sits the Wire. A
generative ambient music track ebbs and flows in the background.
Game mode consists of passing both Bicon and Micon challenges before attaining the next rank.
How?
Watch the Fuze travel down the static or rotating wire while:
- You hit the Bicons (Beat Icons) in time with the Fuze to generate music beats. Points for accuracy, timing and obstacle avoidance.
- You trace the Micons (melody icons) in time with the Fuze to generate melodic gestures. Points for accuracy, timing and obstacle avoidance.
As you move through the ranks new challenges will keep opening ahead – and freeing up more juicy audio, visual and vibrational feedback to help you on your way to “The Zone”.
The music is created on the fly by real time software synthesis, which also enables sounds to be generated that are directly related to player gestural input dynamics, such as speed, motion, and precision. In addition, vibrations and visual effects are synchronized to the generated sounds to impart a unique experience.
It’s a total mind/body Fuzion of 3D movement, sound, visual effects and vibrations!
|
Rank |
Accuracy Required |
Wire type Challenges |
Bicon/Micon Challenges |
Instruments |
Rumble Feedback |
|
Noob |
Low |
Static, single type |
2/1 |
Single pitched |
Single simple |
|
Hacker |
Medium |
Single axis rotating wire, low speed, single wire type |
3/2 |
Single pitched |
Dual simple |
|
Pro |
High |
Dual axis rotating wire, low/med speed, multiple wire types (dynamically generated) |
4/3 |
Two - match Picons to the Bicons/Micons. Switch control by P5 finger gesture. |
Dual complex |
|
Zoner |
High |
Tri axis rotating wire – low/med/high speed, multiple wire types (dynamically generated) |
4/3 |
Three - match Picons to the Bicons/Micons. Switch control by P5 finger gesture. |
Dual complex |
NB: Current proposed configuration. May change as gameplay tuning progresses.
Other modes:
- Practice
- Instrumental freeplay
- Alternative control setups
3. Explanation of how the core gameplay is experimental (30 words or less).
· An innovative user interface utilising bimanual 3D gestural control with vibration feedback
· Interactive real-time audio synthesis:
· Generative sequenced backing music
· SoundFX
· Dynamically generated game objects
4. Current phase of completion / Expected phase of completion at
workshop time. Please be descriptive of the level of functionality
present in the game; don't simply give a time-based or
percentage-done indicator.
Current phase of completion
|
Phase |
Function |
Status |
|
Current |
Engine: OpenGL renderer – lwjgl |
V0.94 Integrated |
|
Engine: Scenegraph API - jME |
V0.8 Integrated |
|
|
Engine: AI - FSM, Messaging, boids |
Complete |
|
|
Engine: Physics - JavaODE, SoftBody |
Complete |
|
|
Engine: Music/Sound - FMOD, OpenAL, JSyn |
Complete |
|
|
Engine: Game framework - GameState management, Configuration, Hud, P5 Input, particle system, Camera management |
Complete |
|
|
Game: FuzeBeat – Beat gesture game |
|
|
|
Game: FuzeMelody – Melodic gesture game |
|
|
|
Expected |
(Prioritised) |
|
|
Mode - Game: FuzeBeat – Beat gesture game |
|
|
|
Mode - Game: FuzeMelody – Melodic gesture game |
|
|
|
Mode: Practice |
|
|
|
Mode: Instrumental Freeplay mode |
|
|
|
Mode - Alternate control setups |
|
|
|
Game Mode – Dual Play: glove vs. glove |
|
|
|
Future |
||
|
Lots of ideasJ |
5. Give more detail on the gameplay experiment you are performing
(take as much space as you need here). Why is this experiment
important? What does it bring to games that is not already ubiquitous?
SonicViz is a framework for rapid prototyping of innovative music game concepts using Multimodal input (audio/physical gesture) combined with dynamic audiovisual content generation.
Some of the core areas I am researching are:
- Enactive Interfaces for computer games ( http://www.enactivenetwork.org/ )
- Bimanual gestural control issues
- Real time audiovisual object modulation and control by 3d gestures in a game context.
- Audio analysis and feature extraction to derive more expressive control signals beyond pitch and amplitude.
- Procedural/generative content
Obviously research into these areas has applications far beyond music games, but that area is my personal choice for prototype developmentJ
While the P5 glove has proven to be another market failure, it makes an excellent low cost 6+dof input device perfectly suited to the experimental research goals of this project.
- Gestural input is through two Essential Reality P5 gloves. The P5 has a total of 11 degrees of freedom (xyz translation/rotation + 5 finger bend sensors).
- The P5’s do not have rumble feedback, so vibrational feedback is provided by controlling a Logitech Wingman RumblePad for test purposes.
In terms of Mechanics, Dynamics, Aesthetics the core game concept was initially stated as: Addictive music/sound based gameplay with some eye/earcandy to show off interaction capabilities, using simple rules to generate engagement through directed and emergent game/play.
Why is this experiment important?
Otogei, or music oriented games, have been around for a decade or more. According to Masaya Matsuura ( "When the Music is the Game" - Game Developer magazine, Sound Principles Oct 2003) … “the success of otogei in general is based on a couple of innate parallels between music and gaming. First, when playing musical instruments, one achieves a desired effect through motion. Furthermore, musicians who train and develop their sense of motion improve their performance. The fundamental motions that create music are simple to perform: sing, beat, pluck and press".
Music games also share common design patterns (eg: Drummania, Para Para Paradise, Samba de Arrigo, Technic Beat, Beatmania, DDR, Rez, Frequency, VibRibbon):
· Notation - scrolling columns, Concentric Columns, Collapsing circles
· Input - Digital Drumset, Infrared Sensors, Maracas, Avatar Movement, Dancepad panels, Buttons, Turntable
· Response - Silent, Audio sample, Rumble (Rez), Interactive Music (Rez/Amplitude), Viz feedback (Rez, Amplitude)
· Scoring - Meter, Song Grading,
ref: http://devlab.flashbangstudios.biz/article.php?story=20030920184841901
There are also some steps towards enactive game interfaces and unique game experiences in products such as Eyetoy (2d gesture), Singstar (pitch detection), Bemani (Beat matching by buttons/switches), WarioWare and passive pen input on the Nintendo DS.
It is mainly due to the commercial factors involved in consumer controller markets that the musical gestures have been confined to the region of rhythmic gestures such as beat, pluck or press. As technology moves on and the power of target platform’s increase we can realistically start to imagine more novel, but useful, forms of interaction. The recent addition of real-time audio analysis (Singstar) has illustrated the uses of continuous motion, or melodic gesture, as a form of expression and control for otogei.
We are also now seeing the diffusion of 3d gestural controllers into the marketplace and can start to explore how these may be used to create engaging and emotionally moving gaming experiences. For example, Samsung has just announced inbuilt 3D motion recognition for their next generation phones.
Http://www.samsung.com/PressCenter/PressRelease/PressRelease.asp?seq=20050112_0000094230
Matsuura-san puts it well when he states:
"One of the big changes we need to see in the music game arena is the introduction and proliferation of new input systems, to replace the ancient button-based input method. The old structure of having four buttons on the right is no longer the ideal interface for full utilization of high performance hardware, and imagination is now far less restricted. Recently, in the area of arcade games, some developers found an easy way to diversify music games by focusing on building unique controllers such as turntables, guitars, drums, keyboards, and other musical instruments. Creative game developers should see this as an opportunity to experiment with innovative input systems too. I'm hoping that console manufacturers will focus their system development efforts in areas that can captalize on such experiences, and push music innovation forward."
What does it bring to games that is not already ubiquitous?
SonicViz is researching the next generation of music game input modes that include:
· Full 6 degree of freedom, with modal control via buttons or dials
· Audio feature extraction to derive more expressive control parameters beyond pitch and amplitude, such as harmonic content, formants, power.
· Dynamic content generation
Fuze points of difference:
· Notation: Specific to gestural interaction modes
· Input – 3D Gestural control
· Response - Real time audio synthesis, Camera control, 3D gestural control, SonicViz tactile integration (Sound/Viz/Rumble feedback)
· Scoring: Based on a number of features unique to gestural control
· Additional: Most music games are reactive but here we are exploring energetic input and feedback mechanisms.
6. How will you know whether the experiment is a success?
- Does it work technically, functionally and aesthetically? (MDA theory)
- Is it fun to play? [loaded question – means gameplay and instrumentJ]
- What can I learn from it? (ie: the feedback , -ve or +ve, is useful)
What problems and limitations might arise?
· Physical interaction is tiring.
· Controller technical performance limitations impede app design. In fact Fuze has been designed around the known limitations of the P5.
· Needs a certain spec of machine to run real-time audio and graphics together.
· Target markets may be predisposed to more visually rich environments
What do these limitations mean for future incarnations of this kind of gameplay?
· Better market research driven product design into human factors for controller design. P5 has a few problems in this area, possibly driven by cost factors.
· Publishers need to do more innovative R&D as the power of platforms grows.
· Good target market testing. This is tricky, but some good points are made in this article on Visual cues in 3d environments: “Self-testing a system or testing based on whether or not it appears realistic to the designers does not constitute proper testing. We have learned that our visual system is capable of learning how to understand various cues and how to separate out information that is deemed unnecessary or conflicting. Those who actively work with a system are not able to detect the problems with that system since they have most likely successfully adapted. When designing a test, it is crucial to consider the population at hand, the types of cues that they will observe (and in particular, what might present conflicting information) and the amount of experience that they are expected to have.” ref: http://www.danah.org/papers/sexvision.pdf
7. URL for Additional Info (downloads / info files / movies / etc.)
None specific to this experiment, but the following are available for more info:
SonicViz (Under construction): http://www.sonicviz.com/
Personal Bio: http://members.ozemail.com.au/~paulcohen/
8. Special Notes: Does your presentation require any unusual equipment
or preparations? Is there anything you feel needs to be said that
was not covered in this entry form?
A system configuration close in spec to my development machine (CPU speed, graphics (shaders etc), audio hardware (EAX, hardware acceleration)):
AMD64 3200+ (or intel equivalent), WinXP sp2, 512MB Ram, Nvidia FX5900XTV, Audigy 2 ZS, JDK 1.5
Best Regards,
Paul Cohen
BSc MTechMgt MMusic
28 Ernest St, Morningside
Qld , 4170
Australia
Ph: +61-7-38991293
Mobile 041 3287481
http://members.ozemail.com.au/~paulcohen/