The starting point
Pachinko is an algorithmic instrument developed with Dennis Enke (Murcielo). A gesture drops a ball into a network of bumpers. Collisions trigger sounds that continue to resonate as the ball moves through the system.
In the browser demo, a participant pinches to hold a ball, moves their hand to choose its entry position, and releases it. Gesture data shapes the event, while probabilistic routing introduces variation. Sound and fading light make the collisions perceptible.
The demo uses webcam hand tracking, a visual pachinko environment, and browser audio. The residency proposal develops this toward a room with multichannel sound, with projection as an optional component. The multichannel installation and the two-person interaction below are proposed developments.
A proposed two-person interaction
One participant initiates an event. Another shapes how it unfolds.
- Participant A releases the ball, choosing when and where it enters the system.
- Participant B uses a gesture to influence its path, changing which sound-producing bumpers it encounters.
- Both hear the result and can adjust their next actions in response.
- Participants can exchange roles and explore the other side of the interaction.
The first version would focus on changing the ball’s path, with audible consequences. Direct manipulation of sound processing remains a possible later variation. The two-person input and role assignment still need to be designed and implemented.
The relation I want to explore is concrete: I can start something, but what it becomes also depends on you. Participants might coordinate, follow one another, interfere deliberately, or discover a way of playing that I did not anticipate.
A question for the prototype
How do participants negotiate shared control when one initiates a sonic event and another shapes its unfolding?
A specific investigation within the broader research question, sitting on SQ2 and SQ3.
What to look for
- Does A wait for B to be ready before releasing a ball? How is readiness communicated?
- Does B follow the apparent intention of A’s action, redirect it, or introduce a surprise?
- How do participants respond when the result differs from what they expected?
- What changes when they exchange roles?
- When, if at all, do they describe the experience as something they are creating together?
These are observation and interview questions. Cooperation is one possible outcome, alongside interference, confusion, withdrawal, or continued individual exploration.
Development through research
Field observations of existing works and conversations with participants and designers will help refine which interaction to investigate. The prototype offers a way to explore a specific design decision emerging from that work.
For the first two-person version, I would reduce the randomness enough for participants to notice how each person’s action affects the result. Later iterations could introduce more uncertainty and examine how they respond. The balance remains a design question: too much variation may make the other person’s contribution difficult to distinguish from the system’s behaviour.
Through Research through Design, I will document the reasons for these choices, what changes across iterations, and what play sessions reveal. Observation notes and participants’ descriptions of particular moments can inform the next version. This study has not yet been conducted.
What this proposal exposes
The existing interaction connects a person’s gesture to a sonic system. Adding another input does not by itself establish shared play. The proposed roles make the participants’ contributions interdependent, but whether they notice and negotiate that interdependence needs to be studied.
Bodily co-presence also needs a concrete role in the design. The current proposal places two people in the same room; it does not yet establish which aspects of their interaction require that shared physical space. Their gestures, spatial arrangement, and shared listening are features to investigate.
Finally, Pachinko’s chance-based behaviour creates a tension: participants need enough uncertainty to explore, and enough perceptible influence to recognise one another’s actions. This is the design problem the next prototype will begin to examine.