Ascend
An interactive climbing wall: projected games that follow every climber, tracked by two depth cameras. Pick a game on the touchscreen, then climb.
Ascend turns a bouldering wall into a game board. Climbers choose a game on a touchscreen next to the wall, step onto it, and the wall comes alive around them: holds light up to be slapped, lava rises from the floor, notes play where they put their hands. Two climbers can race each other side by side, red against blue.
The idea came from my own climbing sessions. Between attempts I kept wondering how my work in multimedia could make the sport more playful and more challenging, for beginners and children as much as for regulars. Ascend is the answer I built, and it was shown at EXPO24.
How it works
Two Azure Kinect depth cameras watch the whole wall and track every climber's skeleton in real time. Each camera runs on its own computer to spread the load, and a synchronisation protocol I wrote merges the two into one picture of the wall. Unity runs the games and TouchDesigner the visual effects, and a projector paints them onto the holds. The wall itself was built to climbing-gym standards, with the cameras and projector built into it.
The games include Wall is Lava, Juggling, Twister and Melody Mount, alongside Pong and whack-a-mole-style hold hunting. Everything starts automatically with the computers, so the wall can run all day without anyone at a keyboard.
Process
I built Ascend from December 2023 to June 2024 as my graduation project. I tested several ways of tracking climbers (Kinect v2, Azure Kinect, ZED 2 and MediaPipe) before choosing two Azure Kinects for their field of view, then built the wall, the software and the touchscreen app. Rounds of user testing changed a lot: more and more varied holds, gentler difficulty for beginners, targets placed lower so children can join in, and calmer visuals and sounds so climbers can concentrate. The full story is in the project book below.
Built with
- Two Azure Kinect depth cameras, synchronised across two computers
- Unity for the games, TouchDesigner for the visuals
- A projector and a touchscreen game picker
- A climbing wall built for the project
The project book
The booklet I made for the project, in Dutch. Turn the pages.
