I moved to a new city on the other side of the country. What better way to meet people than invite strangers to my apartment for a social dance event?
A work colleague said I was brave.
I wanted to make the event different from the multitude of low-turnout meetups and Facebook events.
Being the musician and tinkerer I am, and since I wanted to host similar events in the future, I wanted to create an experience.
What better experience could there be than having the music react to the people dancing?
I needed to map the movement and energy of people dancing to the music arrangement in my DAW.
This is a solved problem right? It's not.
Goals
- Plot spatial coordinates (2D at minimum, 3D if possible) of each person in the room.
- Each person's movements should have an effect on the music.
- Do this without requiring them to carry any additional devices.
Initial thinking
- LIDAR/Sonar won't work.
- Accuracy within ~10-20cm should be adequate (with smoothing).
- Use a wavelength that can pass through people (without harming them) somewhere in the 500MHz to 5GHz range.
- Deploy 2 to 3 nodes (Tx/Rx pairs) statically from a few points in the room.
- Do some signal processing to 'triangulate' (and smooth) coordinates of all the people in the room.
- This is the equivalent of distilling a PhD-level algorithm into a one-liner during sprint planning.
Problems
- There was very little reference material to go on. All of it was highly academic with controlled testing environments.
- I didn't know anything about research-grade signal processing.
The event
My initial thinking might have worked for a sparsely populated room, but there were ~100 people confirmed for this event. Plotting dense/tight groups (like people dancing together) in a crowded room introduces several issues (interference, shadowing, occlusions… signals would get too correlated/complex).
Given the complexity involved, I wasn't able to get this running for the event. Instead, I hacked a few basic depth sensors together and used those instead (it was quite 'dumb', but at least it worked).
For me, it wasn't the 'Whoa, awesome!' moment I hoped for. How could it be when I had my sights set so highly? For everyone else, it was a blast.
Current research
Looks like I was (and am) on to something.
Five/six years later, there are papers coming out about using millimeter-wave radar for this purpose.
- Research on Indoor Multi-Target Detection and Tracking Algorithms Based on Millimeter-Wave Radar
- A multi-target detection and position tracking algorithm based on mmWave-FMCW radar data
- A Real-Time UWB-Based Device-Free Localization and Tracking System
- mmFlux: Crowd Flow Analytics with Commodity mmWave MIMO Radar
Current thinking (as of Jan 2026)
This tech has way more useful applications beyond a fun house party.
- Deploy more nodes in a multi-static config.
- Perform signal processing (range-doppler maps, background subtraction, localization).
- Combine range/angle/doppler to assign unique positions for each person, and track position over time (keep mapping as long as they're in the room).
- Enhance processing speed with AI (use neural network(s) for spectral distortion classification).