RNBO for Web sync with Mac
I would like to use Max on a Mac as a primary (timing) sync source (controller) for multiple Web Instances of RNBO patches running on audience iPhones. The idea is to have all of the audience's phones serve as the audio output — the speakers — for the piece.
I can do this without sync, but I'm hoping for a way to add form to a piece, rather than just having a sort of amorphous disorganized drone.
Order from chaos. Possible?
If so, I would love some guidance, since while I'm pretty adept at Max (proud user since 1990 — no joke) I'm a complete WebSocket/WebRTC novice.
Florian D, Andrew B, or others... can you help?
With appreciation!
Mike
Hi Mike,
fun project! Rather than going heavily into implementation specifics, I'm happy to share what crossed my mind in order to achieve such project.
Reading your post I assume that the playback is happening locally on each device (no audio streaming from Max) and you want to achieve playback sync. Given that I'd assume you need a few things:
some sort of agreed upon general, universal time and a way to synchronize clients to it
some broker that both, phone clients as well as Max, can connect to and communicate with, esp. for events / messages sent from Max
a way to schedule events in RNBO on the client side
For 1/ synchronizing time I'd look into NTP, specifically SNTP to get your clients aligned on a timestamp, that can then also be used as a basis to put the AudioContext's current time in relation.
For the broker and communication you could use some sort of peer-to-peer room concept utilizing WebRTC or go through a server that clients connect to via WebSockets. It's difficult to make a suggestion on what's best without a better understanding of your requirements and the infrastructure you have available and would like to use.
If you'd like to WebRTC based communication you can test something like using PeerJS that AFAIR offers some default, public server to handle the brokering if that fits your security needs. Alternatively you can rely on a server driven driven messaging pattern (e.g. using WebSockets) rather than implementing peer-to-peer communication, esp. with network topology / complexity is a concern.
In order to schedule events ahead of time across devices you can use device.scheduleEvent on the WebBrowser. This way Max would basically become a "special" client that can tell the other devices / clients ahead of time when to start playback or make certain events happen ahead of time in order to ensure some sense of synchronicity.
While googling around there seem to be various pieces available for each portion of this stack if you'd like to avoid rolling things your own entirely. One thing that also stood out in terms of documentation wrt WebAudio was soundworks or just the sync piece.
As an aside if you are further interested in accurate timing with WebAudio and understanding of the clients' audio output latency - This blog post from Paul is a good introduction to the baseLatency and outputLatency properties:
https://blog.paul.cx/post/audio-video-synchronization-with-the-web-audio-api/
Would be great to hear how things come along and see the projection in action!
Florian
Florian,
First of all, thank you for your detailed thoughts here. I appreciate you spending some brain cycles on it!
Of course, as a web tech newbie, most of this is Greek to me, but I do know a few people locally who might understand how to implement this for/with me. It's one of those things that is important enough to me to pay someone to do it right first, and then once I see the implementation I will be able to better understand how it works if/when I need to edit/tweak (and likely, break!) the code. If you know anyone who fits this bill, happy to get their contact information!
I think that for the purposes of a concert, security is not super important. I have a VPS, so I can install libraries and/or host anything I need to.
Hopefully this project won't be too difficult to execute. The hard part is making it a decent audience experience!
Again, my thanks.
Michael