Transmitting OSC data via WebSocket

Sometimes I meet situations where I need OSC but cannot use UDP.

Such as sending OSC message from browser-based p5js to Processing / MaxMSP / Unity.
And I’m developing an OSC controller called BugOSC. It is a WeChat Mini Program, which also does not support UDP.

However, although the web guys usually don’t support UDP, they support WebSocket — another network transmission protocol.

So I built some examples about transmitting OSC data through WebSocket.

Two main things there:

• WebSocket transmitting
• OSC data packing and parsing (unpackIng)

BTW, the data transmitted are binary packets (ArrayBuffer / byte[]) following the standard OSC data format.

I use p5.js as OSC sender, and the receivers are implemented in Processing, MaxMSP, Unity and so on.

p5.js <— — > Max

OSC sender — p5.js WebSocket client

p5.js is a browser-based JavaScript library, which is no UDP supported because of some secure reasons. Fortunately WebSocket sit there.

There is a wonderful library: osc.js.
It supports UDP (NodeJS, no browser of course), WebSocket, TCP and Serial port, wow!
And it can pack and parse OSC data.

//p5.js sketch:
function setup() {
    createCanvas(400, 400);
    oscWebSocket = new osc.WebSocketPort({
        url: "ws://localhost:12345",
        metadata: true
    });
    oscWebSocket.on("ready", onSocketOpen);
    oscWebSocket.on("message", onSocketMessage);
}
function onSendClick() {    
    const msg = input.value();
    input.value('');
    // send the OSC message to server. 
    //(osc.js will convert it to binary packet)
    oscWebSocket.send({
        address: "/p5js/sayhi",
        args: [{
            type: "s",
            value: msg
        }]
    });
}

OSC receiver — Processing

p5.js < — → Processing

WebSocket

There is a very good processing WebSocket library.

But it has one problem: transmitting data type is String only.

If you need binary data type as my OSC controller BugOSC did, fortunately there is a fork supporting binary data transporting, but with some bugs.

I made a bug fixed fork so you can use it before processing WebSocket library author accepting my pull requests.

Or you can just download the JAR library I built (with Processing 3) directly.

OSC data parsing

Just use oscP5.

Note: the parser function of oscP5 is protected. I didn't fork it, while I just sent it to localhost and received it again using oscP5 UDP and then I got the correct parsed data.

// Processing sketch:
void setup(){
    //WebSocket
    ws = new WebsocketServer(this, 12345, "/");
    //use OSC (UDP) to parse the data
    oscP5 = new OscP5(this,12346);
    oscHost = new NetAddress("127.0.0.1", 12346);
}
// The data receiving handler, support byte[]
void webSocketServerEvent(byte[] buf, int offset, int length){
    //send to OSC (UDP) to parse the data   
    OscP5.flush(buf, oscHost);
}

OSC receiver — MaxMSP

p5.js < — → Max

There have been already some “Max to web browser” showcases:

• xebra.js from MiraWeb
• wsserver

And now Max 8 has a wonderful feature Node for Max. I chose it as my solution. So the WebSocket and OSC functions in this example are actually implemented through NodeJS.

Still use the library osc.js to pack and parse OSC data, and transmit data via WebSocket.

webSocketPort = new osc.WebSocketPort({
    socket: ws
});
// receive message
ws.on("message", function incoming(message) {
    const msgParsed = osc.readPacket(message, { metadata: true });
    console.log("received parsed : ", msgParsed);
    
    maxAPI.outlet('message', msgParsed);
});
// send message
maxAPI.addHandler("send", (...args) => {
    if (webSocketPort && isConnected) {
        webSocketPort.send({
                address: "/max/midi",
                args: [{
                        type: "i",
                        value: args[0]
                }]
            });
    }
});

OSC receiver — Unity

p5.js < — → Unity

WebSocket

websocket-sharp is a C# implementation. I have built a websocket-sharp.dll and added it into Unity project folder Assets/.
If the dll does not work for you, you can build it yourself.

OSC data parsing

There are several OSC libraries for Unity, but actually only OSC data packing and parsing is necessary here.
VVVVUnityOSC has convenient methods to do this.

packing: new OSCMessage("/unity/midi", 87).BinaryData
unpacking: OSCPacket.Unpack(oscPacketRawData)

public class WebSocketOSC : MonoBehaviour
{
    public WebSocketServer wssv;
    void Start()
    {
        wssv = new WebSocketServer("ws://127.0.0.1:12345");
        wssv.AddWebSocketService<Echo>("/");
        wssv.Start();
    }
}
public class Echo : WebSocketBehavior
{
    //receive message
    protected override void OnMessage(MessageEventArgs e)
    {
        if (e.IsBinary)
        {
            OSCPacket msg = OSCPacket.Unpack(e.RawData);
            Debug.Log(string.Format("recv OSC addr: {0}, value: {1}", msg.Address, msg.Values[0]));
        }
    }
    protected override void OnOpen()
    {
        //send message
        Send(new OSCMessage("/unity/midi", 87).BinaryData);
    }
}

Other OSC receivers

Pure Data

• Websocket-Server-in-a-Patch

“websocket-server in a patch.pd” single Pd file contains a reusable and fully documented websocket server, meant as an easy way to experiment with Pure Data <-> web-browsers communications through websockets, without the need of other software (like node.js or python) or any specific external to compile (just one deken library is required).

• pd-ws
  Communicate between Pure Data & web browser over a websocket.

VVVV

• C# websocket-sharp
• VVVV Websocket (Network Server)
• VVVV Venode It is more like a NodeJS bridge solution.

Bridge

Of course also can make a third party NodeJS/Python/etc BRIDGE focus on receiving data through WebSocket and forwarding to other OSC receivers:
p5js <---WebSocket---> BRIDGE <---OSC---> receivers

More

All the example source code attached bellow:
github.com/avantcontra/osc-websocket-example
Welcome PR!👏

Cheers~

Contra

• patreon (buy me a coffee XD): avantcontra
• website: floatbug.com
• facebook: avantcontra
• twitter: avantcontra
• 微信公众号/知乎专栏：浮生开方