"Failed to parse server response" when using Google Assistant SDK with GRPC in Node.js

I've created a bidirectional stream with assistant.converse(), and sending audio is successful. When I stop talking and Assistant detects silence, the Assistant throws this error: I'm loading the grpc protocol with grpc.load('embedded_assistant.proto').google.assistant.embedded.v1alpha1;.

{ Error: Failed to parse server response
    at ClientDuplexStream._emitStatusIfDone (/Users/arilotter/Projects/assistant/node_modules/grpc/src/node/src/client.js:201:19)
    at ClientDuplexStream._receiveStatus (/Users/arilotter/Projects/assistant/node_modules/grpc/src/node/src/client.js:180:8)
    at /Users/arilotter/Projects/assistant/node_modules/grpc/src/node/src/client.js:649:14 code: 13, metadata: undefined }

via Ari Lotter