Socket.io not working with more than 100 users

When my site has more than ~80 users online, the socket.io fails to make a handshake. I checked my nginx logs and this error appears often:

*374 upstream prematurely closed connection while reading response header from upstream, client:

So I checked my nginx config and this is what I have:

files:
    "/etc/nginx/conf.d/01_websockets.conf" :
    mode: "000644"
    owner: root
    group: root
    content : |

        upstream nodejs {
            server 127.0.0.1:8081;
            keepalive 256;
        }
        server {
            listen 8080;

            location / {
                proxy_pass  http://nodejs;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                proxy_http_version 1.1;
                proxy_set_header        Host            $host;
                proxy_set_header        X-Real-IP       $remote_addr;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
            }
        }

"/opt/elasticbeanstalk/hooks/appdeploy/enact/41_remove_eb_nginx_confg.sh":
    mode: "000755"
    owner: root
    group: root
    content : |
        mv /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf.old

~

//client side
var origin = window.location.origin;
console.log("origin is: " + origin);
var socket = io.connect(origin, {'force new connection': true});

//server side
app.set('port', process.env.PORT || 8081);

var server = require('http').createServer(app);
var io = require('socket.io')(server);

server.listen(8081);

Is there something I'm doing wrong?

via Chris Jones