Mongoose: Cast to array of object failed for value [{"someKey": "someValue"}]

I'm getting error whenever I'm trying to save into my database a document with this schema:

var schemaForBooks = new Schema({
    book: String,
    author: String,
    who_has_this: Object,
    points: Number,
    upvoted_by_users: [Schema.Types.ObjectId],
    downvoted_by_users: [Schema.Types.ObjectId]
});

Rest all is good, but putting anything into upvoted_by_users or downvoted_by_users, I get this error:

[ERROR] Trace- CastError: Cast to [ObjectId] failed for value "[{"userName":"Vibhu","userId":"3833d1g870feaf4a38723"}]" at path "upvoted_
by_users"

I'm pretty sure I'm doing something wrong with the schema itself, but I don't know what.

Any help would be appreciated.

via Vibhum Bhardwaj