Javascript replace function will not remove quotes

I'm currently doing the following to remove extraneous characters and quotes from my strings:

console.log(word);
word = word.replace(/[.,\/#!$%\^&\*;:{}=\-_`~()]/g, "");
console.log(word);
word = word.replace(/["']/g, "");
console.log(word);

Many words are being scanned, but my output tends to be:
“If
“If
"If

OR

time,”
time”
time”

Is my regex wrong?

via Jestus