Since you live in California and the server is in NJ. Possibly distance lag. Like you know, if people live closer to the server they get a ping around 3-10. Also matters how much your bandwidth is. And when there is more people inside the server everyone's ping slightly increases by 1-3 per person joining.