Possibly or it could be a stretched chain. But you can put a bolt with a nut through the holes and crank it down with some lock washers and you will be fine on that part. Try that then if it pops off again your chain is stretched. Or you could pull off the chain set it flat on the ground and try and bend it into a U shape. If it goes pretty far your chain is stretched. If its good it wont even be able to bearly move into a U shape at all.