Yup can't burn-up a motor (magnet side) from regular speaker use, even if it's abusive. Also speakers operate on AC power.
P.S.
Neodymium may lose BL as heat increases (regarding speaker application) but it's not permanent. It's magnetic properties return with respective temperature.
Adire...