With the latest version, when I push the button, the lights will go OFF, and then a few seconds later they come back ON.
I’ve noticed that this happens if something else changes the state of the lights sometime during the day, before the button is used. I didn’t think this would matter, since it’s looking at the state of the light before taking action. I can test this now, by telling Alexa to turn these lights on, and then trying to use the button to turn them off. Again, lights go off, a few seconds later they come back on. Or vice-versa… Lights will come on, a few seconds later they go off.