The phyical led read was not returing the logical state of the LED. Since they are active low, when on it was returning 0 and the the negation operation in togggle was makeing that a 1 which it already was.