Would you post up a picture so we can see what is happening?
Depending on the damage, I sort of find this to be ridiculous on Toyota's part. I had a trim piece issue with my BMW while under warranty. There's a rubberized finish around the edge of the nav screen that started to flake up/off. I thought this was going to be cut and dry but the dealer wanted the service manager to take a look at it. He did and authorized the repair. I didn't know why until my service advisor told me the plastic housing could not be purchased separately. The entire nav screen needed to be replaced to the tune of $1000 list. So I now understood why the service manager got involved.
Granted BMW is in a different market segment than Toyota, we're not talking about a $1000 item here. I'd push back and if you don't get any satisfaction from the dealer, I'd call Toyota's customer service line.