One downside is that it gives the government powers that it does not need. Another is that it extends people's fear in that some people will continue to believe that coronavirus is a deadly dangerous wide-ranging threat, when it currently isn't. Another is that if the mask compulsion is to be released this year, your proposal would release it when respiratory infections are rising as winter approaches rather than now, when there is a lot of summer still to come. Another is that if masks are mandated for public transport into the next school year then it will encourage that fool Williamson to continue to disrupt and destroy children's education on the flimsiest of pretexts. Another is the continued mental health issues of people being told again and again that their freedom is coming and being repeatedly told that it is not coming after all.
I can and do see the reason for stopping wearing masks now, and I can see why some people want to keep masks for ever. I'll reverse your question. What is the benefit of extending the mask mandate so that under-35's can be fully vaccinated? There are on average 176 people per day dying of respiratory diseases, and your proposal will protect approximately 0.1 of them. Is the cost of compulsory mask wearing worth the very marginal short-term increase in safety for under 35's?
I think my postie would be glad to see the end of masks. She's completely deaf, and relies on lip-reading.