First of all at the tail of the IQ distribution, the normal approximation breaks down. Secondly, we get a big jump in mental development when we hit our growth spurts. So IQ at that age is not massively well-correlated with your IQ a few years later.
But that doesn't get into why it is actively detrimental. It turns out that children are motivated to seek confirmation of what they are complimented on. So a child who is told they are smart seeks easy tasks where they look smart, and tends to avoid hard tasks that might show them not to be so smart. While children who are complimented on their hard work and willingness to learn will seek out challenges to push themselves.
Research indicates that this effect sets in shockingly easily. And the two groups respond very differently to setbacks. The ones who are complimented on what they are gets demotivated. The ones who are complimented on what they do get motivated to tackle it again.
See https://www.psychologytoday.com/us/blog/the-power-prime/2009... for more on this and some verification that I'm not just making this up.
Your personal life is an anecdotal data point, but it fits perfectly. You were identified young as smart. It was drilled in enough that you know exactly how smart. But it sounds like you learned to coast on brains and avoid challenges, and are now suffering the consequences.
If this is true, I would recommend that you read https://www.amazon.com/Cant-Hurt-Me-Master-Your-ebook/dp/B07... for advice on how to build mental toughness. That can help you overcome life challenges which are in your way.
[1] https://www.amazon.com/Cant-Hurt-Me-Master-Your/dp/154451228...