That is the whole problem right there.
IDK if you remember but back in the Athlon XP days AMD were not beating Intel at the high end. They were spanking them on up to mid range and with what most people would buy but Intel had their Xeons up their sleeve.
So AMD decided they would take on Intel right up to the high end. It nearly sent them out of business. Why? because no matter how good the AMD CPUs were (and they were, spectacular even) people simply would not pay Intel prices for them. Which if you are competing you have costs for and they're the same no matter who you are.
You can't expect AMD to be able to make GPUs that compete with, or beat, Nvidia ones without them coming in at a cost. AMD's last attempt would have cost far more than the GPU it was competing with (the 980TI, I'm talking about Fury X) but they could only charge £530 for it. Which is a lot of money, but they probably had far less profit than Nvidia due to HBM and a liquid cooler..
If you really, truly want them to compete then you need to understand it will cost. Well, unless AMD can break the laws of physics and economy and so on.
It's possible I didn't explain myself properly.
I was disappointed with the Fury X. The price was the absolute max I'd be willing to pay for a GPU, but the performance and design was not what I was looking for. So when I say 'I don't want another Fury X', I mean I don't want to pay €700 for a GPU that does not match a €700 graphics card from a competitor. The 980ti beat the Fury X in almost every respect except for DX12, which was a year away from being introduced. The Fury X closed the gap at higher resolutions, which is why I chose the Fury (non X) over the 980 and I'm glad I did, but it was just not good enough.
I have no problem AMD charging €750+ for a GPU that actually is a beast. I just won't buy it. They released the 295X2 and ProDuo and I had very little interest in them; I just couldn't afford them. I could have bought a 295X2 when they were around £500, but I would have been forced to change power supplies and purchase all new custom cables. I had no preconceived notion that AMD were in some way inferior or did not deserve that much money. I just couldn't afford it.
This is purely speaking from personal preference, I want a €600-650 GPU that handily takes on the 1070 in DX11 and handily takes on the 1080 in DX12, all with a TDP of around 200-225W. That's basically a refresh of the 290 and Fury with a lower TDP. If that's too much to ask for, that's too much to ask for. If AMD want to make a €800 behemoth that puts a huge smile across our faces, wonderful, but I won't buy it. It's not because I won't pay that much for an AMD card; it's because I won't pay that much for a graphics card. I wouldn't even dream of paying what nVidia are asking for the 1080. It's under powered for what they're asking, IMO, especially when I know that a 1080ti could come out that offers a better price to performance ratio.
If AMD are really aiming to make Polaris the more affordable option that maintains healthy competition—and that is exactly what it is doing—then why can't they do that with Vega? Does it have to beat nVidia? Not in my opinion. The RX480 didn't have to beat the 1060. All the card had to do was fulfill a particular set of needs. Getting back to Vega, When I say things like, 'matching 1070 in DX11/matching 1080 in DX12', it is purely as a point of reference. If it's behind, so be it. If it is ahead, so be it. Just make sure the price is reflective of that, as much as possible. Maybe my point of reference as far as prices is out of date and skewed, but it's what I hope for.
The 7970 was $550 and the 680 was $500. The 680 was, at first, the faster card and thus better value. A few months later that changed. That extra $50 on the AMD side bought you something—longevity. A win for AMD.
The 290X was $150 cheaper than the 780ti, but it wasn't quite as powerful. It also was hotter and more power hungry. That saving was worth it, though, due to the longevity and maturity of the card. Another win for AMD.
The Fury X was $650 and did not offer any tangible benefits over the $650 980ti for another year or unless at really higher resolutions, and even then it was held back by VRAM. A win for nVidia.
I must admit that I know very little about CPU's further than 3 years back and things may be very different there. Sorry for the wall of text. I just felt I may not have been understood fully. I want AMD to make an €800 GPU that is absolutely phenomenal. I won't be buying it, though. I'll be buying their lower end GPU. I'm hoping it'll be their 1080 competitor, but it may have to be the 1070 competitor as, like I said, my perspective of prices may be out of date.