Shogun 2 CPU Benchmarking (formerly "3570k CPU Bottleneck?" joke title)

[Master&Puppet]

Update 2

Update 3: Big Battles

Just joking!

The purpose of this thread is to give plenty of firepower to the other side of the argument - just how much CPU power do you need for gaming?

I've been around the shops and online stores and have seen that what sells in terms of a "gaming pc" really is a poor way to spend your money. Instead of getting a half decent cpu with a decent gpu, 95% of the pc market is dominated by PCs which are sold on the power of the cpu alone. This is often totally irrelevant to its gaming performance (and don't get me started on the "AMD 8-core!" ads). i7s with a 560ti? I think not. Most of us know this but the argument rages on so I thought over the coming weeks I'd bring the OC3D community some firepower to back it up.

Today we start with the Shogun 2 Direct x 11 1080p high settings benchmark. Due to the nature of the game, with large numbers of units and models on the battlefield, this is a good test for any computer with a decent balance between CPU and GPU power.

The bits of my rig you need to be aware of:

CPU: 3570k. Set with turbo off and the multiplier changed for each run to select 1.6, 2.0, 3.0 & 4.0 GHz. (The multiplier was literally the only thing I touched between each run).

GPU: 2x Sapphire 7950s in crossfire set at stock.

Most people would consider this a pretty decent gaming setup with plenty of GPU power and with which I expect to get some CPU bottlenecks. The advantage of having more GPU power for this kind of test is that it allows me to get some better range on the CPU clocks before I hit the GPU wall.

FPS recorded with Fraps and I also had MSI AB running so I could observe the GPU load.

So without more ado here are some graphs:

Pretty intriguing

You can see from the outset that the cards are forcing a bottleneck on the tied down CPU.

Things get interesting between the 3 & 4GHz tests though. In quite a few sections you can see that the lines meet and that's where we are hitting the limits of the twin GPUs. However that extra 1GHz on the cpu really pulls up the minimum FPS in a number of places showing that a twin GPU and mildly overclocked i5 setup is working in good balance.

Perhaps the most interesting result is that apparently an i5 running at 1.6GHz is enough to get some some very playable benchmark fps. Minimum and average FPS of 29 and 64.5 respectively is pretty much bang on for what many consider to be seamless game play.

More to come...

Thoughts and ideas for more? Please comment!

M&P

 

[Big WigglyStyle]

Nice job interesting to see how much clock speeds can adjust performance in gaming.

Here is something similar but instead of clock speeds it's active cores.

[do not post links to other reviewers on this site - or you will feel my wrath]htt p://w ww.yout ube.com/watch?v=I4PDoy-mi0A&feature=channel&list=UL[/blahmedia]

 

[SieB]

Good god, Linus spends more time marketing and talking about Twitter than talking about what he is making a vid about and doesn't even go in to detail. :\

As for the bottlenecking, it seem that with a 3570k at over 3ghz there is enough power, only when you drop bellow that is where you get bottlenecking or a drop in performance.

 

[VengeanceUK]

thats why ttl's reviews are 1000 times better, no sponsor waffling

 

[jamesriley94]

I may try something similar at some point. It does give food for thought.

I have a 3770k, and I think it would also be interesting to see if there is any difference whatsoever with hyperthreading being on or off, even if it just being 1 or 2 frames per second.

I'd also quite like to see any increase after the 4ghz you tested. As I'm sure most of us here are running our CPUs at around 4.5ghz, I think it would definitely be interesting to see if the increase in temperature and voltages can really be by any means justified in any kind of performance, which judging by your graphs, probably not. Still be interesting to find out though

 

[dukeReinhardt]

lol@dicksucking

this is pretty fascinating though. i didn't think games of today were using that much processing power. and to learn that overclocking a 3570k leads to performance gains is a surprise (but then again i am new to pc building).

thanks for the test.

 

[jamesriley94]

lol@dicksucking

this is pretty fascinating though. i didn't think games of today were using that much processing power. and to learn that overclocking a 3570k leads to performance gains is a surprise (but then again i am new to pc building).

thanks for the test.

I think the point was more so that it doesnt lead to much gains, considering the performance difference between the stock clock of 3.4ghz and the OC 4.0ghz will be next to nothing.

 

[nepas]

Saw thread title.....

Came into thread ready to rage like hell,sees first line....

Damm you

Nice little test though,and some good results.

 

[Master&Puppet]

Saw thread title.....

Came into thread ready to rage like hell,sees first line....

Damm you

Haha nepas

I knew it would get one or two people!

I'd also quite like to see any increase after the 4ghz you tested. As I'm sure most of us here are running our CPUs at around 4.5ghz, I think it would definitely be interesting to see if the increase in temperature and voltages can really be by any means justified in any kind of performance, which judging by your graphs, probably not. Still be interesting to find out though

Yea I was planning on taking the GHz higher but I haven't actually put my water cooling on yet so it's running on the stock intel cooler!

I have a 3770k, and I think it would also be interesting to see if there is any difference whatsoever with hyperthreading being on or off, even if it just being 1 or 2 frames per second.

I did a thread ages ago on 4 core vs 6 core gaming which had some pretty strange results. Some games took advantage some didn't. We never did get any results from HT (I was using a 1100t). In some cases you will see slightly better, or even noticeably better fps using more threads, sometimes it won't make any difference at all because what I've experienced it's very game dependent. The key thing to point out though is that you are better off spending the extra dosh on the GPU than you are on getting the HT because you are always more likely to max out the GPU than the CPU.

As an example I've just added a track to the experiment. This time we have the CPU again at 4.0GHz but xf disabled so we are now running on one GPU.

You can see that with a single card the FPS is chasing the results of the 1.6 run and only beating the 2GHz run on 3 very brief occasions. If I'd set that run at 2.5GHz then I reckon the CPU would be clear of any bottle necking.

The bottom graph suggests to me that:

The minimum FPS is supported by the CPU - somewhere between 2 and 3GHz is ideal here.

The maximum FPS is limited by the GPU - even the 1.6GHz run is providing enough power.

On average a single 7950 performs about as well as the 1.6GHz run.

Bear in mind that I'm using a 7950 here which is no rubbish card, admittedly this is set at stock so there is better single GPU FPS out there, but I think Sieb has it right here - certainly as far as this benchmark goes you don't need anything above 3GHz. Stock will be more than enough for any single card here.

Ultimately this produces perfect playing conditions with a single 7950 and the CPU set at 1.6GHz.

I want to run some massive battles next (ultra unit size with max allowed units) to see what difference that makes.

M&P

 

[giannakos23]

Wow the scores in the singlegpu tests are very similar to the dual gpu with the cpu running at 1.6ghz !!

 

[Rastalovich]

Ultimately this produces perfect playing conditions with a single 7950 and the CPU set at 1.6GHz.

M&P

Love this type of shizzle - and yes I did the same nepas thing

That 1.6GHz result you got is one of the reasons I don't personally think picking an AMD cpu is a bad idea - price versus the Intel alternative obviously.

People moaned about the AMD performance, and I think they are justified only when looking at the prices vS Intel - but if their prices were that much more competitive, I'd be championing them on every purchasing thread.

That figure you got to of 3ghz, which doesn't increase much at 4ghz, which would probably be a similar increase at 4.5 - these are the similar types of results a man got some time ago with a Q6600 overclocking versus 8800 Ultra performance. Everyone seemed to latch onto it as a bottleneck, and it's not, it's just that improvements can-not get significantly better over a certain cpu overclock.

 

[Master&Puppet]

Right, so big battle results are in:

Test Setup:

I had to invent a tougher benchmark and I just ended up running a number of large battles as benchmarks. Obviously this opens up issues regarding the reputability of each experiment because I cannot guarantee that each battle was the same.

Factors I could control

Rig wise I had an identical approach to the first set of runs however I didn't bother with the 1.6GHz test.

At the beginning of each battle the setup was identical to the others:

Graphics settings were maxed out. This includes the blood splatter DLC mod and the use of fire arrows (and we are talking about a LOT of flaming arrows).

Unit size was set to ultra and so was army size (this allowed for massive armies for challenging the rig).

Both my army and the enemy forces were identical: Shimazu with 1 general, 19 units of archers and 20 units of spearmen.

I setup my army in an identical fashion each time and put myself on the defender's side so that I did not have to move my forces for 95% of the battle.

I didn't move the camera or give orders but just recorded the fps.

Battles were fought on the same map with the same weather conditions.

Factors I couldn't control

The ai fights with varying tactics each battle.

For the roughly the last minute of each fight I did give orders to mop up the remaining enemy forces (I was not willing to keep running the benchmark for another 10 minutes just to let their remaining 3 units of archers run out of archers and finally charge in...!).

I picked this particular set up because I believed that most Shogun 2 players with a decent rig would be running this kind of battle format. Commonly on a campaign you'll come across battles with one army vs another army (and they are unlikely to be this big until the end game). However there is always the chance that one, or both sides, will have a supporting force which will also enter the battle so on balance I thought this would generally set a decent standard for the vast majority of a typical campaign although I accept that battles maybe a lot smaller than these and potentially a lot larger.

Note: I only bench marked using one army per side. The setup screen allows for up to 4 armies per side at which scale I expect would basically defeat any CPU out there! For interests sake my set up had just over 14,000 men on the battlefield at the same time!

Results

That all makes it slightly harder to draw conclusions because, as you can see from the graphs below, the actions on were happening at different times but I will do my best to decipher them!

General Trends

Obviously each battle lasted for differing lengths of time lasting between 5 and 11 mins. However each battle had a main engagement at around the 2-3 minute mark and skirmishing after that with two of the tests also receiving a second medium sized engagement after the enemy reformed. The other 3 tests were decisive at the first engagement. With that in mind I'll talk about the engagements separately to the general battle conditions (i.e. the army manoeuvring and skirmishing) since the engagements clearly put a massive strain on the hardware.

Non-Engagement Trends

CPU: Comparing the crossfire results you can see that, ignoring the engagements, there is a definite advantage in having the extra CPU power with all of the runs performing to produce an adequate and stable FPS however none of them really performing to a level I'd like to see (60+ FPS).

GPU: Comparing the 4GHz tests you can see that there is a significant GPU limitation running with just 1 GPU. Scaling wise we've seen nearly 100% looking back at the experiment done earlier in the thread but during the first 2 minutes of this test we're seeing only about 65% which says to me that the CPU is the limit for 2 cards but more than enough for a single card setup. In fact the 4GHz_1GU run was very similar to the 2GHz_2GPU run which confirms what we saw in the Shogun Benchmark tests earlier where it looked like a 2.5GHz clock was a great match for any single card rig.

Engagement Trends

CPU: The main engagement during each battle was by far the most demanding part of each battle. The 4GHz_2GPU result fell from a fairly hefty and stable late 60s FPS trend to a minimum of 21FPS during the heaviest part of the battle.

There was clear advantage to having a higher clock with the 2GHz result producing literally half that the 4GHz result at the same stage and the 3GHz result sitting perfectly in the middle:

2GHz = 11FPS

3GHz = 15FPS

4GHz = 21FPS

That's as perfect a CPU scaling as I've seen and I'd feel safe extrapolating the result to around 25FPS for a 5GHz clock.

GPU: The main engagement graphs for the 4GHz tests were very similar - with the minimum FPS scores of 21 and 17 looking very close too. I believe that there was absolutely no advantage gained by having a second GPU at all. The 4 point difference seems reasonably explained by the nature of the engagements. The 1GPU run was very short and it seems reasonable to assume that the main engagement was therefore heavier than the 4GHz_2GPU run.

Conclusion

The recommendation made previously that a 2.5GHz on a 3570k for a single card setup holds true for the vast majority of Shogun 2 game play.

However if you intend on running large battles then the CPU is going to take a massive beating and you are going to be wanting the best 24/7 overclock that you can achieve.

Whether it is therefore recommendable to get an i7 (or even a 3930k!) is still a questionable topic for me. When I did my 6core vs 4core comparison I didn't see any improvement in Shogun 2 so frankly I think that's your lot - Shogun 2 is just too demanding if you want to take it to the extremes!

I'm going to do BF3 next in a new thread. Interestingly in the 4core/6core comparison I did see some improvement in having extra cores. Obviously Ivy Bridge is a far more powerful architecture than Phenom II so although BF3 can choose to take advantage of extra threads the key is finding out whether BF3 is capable of filling up 4 IB cores because if it can't then there is still no advantage to getting those i7s (that's exactly what Linus's video suggests above).

Let me know you thoughts on the test and subject, more to come!

M&P

 

[dugdiamond]

Nice job interesting to see how much clock speeds can adjust performance in gaming.

Here is something similar but instead of clock speeds it's active cores.

[do not post links to other reviewers on this site - or you will feel my wrath]htt p://w ww.yout ube.com/watch?v=I4PDoy-mi0A&feature=channel&list=UL[/blahmedia]

nuff said

 

[Rastalovich]

Shogun 2 doesn't surprise me too much. I play TW games too much - I've lost years of my life to campaigns. I don't personally use the massive unit sizes as it's messy. 1 core engine(s) afaic.

 

[KING_OF_SAND]

The one thing that I noticed with CPU overclocks and gaming is you min FPS is higher and more stable.

 

[Rastalovich]

You can see that with a single card the FPS is chasing the results of the 1.6 run and only beating the 2GHz run on 3 very brief occasions. If I'd set that run at 2.5GHz then I reckon the CPU would be clear of any bottle necking.

The bottom graph suggests to me that:

The minimum FPS is supported by the CPU - somewhere between 2 and 3GHz is ideal here.

The maximum FPS is limited by the GPU - even the 1.6GHz run is providing enough power.

On average a single 7950 performs about as well as the 1.6GHz run.

You can look at this another way - which I wouldn't believe or would suggest to anyone - 2G cpu & 2 gpu - slower than 4G cpu & 1 gpu. Almost suggests that 2G of the cpu clock is worth a gpu, which doesn't make sense.

 

[Master&Puppet]

You can look at this another way - which I wouldn't believe or would suggest to anyone - 2G cpu & 2 gpu - slower than 4G cpu & 1 gpu. Almost suggests that 2G of the cpu clock is worth a gpu, which doesn't make sense.

I thought that's exactly what was going on though?

Taking a look at the FPS line graph the 2GHz run and single GPU run gets similar results. So surely 2GHz was worth 1 GPU?

 

[Rastalovich]

It looks like it doesn't it. But thinking about it, in old money as it's easier, it's almost like having a Q6600 @ stock.. 2.4 or 2.66 iirc, and oc'ing it to 4.4 or something.

Now if a person with a stock Q6600, and a nice x48 mobo, and a 7950 came to you asking about fps. Would your immediate thought be oc'ing the Q6600 or buying another 7950 - which would be the better option for fps. Oc'ing the thing to 4g+ I'd not dream of being the equiv or better than adding another 7950.

For arguments sake, I'd not expect a well overclocked Q6600, x48 and decent ddr2/3, to cause issues.

 

[Master&Puppet]

Assuming linear relationships for a moment:

2GHz CPU = 60 FPS

1GPU = 60FPS

Gameplay = 60FPS

Since the game will always perform at pace of the weakest link in either of the following situations:

4GHz = 120FPS

1GPU = 60FPS

or

2GHz = 60FPS

2GPU = 120FPS

Gameplay remains at 60FPS

Only with 4GHz and 2GPU will there be any improvement. So in your example I'd have to advise them to do both, doing one will have no impact on performance. Is that what you mean?

 

[Rastalovich]

No, don't look too deeply into it, it's merely an observation of the graph, where you see the right hand side purple bar (1 GPU) above the blue on the left (2 GPU).

As opposed to buying a 2nd gpu, you could just oc the cpu, as a suggestion to someone - that's a quick verdict which I'd not go along with.

It's just an off-the-cuff comment looking at the results merely pictorially