PDA

View Full Version : The Objective Truth. An experiment.



Chapters Unwritten
19-11-2012, 17:33
So there is a lot of hyperbole and random quantifiable statements that get thrown around a lot in this game. One such statement is about thunder wolves, and how undeniably awesome they are. I'd like to do an experiment where several people use the same unit and record its performance over. series of games and report their findings. Do you all think this idea could be interesting?

I'm just every weary of every discussion being based on hyperbole, anecdotes, and "vacuumathematic" analysis. So I figured my h npol ular opinion that thunderwolves aren't actually that good might be a good starting point. What do you all think - waste of time or interesting bit of research?

Sent from my SCH-I535 using Tapatalk 2

IcedCrow
19-11-2012, 17:40
I find it good that you are willing to exercise your critical thinking skills and not just bow down to "because the internet says so" which is the majority of logic people on the internet wield.

Pertaining to warhammer, what is good or not good is dependent on many factors, which are hardly ever taken into account:

* your meta
* your meta's playstyle (ultra competitive spam lists, more casual varied lists, a bit in between, other)
* whether or not you use missions and scenarios
* whether or not you use the rules as written or houserule
* whether or not you feel that if something is not ultra effective / optimal that that determines if something is good or not good

I think that the biggest bullet point is the meta and meta playstyle. If everyone had to have that as part of their signature / profile that identified them as "competitive, casual, etc" that that would help to understand why they feel it is "bad" or "good".

A competitive only spam player is going to have a vastly different opinion over a casual campaign player.

Chapters Unwritten
19-11-2012, 18:05
There would definitely need to be some initial guidelines to deal with a lot of them. First and foremost, the unit in question would have to be used in the same loadout by everyone involved, at the same points level. Their opponents should be well versed on rules and any rules issues that arise should be looked up instead of glazed over. The missions could even always be the same, just to test. The tough part would be local meta, but a description of it and of the opposing player as perceived by their groups could help us quantify that a little bit, or at least be aware of it.

We'll never get fully objective data, but we will see correlation between all the battles for sure, and we might be able to so a more patent truth in things.

Navar
19-11-2012, 18:12
I think that local meta and play-style are likely the defining factors.

I remember a while back I had almost this exact exchange here on Warseer:

Them: Unit X is so awesome. Here is an example army list based around unit X.
Me: That list looks horrible. How does it ever hold a single objective?
Them: Objective?

stroller
19-11-2012, 18:19
I applaud the initial intent. However, I'm not sure it's achievable. A counter example to illustrate: I like nids. I almost always stash 3 biovores with the cheapest synapse creature I can find at the back of the board to pop spore mines as often as I can. The sensible response to this is just to ignore the mines and take the odd hit. Instead, my regular opponents shoot at the mines (stopping to do so) or go into movement gyrations to avoid them, thereby slowing their advance. I've played mind games with them, and they fear the perception of the mines more than the actual likely damage. The mind games are the key, and overrule the objective stats. Spore mines and biovores are, undeniably, IN THAT PARTICULAR CIRCUMSTANCE, awesome. Your mileage will almost certainly vary.

Chapters Unwritten
19-11-2012, 18:54
While that's definitely true, I feel like the players involved would ideally be sharp enough to document that, i.e. My thunderwolves died on turn one because the enemy perceived them to be a huge threat and overcompensated firing at them.

We could even have objective criteria for them, such as recording the number of successful wounds inflicted, recording their assaults, how they died, all of their actions. This would have to be all coded, of course, but it could be theoretically at least partially understood.

Charistoph
19-11-2012, 19:44
Everyone knows the internet is only good at 2 things: (relatively) anonymous hyperbole and porn. Porn isn't really allowed much on this site, so that leaves...

As one of the guys at my LGS says, mathhammer's fine until the dice are thrown. No matter how 'well' you build a list, 1's will always miss/fail to Wound/fail to Save.

unknown_lifeform
19-11-2012, 19:47
IcedCrow has given a number of factors that will influence how effective a unit is. There are others he hasn't listed - like the ability of the player (some units are easy to use, others only shine in the hands of a good player, if a player is especially bad no unit will shine etc.) and what is in the rest of the army (I find tyranid raveners to be extremely effective when in a list which also contains a lot of other fast moving units and warrior sized models, on their own they tend to get focused down).

There are a massive number of factors you need to control for. If you eliminate all of these you end up with a sterile, abstract set of data which means nothing because battles don't take place in such an environment - and you could have got there a lot easier by just math hammering. Also, how exactly do you measure performance? How many points of enemy something kills? What about the unit that kills no enemy but absorbs a massive amount of firepower? Or the unit that kills nothing but so worries your opponent they go out of their way to avoid it? Or that does nothing except cost hardly any points and sits on an objective all game, freeing up other units to fight?

If this were easy GW codexes would be balanced :) Ultimately your best guide is personal experience and the experiences of those in your group - that alone will give you an indication of how good a unit is for you. So don't rely on mathhammer and don't treat internet wisdom as gospel. Be prepared to experiment and think how to get the most from each unit. I think the ability to do this for yourself is what separates a good player from one who just netlists in the hope of an easy route to wins.

Freman Bloodglaive
19-11-2012, 19:49
Everyone knows the internet is only good at 2 things: (relatively) anonymous hyperbole and porn. Porn isn't really allowed much on this site, so that leaves...

As one of the guys at my LGS says, mathhammer's fine until the dice are thrown. No matter how 'well' you build a list, 1's will always miss/fail to Wound/fail to Save.

Mathhammer is about biasing your rolls towards a successful outcome. Sure, sometimes the extremes of the bell curve will screw you over, but if your mean is higher than your opponents then all else being equal you'll win.

IcedCrow
19-11-2012, 20:00
Mathhammer is about biasing your rolls towards a successful outcome. Sure, sometimes the extremes of the bell curve will screw you over, but if your mean is higher than your opponents then all else being equal you'll win.

Which is why spam is so popular at tournaments. The theory of spam being: if element X sits on the top of the bell curve, then take as many of element X as you can.

Which goes back to imbalanced codices.

Which goes back to the meta that one plays in :D

Chapters Unwritten
19-11-2012, 20:41
I don't really feel the game is imbalanced as much as I feel people do not really use all the tools given to them, honestly. That is a part of why I looked into Thunderwolves; despite the alleged popularity of the Space Wolves I find myself losing rather handily lately.

I found them to be quite wanting. No transport means an easy target, the unit has such a huge footprint on the board that it obstructs half your army and rarely can move without being hindered, and keeping their costs down is practically suicide as everything that shoots them is high strength (and often also low AP). What they did hit, they hardly damaged thanks to armor saves; I would take 30 Grey Hunter attacks over the comparably costed 15 Thunderwolf attacks any day. The Lord did better, but being AP3 instead of AP2 hurt his frostblade (and my usual HQ, a cheapo Lord with Frost Blade and SS...he will never be the same again :( ).

I realize there would be a lot to account for but at the very least such a study would be closer to objective reality than anecdotes or hyperbolic math.

Nurgling Chieftain
19-11-2012, 20:51
Thunderwolves took a hit in 6th edition, where it's more difficult to spread wounds around. My Dark Eldar never feared them much in the first place, but that's more of an interaction with poison. Even so, they were always bolter bait; it seemed to me that their primary cache was just how few bolters were around in late 5th edition.

Theocracity
19-11-2012, 21:50
As IcedCrow notes, the meta had a huge effect on what actually works for someone. The problem is that the Internet is not a very efficient environment for discussing the intricate particulars of each person's meta. Therefore the baseline of Internet Wisdom sits somewhere between the meta of the largest standardized tournaments and an Excel spreadsheet.

It's efficient to discuss in text and on some level true - but it's not necessarily relevant to everyone.

Denny
20-11-2012, 10:58
IMO No matter how many opinions are offered on a subject the sum of their knowledge will never be an objective truth. Objective truths (in theory) are demonstrable truths that exist independent of thought. As long as you are measuring a subjective quality (like ‘usefulness’) you cannot come to an objective truth.

. . . unless it’s about Grey Knights who are objectively lame. Fact. :shifty:

RandomThoughts
20-11-2012, 12:09
So there is a lot of hyperbole and random quantifiable statements that get thrown around a lot in this game. One such statement is about thunder wolves, and how undeniably awesome they are. I'd like to do an experiment where several people use the same unit and record its performance over. series of games and report their findings. Do you all think this idea could be interesting?

I'm just every weary of every discussion being based on hyperbole, anecdotes, and "vacuumathematic" analysis. So I figured my h npol ular opinion that thunderwolves aren't actually that good might be a good starting point. What do you all think - waste of time or interesting bit of research?

Nice idea, examining the value of a unit through experiments under controlled conditions, very scientific. Except the controlled conditions part would be pretty much impossible, unless you also take account for what list the unis is being used in, player skill, enemy lists, etc.

By the way, before you diminuish mathhammer, if the math doesn't cover what you're after, you most likely missed something. If your Pathfinders perform worse than the math suggests they did, you might be either rolling badly (which can be documented easily enough), or you missed something in your math. Like, say they always die to flamers, which denies the 2+ cover you took for granted. Adjusting the formula accordingly, you suddenly realize that the math actually works out.


I applaud the initial intent. However, I'm not sure it's achievable. A counter example to illustrate: I like nids. I almost always stash 3 biovores with the cheapest synapse creature I can find at the back of the board to pop spore mines as often as I can. The sensible response to this is just to ignore the mines and take the odd hit. Instead, my regular opponents shoot at the mines (stopping to do so) or go into movement gyrations to avoid them, thereby slowing their advance. I've played mind games with them, and they fear the perception of the mines more than the actual likely damage. The mind games are the key, and overrule the objective stats. Spore mines and biovores are, undeniably, IN THAT PARTICULAR CIRCUMSTANCE, awesome. Your mileage will almost certainly vary.

Kind of reminds me of Poker. First you figure out the math, then in the rare cases that math doesn't answer your question, you start mind games with your opponent.


Everyone knows the internet is only good at 2 things: (relatively) anonymous hyperbole and porn. Porn isn't really allowed much on this site, so that leaves...

As one of the guys at my LGS says, mathhammer's fine until the dice are thrown. No matter how 'well' you build a list, 1's will always miss/fail to Wound/fail to Save.

Not really. It doesn't matter whether your experience tells you that unit X should be able to absolutely dominate unit Y on range or whether your math tells you that unit X should be able to absolutely dominate unit Y on range. In both cases, the dice can screw you over. However, relying on math and experience is usually better than just relying on just one of them.


Which is why spam is so popular at tournaments. The theory of spam being: if element X sits on the top of the bell curve, then take as many of element X as you can.

Which goes back to imbalanced codices.

Which goes back to the meta that one plays in :D

Which is one of the reasons I love Warmachine. There are a lot of different match-ups with a lot of hard counters. You spam one unit because it is awesome, you run into one of the better counters to the unit, you take a hard dive. It's a bit like rock-paper-scissors, I'll admit that, but it encourages diversity and combined arms.


IMO No matter how many opinions are offered on a subject the sum of their knowledge will never be an objective truth. Objective truths (in theory) are demonstrable truths that exist independent of thought. As long as you are measuring a subjective quality (like ‘usefulness’) you cannot come to an objective truth.

. . . unless it’s about Grey Knights who are objectively lame. Fact. :shifty:

Actually, there are objective truths in tabletop games. The ones that can be backed up and proven with math. Except that in a complex game, you can never be entirely sure you didn't miss a vital variable or that your results are even relevant.

RobPro
20-11-2012, 12:26
You would need everyone using the same list at the same points level to really see how a unit performs in synergy with everything else.

skorczeny
20-11-2012, 12:40
If you really want to conduct an experiment - without relying on reported stories of their success...

There was an experiment in White Dwarf many many years ago - it compared the effectiveness of chariots from each of the WHFB armies. This was done by taking each chariot, starting at certain distance away from a unit of Empire Spearmen, and charging. The did this several times for each chariot, and recorded the results in a spreadsheet. THEN, the chariot was lined up against archers, who had a turn of shooting on the chariot... etc etc. The beastmem chariot won, because it was good at charging and breaking the spearmen, and good at surviving the volly of arrows to reach the enemy targets intact.

Anyway - try doing this with the thunder wolves. I don't know how thunder wolves work, but take ... 3 or 5 of them, with out upgrades (or with power weapon for sarg or whatever). Line them up against a tactical marine unit in open field at say 20 inches away. They have the first turn. Charge them into the tacs repeatedly, recording the results each time (# of hits scored, # wounds taken, #models killed, etc).

You'll need something to compare against the thunder wolves - all I can suggest is an equally priced dedicated assault unit that has either jump, cavalry, or bike sup-type. SM bikes, assault marines, warp talons, IG rough riders... you get the idea. Record the results and compare.

You could do this experiment with cardboard cut-outs.
Or pen and paper and dice.
Or just a calculator if you're good a math (I'm not).
Or this is an awful idea.

Denny
20-11-2012, 13:24
Actually, there are objective truths in tabletop games. The ones that can be backed up and proven with math. Except that in a complex game, you can never be entirely sure you didn't miss a vital variable or that your results are even relevant.

There are certainly truths in terms of maths, but you cannot mathematically prove the subjective value of a unit because it is exactly that; subjective.

Knowing the WS of a Grey Hunter is fine, but the number is meaningless in isolation. As soon as we take it out of isolation by comparing it to another value we are making a subjective assessment of a probable opposition. It is at this point no longer objective but based on a perceived likelihood.

I think we’re pretty much coming from the same place. I agree that in theory one could use maths to calculate the ‘value’ of a unit, but would need to average out the opposition, the deployment, the terrain, player skill, meta, random scenery/warlord traits, throw in a bit of Rational Choice theory, Game Theory . . . and at this point the probabilities are going to be too unreliable to be of any use.

Besides, I think Objective Truth is overrated. My subjective truth may not work for other players, but if it works in my player group then I’d be an idiot to change it based on an objective truth that may have no bearing on my experiences.

Chapters Unwritten
20-11-2012, 13:50
Well, I'm more looking for the baseline so that uncorroborated hyperbole isn't quite so easy to come by. I think you could get some useful data out of it, it wouldn't be 100% by any means but it would be nice to see. The "Of X times this unit engaged Unit Y under Z conditions, it averaged out as [n]" type of idea doesn't really help us; a unit doesn't fight one other unit, it fights another army, and realistically all that is, is setting up the usual mathhammer assumptions on a table and letting them play out.

RandomThoughts
20-11-2012, 14:07
If you really want to conduct an experiment - without relying on reported stories of their success...

There was an experiment in White Dwarf many many years ago - it compared the effectiveness of chariots from each of the WHFB armies. This was done by taking each chariot, starting at certain distance away from a unit of Empire Spearmen, and charging. The did this several times for each chariot, and recorded the results in a spreadsheet. THEN, the chariot was lined up against archers, who had a turn of shooting on the chariot... etc etc. The beastmem chariot won, because it was good at charging and breaking the spearmen, and good at surviving the volly of arrows to reach the enemy targets intact.

Anyway - try doing this with the thunder wolves. I don't know how thunder wolves work, but take ... 3 or 5 of them, with out upgrades (or with power weapon for sarg or whatever). Line them up against a tactical marine unit in open field at say 20 inches away. They have the first turn. Charge them into the tacs repeatedly, recording the results each time (# of hits scored, # wounds taken, #models killed, etc).

You'll need something to compare against the thunder wolves - all I can suggest is an equally priced dedicated assault unit that has either jump, cavalry, or bike sup-type. SM bikes, assault marines, warp talons, IG rough riders... you get the idea. Record the results and compare.

You could do this experiment with cardboard cut-outs.
Or pen and paper and dice.
Or just a calculator if you're good a math (I'm not).
Or this is an awful idea.

At which point you start doing mathhammer, which will tell you exactly how a unit performs in a certain situation, and how well it performs compared to another unit in the same situation.


There are certainly truths in terms of maths, but you cannot mathematically prove the subjective value of a unit because it is exactly that; subjective.

Knowing the WS of a Grey Hunter is fine, but the number is meaningless in isolation. As soon as we take it out of isolation by comparing it to another value we are making a subjective assessment of a probable opposition. It is at this point no longer objective but based on a perceived likelihood.

I think we’re pretty much coming from the same place. I agree that in theory one could use maths to calculate the ‘value’ of a unit, but would need to average out the opposition, the deployment, the terrain, player skill, meta, random scenery/warlord traits, throw in a bit of Rational Choice theory, Game Theory . . . and at this point the probabilities are going to be too unreliable to be of any use.

Besides, I think Objective Truth is overrated. My subjective truth may not work for other players, but if it works in my player group then I’d be an idiot to change it based on an objective truth that may have no bearing on my experiences.

I think I agree. There are things that can be solved with math, like whether a unit effectively performs a certain task you want it for, say the effectiveness of different weapons as tank hunters, or if a unit can be used to successfully tie down an isolated Ork Boss Mob, but the moment you start using units in an unpredictable game, the mathgoes out the window.

I guess what yould be used at that moment are statistics like in Football. Nobody so far could explain to me why a team is bad at third down conversions even though it runs and passes well on first and second downs, for example, but if the stats hold up and the team continues to be bad at third downs after the sample was examined, then I guess the stat means samething and isn't just a random fluke.

I once had this weird experiment in mind. In a hypothetical situation, every single possible Magic the Gathering deck is played against every other possible deck a fixed number of times. Afterwards, the total win tally is messured up to determine how effective each deck and in extension how valuable each card is.

Peevious experiments of fimilar nature tell us that some of the winning decks will be absolutely impossible to explain, appearing as nothing more than a series of lucky draws and good combinations of the right cards at the right moment, and still the deck must be powerful if it held it's own against the best decks ever designes by humans (which would also be part of the experiment). Of course, they might also just be really good against a lot of bad decks that never get played in real life, or perhaps the good decks simply have hard counters that nobody knows about. Statistics and phase spaces are awesome, if you think about it. :)

skorczeny
20-11-2012, 14:34
Okay. Overall, I think it would be fun and interesting to use thunderwolves in a series of games and record how well they performed for qualitative analysis, rather than quantitative.

Are you suggesting going to the internet and asking people to provide this data? Probably a waste of time. Better off asking select people who you know to be reliable (and also use thunderwolves).

Chapters Unwritten
20-11-2012, 16:17
I would have to do it with other locals. Problem there is there are none of us; we've all already let that ship sail due to finding them sucky. Heh. I would NEVER dream of having it done through the internet unless maybe the entire game was video taped, unedited, so that no personal opinion could sway the reports (also be a good way to see mistakes in gameplay...)

The_Klobb_Maniac
20-11-2012, 16:54
They're decent; but I think they're overrated if you've found that much positiivity on here for them. What they have going for them is reliably long charge ranges (roughly 21"[almost] regardless of terrain), a high number of attacks, rending, etc..

The biggest gripe about them originally (at least for me) was true T5 being completely unjustified IMO, having Warboss stats + better wargear + better unit type + rending for less points and being entire squads of them, and just being lame in general.

You're unlikely to get gobs of people to participate. They're actual impact on game also depends on if they're focused down or not; the opponent's prowess, luck of the dice, etc.. Things boil down to army lists, point-for-point effectiveness and similar on the internet because there are so many difficult factors to measure. The difference between the good and bad advice on here is usually experience vs. parroting or deduction. Also, one has issues truly measuring effectiveness if they can only get X games in. I'm still on the fence with defilers; they're probably enemy-dependent; but the only game in 6th I've gotten to test them out (with the new rules) they were getting waxed too easily. The ruins were easy enough to hide behind that the invul didn't matter, daemonforge doesn't help them much, and mine exploded rather than get whittled down. YMMV, and certainly it'd take several games to figure out whether their survivability comes up enough against various opponents to matter. Most would argue no.