TOPIC: Final 2023 treasure stats - year in review

Marc D wrote: Two more things to note about a "best pulls" tracker:
1. Google Sheets doesn't make it easy to get stats like percentage of parent row in a pivot, so it'd either be doing that work manually or taking it into Excel
2. There's a large disparity between the largest and smallest quantity entered (3756 and 5). That's enough to easily skew the numbers on who had the "best luck", because one good pull will push someone way up.

And sure enough, when I take the data into Excel and do % of Parent Row, people with fewer pulls are overly represented in the good luck/bad luck dichotomy. And the person with the most pulls skews very, very close to the overall averages.

So, seems like "best luck" is more a measure of "who got the best tokens with the fewest pulls"

It's pretty amazing that one person had almost 4000 pulls. Maybe that's someone that doing the pulls for the whole group.

It’s pretty amazing that one person had almost 4000 pulls. Maybe that's someone that doing the pulls for the whole group.

That’s not someone doing the pulls for a whole group. It’s someone questioning his life choices after seeing the total.

Mike Steele wrote:

Marc D wrote: Two more things to note about a "best pulls" tracker:
1. Google Sheets doesn't make it easy to get stats like percentage of parent row in a pivot, so it'd either be doing that work manually or taking it into Excel
2. There's a large disparity between the largest and smallest quantity entered (3756 and 5). That's enough to easily skew the numbers on who had the "best luck", because one good pull will push someone way up.

And sure enough, when I take the data into Excel and do % of Parent Row, people with fewer pulls are overly represented in the good luck/bad luck dichotomy. And the person with the most pulls skews very, very close to the overall averages.

So, seems like "best luck" is more a measure of "who got the best tokens with the fewest pulls"

It's pretty amazing that one person had almost 4000 pulls. Maybe that's someone that doing the pulls for the whole group.

[edit]: Didn't see Manimal's post till after I posted this ...that's an impressive feat!

Wouldn't be surprised. 4,000 pulls is equivalent 160 runs. Last year there were 8 VTD weekends (including 2 Patron runs), Momocon, GENCON, and Gameholecon. My guess is would be hard for one person to get to 4,000 pulls on that schedule. But could include a large number from Maps/Treasure Chips.

I have 2,100 on my tally and it is group treasure, typically 7-8 people running once each VTD and multiple runs at GENCON and GHC. I also turned in 500 pulls from Maps/Treasure Chips last year.

I had over 600 from maps and treasure but cannot recall if I added them to my GC numbers.

EDIT: I did add them, as I have 703 pulls posted.

What I see is that manimal, thebombsquad, and I tied for most Boots of Protection. I just did it in about 1/3 the pulls.

David Autzen wrote: Looks like I posted both as “Dave A” and “David A” along the way last year. I’ll try to be more consistent this year.

No worries!

As an aside, I did discover you could probably do an "all Davids" run

Marc D wrote:

David Autzen wrote: Looks like I posted both as “Dave A” and “David A” along the way last year. I’ll try to be more consistent this year.

No worries!

As an aside, I did discover you could probably do an "all Davids" run

I was in one with several Daves and Davids. Some of the other players were related to Daves or Davids.

Some of us have David as our middle name.

Ok, here's the final bit of analysis. This one was more involved to calculate and I'm not 100% sure I did it all correctly.

One of the things that people discuss after an even is whether or not that event had a "rich mix". A "rich mix" is shorthand for higher than average amounts of good stuff - stuff in that "Other" category.

This is a more complex question than it first seems. It requires looking the average of all events, finding the standard distribution, and seeing how individual events compare.

A calculated "Z Score" indicates how many standard deviations from the mean (average) a particular event was. Things with a Z Score between -1 and 1 fall within 1 standard deviation and would be considered the most common. Between -2 and -1 and 1 and 2 would be next most common, -2 or lower or 2 or higher less common than that, etc.

One thing to note is that some of these categories have a very small standard deviation. That means things will be outside the "normal" range most often just because that range is very very small.

Instead of going through event by event, let's take a look at the categories. We'll start with the good stuff first: Ultra Rare and Better!

Ultra Rare and Better
Average pull percentage: .94%
Standard deviation: .001 (very small)
Z-Score range: -1.34 - 1.58
Commentary: The very small standard deviation makes it harder to evaluate this. Low numbers pulled mean that even a handful of extra pulls will skew the results. So while it may seem that V14 and Gen Con had "rich mixes", the small number of pulls probably mean that this is reporting bias. (I'm not good enough to do margins of error.)



Other - Under Ultra Rare
Average pull percentage: 2.17%
Standard deviation: .003 (very small)
Z-Score range: -1.97 - 1.51
Commentary: Much like Ultra Rare and Better category, this had very few pulls to work with. Any under or over reporting in a category can have big effects due to the small standard deviation. Add in that MomoCon had some of the lowest reporting and it's easy to see why it became the lowest scoring convention here.



Rare
Average pull percentage: 41.32%
Standard deviation: .011 (large)
Z-Score range: -1.22 - 1.78
Commentary: Rares had one of the widest spread of normalized distribution across the various events. GHC, Gen Con and V15 all had very high Z-Scores. I'm not entirely sure why this could be; happy to theorize in the comments with you all! I don't think it's indicative of more Rares in the mix though. It's probably just random.



Uncommons
Average pull percentage: 29.44%
Standard deviation: .012 (large)
Z-Score range: -1.42 - 1.63
Commentary: Uncommons had a distribution that looked very much like Rares. V19 had the highest Z-Score; GHC and V15 very close for the lowest. Again, I wouldn't read into those scores too much. Maybe the mix was different, but I'm not confident enough to say that it was. (Although GHC at the bottom here and near the top of Rares could be a push/pull.)



Monster Trophies
Average pull percentage: 8.18%
Standard deviation: .004 (small)
Z-Score range: -1.60 - 2.02
Commentary: Monster Trophies appear to have high variance in pulls, but keep in mind that the standard deviation is very small. So even a few lucky pulls or having monster trophies over-represented could skew the numbers. And I'm guessing that's what happened with V17: a few lucky people pushed stuff higher. Most events had monster trophy distributions within 1 standard deviation, indicating that this was highly consistent across events.

Keep in mind that this category was explicitly noted for having a lower drop rate this year, and the numbers seem to bear that out.



Stalker Tokens
Average pull percentage: 17.95%
Standard deviation: .008 (medium)
Z-Score range: -1.55 - 1.60
Commentary: Stalker tokens probably had the most characteristic bell curve of any of the other tokens. Of note is that V18 and V19 had the lowest Z Score of any of the events. I'm not sure if this indicates that the Stalker tokens ran a bit lower towards the end of the year, if there was some supplementing of the treasure mix or if it's just random chance. One could make a push/pull argument for more uncommons in V19 as evidence that the mix changed, but neither are a strong enough indicator to really draw that conclusion.



Overall
I don't see evidence that any particular convention mix was "rich" in comparison to others. Almost all categories in all events fell with 1 or 2 standard deviations of the mean. The one item that fell above 2 standard deviations - V17 Monster Trophies - is explainable given how tight the standard deviation is. Same with the one item that almost fell outside the low end of 2 standard deviations - Momo Con Under Ultra Rare. This is such a tight grouping that I'd expect just a few tokens to push something one way or another.

Mike Steele wrote: Some of us have David as our middle name.

Yes, some of us do.
Probably more than one of us.

Working on the long task of getting 2023 loot sorted packed and sent out!

Marc D wrote: Ok, here's the final bit of analysis. This one was more involved to calculate and I'm not 100% sure I did it all correctly.

One of the things that people discuss after an even is whether or not that event had a "rich mix". A "rich mix" is shorthand for higher than average amounts of good stuff - stuff in that "Other" category.

This is a more complex question than it first seems. It requires looking the average of all events, finding the standard distribution, and seeing how individual events compare.

A calculated "Z Score" indicates how many standard deviations from the mean (average) a particular event was. Things with a Z Score between -1 and 1 fall within 1 standard deviation and would be considered the most common. Between -2 and -1 and 1 and 2 would be next most common, -2 or lower or 2 or higher less common than that, etc.

One thing to note is that some of these categories have a very small standard deviation. That means things will be outside the "normal" range most often just because that range is very very small.

Instead of going through event by event, let's take a look at the categories. We'll start with the good stuff first: Ultra Rare and Better!

Ultra Rare and Better
Average pull percentage: .94%
Standard deviation: .001 (very small)
Z-Score range: -1.34 - 1.58
Commentary: The very small standard deviation makes it harder to evaluate this. Low numbers pulled mean that even a handful of extra pulls will skew the results. So while it may seem that V14 and Gen Con had "rich mixes", the small number of pulls probably mean that this is reporting bias. (I'm not good enough to do margins of error.)



Other - Under Ultra Rare
Average pull percentage: 2.17%
Standard deviation: .003 (very small)
Z-Score range: -1.97 - 1.51
Commentary: Much like Ultra Rare and Better category, this had very few pulls to work with. Any under or over reporting in a category can have big effects due to the small standard deviation. Add in that MomoCon had some of the lowest reporting and it's easy to see why it became the lowest scoring convention here.



Rare
Average pull percentage: 41.32%
Standard deviation: .011 (large)
Z-Score range: -1.22 - 1.78
Commentary: Rares had one of the widest spread of normalized distribution across the various events. GHC, Gen Con and V15 all had very high Z-Scores. I'm not entirely sure why this could be; happy to theorize in the comments with you all! I don't think it's indicative of more Rares in the mix though. It's probably just random.



Uncommons
Average pull percentage: 29.44%
Standard deviation: .012 (large)
Z-Score range: -1.42 - 1.63
Commentary: Uncommons had a distribution that looked very much like Rares. V19 had the highest Z-Score; GHC and V15 very close for the lowest. Again, I wouldn't read into those scores too much. Maybe the mix was different, but I'm not confident enough to say that it was. (Although GHC at the bottom here and near the top of Rares could be a push/pull.)



Monster Trophies
Average pull percentage: 8.18%
Standard deviation: .004 (small)
Z-Score range: -1.60 - 2.02
Commentary: Monster Trophies appear to have high variance in pulls, but keep in mind that the standard deviation is very small. So even a few lucky pulls or having monster trophies over-represented could skew the numbers. And I'm guessing that's what happened with V17: a few lucky people pushed stuff higher. Most events had monster trophy distributions within 1 standard deviation, indicating that this was highly consistent across events.

Keep in mind that this category was explicitly noted for having a lower drop rate this year, and the numbers seem to bear that out.



Stalker Tokens
Average pull percentage: 17.95%
Standard deviation: .008 (medium)
Z-Score range: -1.55 - 1.60
Commentary: Stalker tokens probably had the most characteristic bell curve of any of the other tokens. Of note is that V18 and V19 had the lowest Z Score of any of the events. I'm not sure if this indicates that the Stalker tokens ran a bit lower towards the end of the year, if there was some supplementing of the treasure mix or if it's just random chance. One could make a push/pull argument for more uncommons in V19 as evidence that the mix changed, but neither are a strong enough indicator to really draw that conclusion.



Overall
I don't see evidence that any particular convention mix was "rich" in comparison to others. Almost all categories in all events fell with 1 or 2 standard deviations of the mean. The one item that fell above 2 standard deviations - V17 Monster Trophies - is explainable given how tight the standard deviation is. Same with the one item that almost fell outside the low end of 2 standard deviations - Momo Con Under Ultra Rare. This is such a tight grouping that I'd expect just a few tokens to push something one way or another.

Gencon, according to Jeff, was "juiced" in terms of having 10 extra legendaries that we wouldn't normally have seen (10 Widseth's Lutes.)

From personal experience, I saw the starter dungeon have much better draws than the regular one in the 3 runs I saw. Out of 30 draws, I saw 1 legendary, 2 relics, and 4 URs. That's a small sample size, though but I've head it from others as well. It also makes sense - give the new players something to get them hooked and they are more likely to come back. If they don't, the high value tokens aren't likely to be resold and won't impact the overall token economy.

In terms of draws for the year, I can see 4,000 happening. Isabelle and I do VTDs together. so on 9 VTDs we averaged 2.5 runs per at 25 draws each. That's just over 1,100 draws. Add GenCon and GHC to that with about 5 runs each where we likely had a ghost in every run, that's another 500 draws. Add treasure tokens from orders and maps, etc and 2,000 is reasonably attainable for just 2 people without going too crazy. My guess is we probably had 2,500 draws overall. If you have 3 people, 4,000 is definitely within reason before you can say you have a problem. We definitely don't have a problem. We can stop any time we want. Yeah. That's it. How many weeks until V21?

Fred