How useful are mod endorsemant rankings?

Post » Sat Jul 07, 2012 5:31 am

The thread I just read on Skyrim Redone (SkyRe), brought up a point that I've been wondering about for a long time. How accurate are mod endorsemant rankings? How useful is that information? I'd like to know others' opinions on this.

Much like polling data, I suspect that a lot of endorsemant votes don't necessarily reflect current opinions on mods. How many mods have you endorsed in the past but you don't use in your mod list? How many mods have you started using, endorsed after 3-4 hours of play, and then started to have major issues with the mod 12 hours into your game? How many times have you decided that a particular mod isn't for you, but endorsed it anyway because you feel that the author put a lot of effort into the mod and that it was still high quality? How many times have you un-endorsed a mod? Are you more likely to criticize Bethesda for something you feel wasn't done well or a mod author for the same reason?

I know that the folks over at the Nexus do their best to get good information out of those statistics. They don't allow you to endorse a mod that you haven't downloaded and (presumably) used for at least a few hours. But consider that many of the most highly rated mods are sixual and nudity mods, mods which break game lore, etc. I don't know anyone that uses those mods. I've never even downloaded them to give them a try. I never see those kinds of mods when people on these forums post their mod lists. I never see them on YouTube reviews or "Let's Play" videos either. Who actually uses these mods?

Is there something funny going on with the numbers, or is there just a huge population of anonymous people out there who make up the majority that we don't interact with on places like these forums?

I have always suspected that a lot of people download and endorse mods for which they really like the concept, but don't judge too harshly on the execution of that concept, are more likely to put up with instability and bugs, become more lenient on things that break game-balance, etc. I'm not suggesting that everyone who endorses a mod should evaluate those mods according to some particular set of principles or standards. I'm just interested in how they do things already. If the mod sounds really good in the description, claims to "fix" Skyrim in various ways, improve the game, and has lots of neat looking screenshots, are users more likely to judge the mod based on the authors ambitions or on the actual execution? How much does the marketing of a mod affect peoples' endorsemants?

What do you guys think?
User avatar
Reven Lord
 
Posts: 3452
Joined: Mon May 21, 2007 9:56 pm

Post » Sat Jul 07, 2012 9:03 am

Also, think back to all of the mod review videos that you've seen. When was the last time you saw someone with a popular channel (Gopher et. al.) give a mod a bad review? I never see negative reviews other than in comment threads on the Nexus. Are reviewers afraid to give a mod a bad review? Is it some kind of a cultural taboo within the modding community to criticize a mod? If so, why?
User avatar
Breautiful
 
Posts: 3539
Joined: Tue Jan 16, 2007 6:51 am

Post » Sat Jul 07, 2012 5:52 am

I think it's useful for finding out what's popular. It helps to filter by category or searchword. You don't have to wade through body replacers if you're not looking in that category. It's more useful than downloads as a metric, I think, since endorsing a mod requires a bit of extra effort. It's not the ONLY good way to find mods, but sorting a search/category by endorsemants is usually more useful than sorting by date, personally.

As for the ranked lists, I think they more or less work. The majority of the top 25 (past two weeks) or top 100 (all time) are more substantial mods than body replacers or lore breaking outfits. They're in there too, but they make up a fairly small portion overall. As for the relevancy, you've got a filter for current trends with the top 25, and a long term filter with that top 100, and generally speaking the lists have a decent collection of mods that remain relevant today. Maybe not anyone in particular's top 25 or top 100, but I think it works.

As for the front page, I suspect it reflects an even shorter trending data set than the two weeks the top 25 uses. So you end up with things that are immediately popular but lack of the staying power of more substantial entries. Sometimes the gameplay stuff breaks through too, but I find most of my mods through filtering my searches rather than relying on those six on the front.

Also, think back to all of the mod review videos that you've seen. When was the last time you saw someone with a popular channel (Gopher et. al.) give a mod a bad review? I never see negative reviews other than in comment threads on the Nexus. Are reviewers afraid to give a mod a bad review? Is it some kind of a cultural taboo within the modding community to criticize a mod? If so, why?
I think it's generally taboo on the Nexus. Depends a lot on the author and what the criticism is, really.

As for YouTube videos, I don't think they're really reviews are they? So much as mod showcases? There's TONS and TONS of crap mods. You don't really need scores to highlight the bad or mediocre ones when you could spend that same time reviewing mods that are good and giving those attention. If people want to actively seek out constructive criticism I think there's plenty of ways to go about that, but there's far too much content to have people really focused on the middle to low stuff in the hope that the author will make it better.
User avatar
Rich O'Brien
 
Posts: 3381
Joined: Thu Jun 14, 2007 3:53 am

Post » Fri Jul 06, 2012 7:25 pm

Most face and body replacer mods (and tweakings of vanilla armors to go with them) don't have .esp files, so they won't show up in posted load orders.

Endorsemants are what they are: useful but not the whole story. I tend to look at not the raw number of endorsemants but the endorsemants/unique download ratio--I find that on the Nexus, around 1:40 or better is a good predictor that the mod has satisfied its downloaders as doing what it says it will do.

But that's just one factor among others, like the mod's description (does the mod do something that I want in my game?), compatibility concerns (will it work with the mods I already have and like?), and the recent comments (is the mod author still active? Have there been any troubles with recent game updates?).
User avatar
Veronica Martinez
 
Posts: 3498
Joined: Tue Jun 20, 2006 9:43 am

Post » Sat Jul 07, 2012 8:36 am

I use them to look at how reliable/stable a mod is. A mod with a lot of endorsemants and downloads is usually of good quality. That doesnt go without saying though that that is not always the case. I mainly use it like the old saying, "where there is smoke, there's fire".

Then again there are mods that fly under the radar that are gems but didnt get noticed because of various reasons. These mods when i happen across them, i read them through (several times) to ensure i know what it is doing. Then i read the comments ( also read these on popular mods too for a better idea of what i'm getting) and when i actually get the file(s) i prod through it to see what it does. If it looks of good quality i upload my test character and give it a spin via Wrye Bash. If it passes with my test character with my other mods installed then it will make it into my load order.
User avatar
Hearts
 
Posts: 3306
Joined: Sat Oct 20, 2007 1:26 am

Post » Sat Jul 07, 2012 10:23 am

Also, think back to all of the mod review videos that you've seen. When was the last time you saw someone with a popular channel (Gopher et. al.) give a mod a bad review? I never see negative reviews other than in comment threads on the Nexus. Are reviewers afraid to give a mod a bad review? Is it some kind of a cultural taboo within the modding community to criticize a mod? If so, why?

You're digging waayy too deep. Its not Gopher's job to review bunches of mods and provide his honest feedback on them. It is Gophers hobby to do video reviews of mods that he (as well as his fans and various others) find worthy of reviewing. Basically, he gets to choose what mods he reviews. Really he isn't providing a thumbs up or down to any particular mod, but providing a more in-depth description of mods that have already been deemed as "good mods".

Overall you have to think,read and play for yourself. Video reviews, comments, endorsemants and downloads can help steer you towards your goal. I usually look for a mod based on what I'm trying to fix or add to the game, then sort the results based upon endorsemants or downloads, but i still read through about 15-20 mods to make sure I like what I'm getting. Sure, I've downloaded mods that I don't use anymore, but I've always been a skeptic. For example, I've never paid to see a bad movie in theatres, this isn't to say I haven't seen a poorly reviewed movie in theatres, or that I simply avoid going to the movies. Rather, I make sure I know what I'm walking in the theatre to see before I actually buy the ticket. Sure, tastes will differ, but the only person that I know exactly what their tastes are and what they are thinking is me.
User avatar
ZANEY82
 
Posts: 3314
Joined: Mon Dec 18, 2006 3:10 am

Post » Sat Jul 07, 2012 8:12 am

Endorsemants are what they are: useful but not the whole story. I tend to look at not the raw number of endorsemants but the endorsemants/unique download ratio--I find that on the Nexus, around 1:40 or better is a good predictor that the mod has satisfied its downloaders as doing what it says it will do.

That's a clever metric. Have you ever pitched that as an idea to Dark0ne or anyone else at the Nexus? I would be really interested in what the results would be for for such a filter.
User avatar
Bereket Fekadu
 
Posts: 3421
Joined: Thu Jul 12, 2007 10:41 pm

Post » Fri Jul 06, 2012 9:09 pm

You're digging waayy too deep. Its not Gopher's job to review bunches of mods and provide his honest feedback on them. It is Gophers hobby to do video reviews of mods that he (as well as his fans and various others) find worthy of reviewing. Basically, he gets to choose what mods he reviews. Really he isn't providing a thumbs up or down to any particular mod, but providing a more in-depth description of mods that have already been deemed as "good mods".

Overall you have to think,read and play for yourself.....

I'm not making any judgement as to what Gopher should or should not be doing in his videos. Zherok mentioned that most of the YouTube videos are more "showcases" than reviews, and gave good reasons as to why it makes sense for that to be true. Fair enough. But what I'm really interested in, is whether or not there is a bias against being critical of mods, not so much what the goals of Gopher and others' videos are. Let's say I put together a blog or a YouTube channel that reviewed what I felt were both good and bad mods. What do you think the result would be? If I were to be critical of some mods and point out flaws, do you think I would be scorned and ignored as a result? If so, why do you think that is? Are mods considered sacred by the community?
User avatar
Ice Fire
 
Posts: 3394
Joined: Fri Nov 16, 2007 3:27 am


Return to V - Skyrim