In the August, 2011 issue (Issue 048) of the Official Playstation Magazine, the developers of Skyrim claim to have used a type of "heat map" to gather data.

"Everybody on the team plays the game a ton. Every week, we playtest. We have a really cool system that shows where everybody goes, like a heat map of the world. We know where everybody went, and how much time they spent there. We can internally track every single thing, who finished a quest when. We track everything everybody does. The issue is not in the actual work of doing the balance, it's the issue of collecting the data and then deciding what to do. We could tune the game pretty fast, it's just know what dials to press." - Todd Howard, Skyrim Game Director


This reassures me that there was QA and game testing done. On the same note, gamesas may be relying on the community to help them prioritize which data to focus on in their fixes.

"The issue is not in the actual work of doing the balance, it's the issue of collecting the data and then deciding what to do. We could tune the game pretty fast, it's just know what dials to press."