[Emily abigail Villarreal]

After utumno mentioned in the BOSS thread about how keeping track of which mods you have are clean and which aren't, the gears in my brain started to turn. I thought I would put my ideas out in the open for discussion.

Basically, cleaning mods is:

• Simple.
  
• Relatively quick.
  
• Fun!

Keeping track of which mods you have are clean and which need cleaning, however, is:

• Complicated.
  
• Confusing.
  
• Not fun either.
  
• Time consuming.

I don't know if the cleaning process itself can be automated or improved in any way, and it's not my concern at this time. What I am focussing on is raising awareness of cleaning mods to more mod users and trying to make managing the process as easy as possible.

I'm basing my ideas off the current format of the information on mod cleaning, being a whitelist of mods that need cleaning and a blacklist of mods that shouldn't be cleaned. There's also a list of pre-cleaned mods, but I don't think this is required.

For the whitelist, each plugin will have a CRC attached to it, which would be the CRC of the unclean mod. A program would then look through your installed mods for any that are on the list. If it finds one, it then checks the CRC of your plugin against the unclean plugin CRC. If they match, it then tells you that plugin needs cleaning. For the blacklist, no additional information is needed, and instead if the program finds a plugin installed that is on the blacklist, the program will tell you not to clean the mod.

The program that does all this can also have an updater to grab the latest copy of the list from some storage location on the internet. The idea is basically that there is an always up-to-date and personalised version of the information on CS Wiki and the Cleaning thread in this forum that people can use to help manage their mod cleaning.

I've been pretty vague on the program itself: either something new could be written, or such functionality could be built into an existing utility, with the primary candidates being Wrye Bash or BOSS. Both have roughly the same number of downloads on TES Nexus, so they are pretty equal in terms of user penetration, which is one thing that a new program would be severly lacking.

I don't know much about Wrye Bash's functions, but BOSS v1.7 will have all the functionality required to implement such a system, if the whitelist and blacklist are combined in the masterlist. The only possible addition to be made would be a new Cleaning message type to differentiate such messages from other notes. I've also recently already been adding notes about unclean mods to the masterlist, which some may have noticed - currently these don't check your installed plugin because the current BOSS doesn't support that.

Of course, there's some things I've thought of that need a bit more thinking, and that's where you come in:

• Should the program be a new one, Wrye Bash or BOSS?
  
• Volunteers - would you be interested in helping keep the lists up to date? Basically all this requires is to add a mod's filename to one of the lists when you update/install it and check if it needs cleaning. Of course, supplying details are good, but that just requires reading off TES4Edit. Most of the time there's no special experience or knowledge needed.
  
• Handling of different versions of mods. I currently have in my toolkit stuff to check plugin CRCs and the version included in the mod's description, so we can check for specific plugins. This does mean that each version of a dirty plugin will need to be recorded, which takes up more space.
  
• Blacklist/whitelist precedence. I'm currently thinking that the blacklist would hold all dirty mods, whereas the whitelist would only hold dirty mods that should not be cleaned, to save a massive list of plugins that are OK. My thinking is that we don't need to do anything about clean mods, so we don't need to be told anything about them.

I'm asking for volunteers because I don't want to just assume that if this does get incorporated into Wrye Bash or BOSS, that the current teams can handle it, since they're (ie. PacificMorrowind ) pretty busy as it is.

Any input you may have is appreciated. I've typed this up in a bit of a rush, so I probably forgot some things, and I'll update when I remember them.

[Laura Wilson]

What, take away the fun of OCD's enthusiasts like me who do everything manually with http://i53.tinypic.com/9lbzg0.jpg? Nooo. :tongue:

But seriously this will probably be very nice to have. I'd like to know if my version of a mod is clean though, if only as extra info. You can probably find a lot of supporters to maintain the list so no worries there.

Edit: My list has the website link to each of the mod so I can simply click and go grab an update if necessary, would be nice if the utility can have that too.

[Amy Melissa]

Clarify please: is this just a project for building the white and black lists, or are you proposing some form of automation of the cleaning process?

.

[Sierra Ritsuka]

Very nice idea. I assume you mean to automate the cleaning process too, or do you just want a list of things that need to be cleaned and how?? You could ask the people that do the listings for the SVN to put a special character/string (similar to a bash tag) on the list next to the filename, where something like /c/ means "clean" and /u/ means "unclean/needs cleaning", and if the cleaning is automated, then you can put the /u/ in the .esp description so that you know this one has been cleaned and to skip over it next time.

EDIT: I notice that the masterlist has already got a ? Needs cleaning etc. part added in on some of the mods. A simple solution in the interim is to just get people to change the description of the mod and add in a /c/ or something after they've done the cleaning, and then just make BOSS check whether or not the mod has a /c/ in it or something along those lines, and then it won't remind you to clean it if it does. If I find some time over the next few days I could even try and add that in to the code?? Last time I worked on the source code it was before you separated it out into different .cpp for the different functions, but it shouldn't take me too long to get readjusted. In case you think I'm insane/wonder what I'm on about I added in the checks for a FOOK or FWE game for the Fallout version of BOSS a while back.

[Setal Vara]

Clarify please: is this just a project for building the white and black lists, or are you proposing some form of automation of the cleaning process?

.

Very nice idea. I assume you mean to automate the cleaning process too, or do you just want a list of things that need to be cleaned and how?? You could ask the people that do the listings for the SVN to put a special character/string (similar to a bash tag) on the list next to the filename, where something like /c/ means "clean" and /u/ means "unclean/needs cleaning", and if the cleaning is automated, then you can put the /u/ in the .esp description so that you know this one has been cleaned and to skip over it next time.

EDIT: I notice that the masterlist has already got a ? Needs cleaning etc. part added in on some of the mods. A simple solution in the interim is to just get people to change the description of the mod and add in a /c/ or something after they've done the cleaning, and then just make BOSS check whether or not the mod has a /c/ in it or something along those lines, and then it won't remind you to clean it if it does. If I find some time over the next few days I could even try and add that in to the code?? Last time I worked on the source code it was before you separated it out into different .cpp for the different functions, but it shouldn't take me too long to get readjusted. In case you think I'm insane/wonder what I'm on about I added in the checks for a FOOK or FWE game for the Fallout version of BOSS a while back.

For the above questions:

Basically, I think that which mod cleaning itself isn't necessarily something that can be automated, the task of what requires it can be. In that sense the thread title is a bit misleading, but there you have it.

So I'm not thinking about automating TES4Edit's job. Perhaps it can be done, but I don't have the knowledge of cleaning required to make that judgement. I'm just focussing on raising the awareness of mod cleaning and getting up-to-date, relevant information out to as many people as possible. Perhaps introducing yet another utility isn't the best way to do it, but as I've said, integration with Wrye Bash or BOSS are two possibilities once there's a working model.

@ Alithelord: I'd really appreciate if you didn't add anything to the BOSS code until 1.7 is released. As for your suggestion, it smells like a hack to me, not a solution, and hard-coding in hacks where solutions are possible isn't something I would be happy with. Look up all the changes I've made in relation to the Masterlist Format and BOSS's capabilities for 1.7, in particular CRCs (I've written stuff up on the Google Code page Wiki). You don't need to be editing any code to do what you're wanting.

(By hack I mean that it's a very specific special case that has no wider context, usage or meaning. If there is no other way to achieve the effect, that's OK, but if there is the hack is basically just bloating BOSS.)

[Emmie Cate]

Keeping track is never an issue for me.

Simply put - most mods get repackaged - during the repackaging I clean the mods - and the cleaned plugins go into the repackaged BAIN archive, so if installed via BAIN it is already clean.

Adding the mods via BAIN that need cleaning is annoying because then after their cleaned the BAIN archive shows up as red.

Of course there are modders I've grown to trust and will often take their word for it.

Most things then never get added unless already cleaned - This system works 95% of the time for me.

[LuBiE LoU]

For the above questions:

So I'm not thinking about automating TES4Edit's job. Perhaps it can be done, but I don't have the knowledge of cleaning required to make that judgement. I'm just focussing on raising the awareness of mod cleaning and getting up-to-date, relevant information out to as many people as possible. Perhaps introducing yet another utility isn't the best way to do it, but as I've said, integration with Wrye Bash or BOSS are two possibilities once there's a working model.

@ Alithelord: I'd really appreciate if you didn't add anything to the BOSS code until 1.7 is released. As for your suggestion, it smells like a hack to me, not a solution, and hard-coding in hacks where solutions are possible isn't something I would be happy with. Look up all the changes I've made in relation to the Masterlist Format and BOSS's capabilities for 1.7, in particular CRCs (I've written stuff up on the Google Code page Wiki). You don't need to be editing any code to do what you're wanting.

(By hack I mean that it's a very specific special case that has no wider context, usage or meaning. If there is no other way to achieve the effect, that's OK, but if there is the hack is basically just bloating BOSS.)

I'll take a loot at the wiki . But I see where you're coming from; in the interim people could just write "clean" in the mod description once they've cleaned it instead of adding all of that stuff into BOSS .

[Ron]

I'm currently thinking of writing a utility to handle this, using a pretty similar system to BOSS: a whitelist of not-to-be-cleaned mods and a blacklist of dirty mods, with a file parser. Said system could also hold information about how many deleted references or identical to master records were found, and information on what a particular plugin's wild edits are to make finding them in TES4Edit easier. With my experience with BOSS, this should be pretty simple, and this project could benefit from the same type of auto-updated lists that BOSS does using an online SVN system.

Essentially the idea is that it's an always up-to-date and personalised version of the information found on the CS Wiki and on the Cleaning thread in this forum. The personalisation will be done by checking your plugin's version/contents against the unclean plugin's version/contents for all your plugins.

So basically, you want to make a clone of BOSS that does nothing more than check your load order against an existing archive to see if you have mods that need cleaning. Seems like an Excel spreadsheet could handle it just as easily; all you'd have to do is list the plugins in alphabetical order, with version number, ITM, and UDR info. You can easily maintain such a thing through Google without having to resort to writing another program. But that's just me.

In either case, I'd be willing to help.

[Austin Suggs]

Ok this might be stupid and base on misunderstand of the cleaning process :
A reposit server loaded with already cleaned esp(s)
A BOSS-like tools that scan your mods and compare checksum/whatever with ones on the repo; then list in green the ok ones and in red the bad ones.
You tick the red ones and click on
[Replace with Clean ESP]
Are you sure ?
[OK]
Sure, sure ?
[Yes...]
This your last chance...
[Execute Order 66]

[Jesus Sanchez]

So basically, you want to make a clone of BOSS that does nothing more than check your load order against an existing archive to see if you have mods that need cleaning. Seems like an Excel spreadsheet could handle it just as easily; all you'd have to do is list the plugins in alphabetical order, with version number, ITM, and UDR info. You can easily maintain such a thing through Google without having to resort to writing another program. But that's just me.

In either case, I'd be willing to help.

Well, Excel spreadsheets must be able to do a whole lot more than what I use them for. :unsure: If it saves me work, then perhaps that's a better way of doing things.

The way I see it is this, either:

• Things stay as they are. This wouldn't be terrible, but I think they could be better. Mod cleaning needs more general awareness.
  
• A utility devoted to cleaning gets created to make the process simpler. ATM we can't automate actual cleaning, so it's the 'knowing what needs cleaning' that can be improved.
  
• A system that does the above gets built into another utility. BOSS and Wrye Bash are equally viable as they are roughly of the same popularity, and the only two utilities of that level of popularity still updated.

If something new does get done, the best way I can see is via a whitelist/blacklist system such that mods needing cleaning are whitelisted and mods that shouldn't be cleaned are blacklisted. It's this idea, together with how clean/unclean mods are identified, that I thought I'd throw out for community discussion.

I think the responses I've had are beginning to sort things out in my head a bit, I'll update the OP to be a bit clearer.

[Nicole Elocin]

Of course, there's some things I've thought of that need a bit more thinking, and that's where you come in:
The most important thing first - it needs a name. Preferably something with an amusing acronym.

Thinking you already found the name...CMCP = Community Mod Cleaning Project

Lol....OR
CUCMM = Can you Clean My Mod / Community Unofficial Clean Mod Manager

Since the Forum can deal with 3 letter acronyms....
CMM = Clean Mod Manager!

[Nicole Kraus]

I've tried to tidy up the OP - hopefully my thoughts should be a bit clearer now.

Ok this might be stupid and base on misunderstand of the cleaning process :
A reposit server loaded with already cleaned esp(s)
A BOSS-like tools that scan your mods and compare checksum/whatever with ones on the repo; then list in green the ok ones and in red the bad ones.
You tick the red ones and click on
[Replace with Clean ESP]
Are you sure ?
[OK]
Sure, sure ?
[Yes...]
This your last chance...
[Execute Order 66]

This probably falls under redistributing mods without author consent, which I don't want to do. I don't feel comfortable with the idea of providing mod files for mods that I don't have permission to distribute. Even if that weren't an issue, file hosting for what is probably a few thousand plugins could be an issue...

[SEXY QUEEN]

Keep it simple, please. Just a reminder of existing functionaity in Wrye Bash.

When you install a mod using BAIN, it already checks the CRC for each file. You could have a CSV file (or some other method) to identify a list of unclean plugins with the CRC as match basis. Then implement a feature - such as highlight or the ability to generate a report, just like there is a report generator for i.e. the bashed patch - and report the known plugins that match the list. CRC is IMO the best way to do it.

I suspect making a utility just to maintain a list of uncleaned esp plugins is redundant.

In case you want to try harder to complicate things, you could use a diff patcher to make diff updates compared to the version of the file cleaned using the TES4Edit steps. Then some good samaritan (who already cleaned the plugin and care to make the diff) could perhaps upload the diff file in a repository where the new tool would fetch the needed update diff. The user running the tool would have the plugin upgraded to the cleaned version by binary patch based on the respective diff. Of course, you would also have to maintain - or monitor - every possible mod selection because I suspect not every mod author would be willing to let you know of updates or even do them for you. Authors just make a new version of their own mods and upload them. A big disaster, since even BOSS does not reach 100% of every mod out there.

Hope this makes sense and helps shed some light.

[StunnaLiike FiiFii]

Well, Excel spreadsheets must be able to do a whole lot more than what I use them for. :unsure: If it saves me work, then perhaps that's a better way of doing things.

Oops. That's not what I meant. I meant, just maintain a database using Excel - like Ulrim said, I don't see the need for a separate utility. Ulrim's hit on a good idea with a CSV list in Wrye Bash, though.

If something new does get done, the best way I can see is via a whitelist/blacklist system such that mods needing cleaning are whitelisted and mods that shouldn't be cleaned are blacklisted. It's this idea, together with how clean/unclean mods are identified, that I thought I'd throw out for community discussion.

Um, don't you have those backwards?

[Jason King]

Um, don't you have those backwards?

Not to my mind - the whitelist tells you which dirty mods are safe to clean, the blacklist tells you which are not safe.

The CSV thing sounds good. I don't have any experience with them myself, so perhaps a Bash team member can give some insight on whether or not this would be a good idea and something they'd consider for Bash?

[Music Show]

The way I see it is this, either:

• Things stay as they are. This wouldn't be terrible, but I think they could be better. Mod cleaning needs more general awareness.
  
• A utility devoted to cleaning gets created to make the process simpler. ATM we can't automate actual cleaning, so it's the 'knowing what needs cleaning' that can be improved.
  
• A system that does the above gets built into another utility. BOSS and Wrye Bash are equally viable as they are roughly of the same popularity, and the only two utilities of that level of popularity still updated.

If something new does get done, the best way I can see is via a whitelist/blacklist system such that mods needing cleaning are whitelisted and mods that shouldn't be cleaned are blacklisted. It's this idea, together with how clean/unclean mods are identified, that I thought I'd throw out for community discussion.

I'm not sure which of the three options is best right now (Bash/BOSS/new program). Either one, I would definitely like to get it integrated into the Wrye Bash interface for easy cleaning (Right Click->Clean Mod). As far as the lists go, if Bash isn't the one to maintain it, then it will at least need to be exported/saved in a format that Bash can read it so it can provide visual indicators on the status of their mods (sort of like how it reads the masterlist for bashed tags). Either way, I for one would be willing to support the time getting it integrated into Wrye Bash.

For the automated cleaning I have one question, then a few possibilities: I'm new to cleaning, but I couldn't find any command line options for TES4Edit, so I'm assuming you can't clean a mod just via command line, correct? Assuming not, I'll try to contact Elminster (haven't seen him in quite a while), to see if it could be added. I'll also ask Waruddar if it could be something to be implemented in CBash (just the cleaning portions of the TES4Edit code). I'm guessing that most of the capabilities are already there in CBash, but I haven't looked at its code in months, so :shrug: I know Wrye Bash could definitely handle the Identical to Master records, I just don't know about the undelete/disable ones, or how exactly that's done.

The hardest part of the whole process will be getting BAIN to accept the new esps/esms as good copies of the ones in the archives, unless we have BAIN automatically repackage them with the clean versions.

Anyway, the way I see it:

• We do nothing (noooo!)
  
• The tracking of clean/dirty mods is handled by BOSS or another program, with an exported .txt/.csv/whatever telling Wrye Bash what mods are dirty and what mods can be cleaned
  
• The clean/dirty mod tracking is handled by Wrye Bash (not a huge fan of this one, as I'm not on often enough to update such a list, and I know myself/PM/War often have periods of time where we just can't be around)

I'll start working on a way to get Wrye Bash to do the cleaning (either by calling a different program, or internally via it's own code). How we decide to handle the list, my vote is to put it in BOSS, since you guys already have the infrastructure, etc for getting people to report mods, etc

[Craig Martin]

I'm not sure which of the three options is best right now (Bash/BOSS/new program). Either one, I would definitely like to get it integrated into the Wrye Bash interface for easy cleaning (Right Click->Clean Mod). As far as the lists go, if Bash isn't the one to maintain it, then it will at least need to be exported/saved in a format that Bash can read it so it can provide visual indicators on the status of their mods (sort of like how it reads the masterlist for bashed tags). Either way, I for one would be willing to support the time getting it integrated into Wrye Bash.

For the automated cleaning I have one question, then a few possibilities: I'm new to cleaning, but I couldn't find any command line options for TES4Edit, so I'm assuming you can't clean a mod just via command line, correct? Assuming not, I'll try to contact Elminster (haven't seen him in quite a while), to see if it could be added. I'll also ask Waruddar if it could be something to be implemented in CBash (just the cleaning portions of the TES4Edit code). I'm guessing that most of the capabilities are already there in CBash, but I haven't looked at its code in months, so :shrug: I know Wrye Bash could definitely handle the Identical to Master records, I just don't know about the undelete/disable ones, or how exactly that's done.

The hardest part of the whole process will be getting BAIN to accept the new esps/esms as good copies of the ones in the archives, unless we have BAIN automatically repackage them with the clean versions.

Anyway, the way I see it:

• We do nothing (noooo!)
  
• The tracking of clean/dirty mods is handled by BOSS or another program, with an exported .txt/.csv/whatever telling Wrye Bash what mods are dirty and what mods can be cleaned
  
• The clean/dirty mod tracking is handled by Wrye Bash (not a huge fan of this one, as I'm not on often enough to update such a list, and I know myself/PM/War often have periods of time where we just can't be around)

I'll start working on a way to get Wrye Bash to do the cleaning (either by calling a different program, or internally via it's own code). How we decide to handle the list, my vote is to put it in BOSS, since you guys already have the infrastructure, etc for getting people to report mods, etc

Hmm, interesting. That actually sounds like a pretty good system that plays on both WB and BOSS's strengths. Rather than having BOSS export a file containing the list, could Wrye Bash not parse the masterlist like it currently does for Bash Tags?

I can add a cleaning-specific message type that WB could then look for, in pretty much perfect anologue to its current handling of Bash Tags in the masterlist. Perhaps the keyword could be DIRTY, then followed by a message stating whether the mod should be cleaned. For example:
In the masterlist:

DirtyMod.esp
IF (03439A49F|"DirtyMod.esp") DIRTY: Needs TES4Edit cleaning: "http://cs.elderscrolls.com/constwiki/index.php/TES4Edit_Cleaning_Guide"

DirtyFCOMMod.esp
IF (84538FAE2|"DirtyFCOMMod.esp") DIRTY: Do not clean. This mod should not be cleaned, as its dirty edits are required for the mod to work.

Giving the following in the BOSSlog:

DirtyMod.esp
* Dirty Mod: Needs TES4Edit cleaning: "http://cs.elderscrolls.com/constwiki/index.php/TES4Edit_Cleaning_Guide"

DirtyFCOMMod.esp
* Dirty Mod: Do not clean. This mod should not be cleaned, as its dirty edits are required for the mod to work.

How about that? The bit in brackets for the masterlist stuff is just the CRC conditional syntax. You could look for the "DIRTY" keyword in lines, then either check the last line with a mod name on it, or do a search for the last two quote marks and grab the text in between to get the plugin name.

[Stryke Force]

My personal preference would be a Wrye Bash option that makes use of CBash to do mod cleaning. The ITM and UDR steps are fairly safe. Resolving wild edits can't ever be done automatically since you have to examine the change records by hand in each case. People will still need to be aware of issues with implicit dependencies though. One wrong move on a patch mod that hasn't loaded all its constituent parts will be a problem.

As far as tracking which ones need cleaning and which ones don't, do we not already have at least two places doing that now? Probably 3 if BOSS is starting to do this? Would yet another spreadsheet really be helpful?

[Brentleah Jeffs]

My personal preference would be a Wrye Bash option that makes use of CBash to do mod cleaning. The ITM and UDR steps are fairly safe. Resolving wild edits can't ever be done automatically since you have to examine the change records by hand in each case. People will still need to be aware of issues with implicit dependencies though. One wrong move on a patch mod that hasn't loaded all its constituent parts will be a problem.

As far as tracking which ones need cleaning and which ones don't, do we not already have at least two places doing that now? Probably 3 if BOSS is starting to do this? Would yet another spreadsheet really be helpful?

It's at least three now. I've been trying to keep BOSS's notes up-to-date with what happens on the Cleaning thread (though there seems to be three of them too. :wacko: )

[Zosia Cetnar]

Hmm, interesting. That actually sounds like a pretty good system that plays on both WB and BOSS's strengths. Rather than having BOSS export a file containing the list, could Wrye Bash not parse the masterlist like it currently does for Bash Tags?

I can add a cleaning-specific message type that WB could then look for, in pretty much perfect anologue to its current handling of Bash Tags in the masterlist. Perhaps the keyword could be DIRTY, then followed by a message stating whether the mod should be cleaned. For example:
In the masterlist:



How about that? The bit in brackets for the masterlist stuff is just the CRC conditional syntax. You could look for the "DIRTY" keyword in lines, then either check the last line with a mod name on it, or do a search for the last two quote marks and grab the text in between to get the plugin name.

I think that could work fine I think. So this portion here will be the part in the masterlist that Wrye Bash can parse then?

DirtyMod.esp
IF (03439A49F|"DirtyMod.esp") DIRTY: Needs TES4Edit cleaning: "http://cs.elderscrolls.com/constwiki/index.php/TES4Edit_Cleaning_Guide"

DirtyFCOMMod.esp
IF (84538FAE2|"DirtyFCOMMod.esp") DIRTY: Do not clean. This mod should not be cleaned, as its dirty edits are required for the mod to work.

If that's the case, then Bash can just compare CRC's to those in the masterlist to see if the mod is dirty, and if it is, give a visual indicator of it being dirty, even clean them once we figure out how to automate the process. I'm assuming that there can be multiple entries for the same name (like for different versions of the same mod) like such:

IF (CRC_HERE_1|"DirtyMod.esp") DIRTY: Needs TES4Edit cleaning...
IF (CRC_HERE_2|"DirtyMod.esp") DIRTY: Needs TES4Edit cleaning...
IF (CRC_HERE_3|"DirtyMod.esp") DIRTY: Do not clean...

[Shianne Donato]

I think that could work fine I think. So this portion here will be the part in the masterlist that Wrye Bash can parse then?

If that's the case, then Bash can just compare CRC's to those in the masterlist to see if the mod is dirty, and if it is, give a visual indicator of it being dirty, even clean them once we figure out how to automate the process. I'm assuming that there can be multiple entries for the same name (like for different versions of the same mod) like such:

Yep, that's the idea. I just need to double check that BOSS is giving the same CRCs as Bash does, we had a bit of trouble with that earlier.

[Prue]

The hardest part of the whole process will be getting BAIN to accept the new esps/esms as good copies of the ones in the archives, unless we have BAIN automatically repackage them with the clean versions.

Anyway, the way I see it:

• We do nothing (noooo!)
  
• The tracking of clean/dirty mods is handled by BOSS or another program, with an exported .txt/.csv/whatever telling Wrye Bash what mods are dirty and what mods can be cleaned
  
• The clean/dirty mod tracking is handled by Wrye Bash (not a huge fan of this one, as I'm not on often enough to update such a list, and I know myself/PM/War often have periods of time where we just can't be around)

I'll start working on a way to get Wrye Bash to do the cleaning (either by calling a different program, or internally via it's own code). How we decide to handle the list, my vote is to put it in BOSS, since you guys already have the infrastructure, etc for getting people to report mods, etc

I thought of it too - didn't notice this thread though till now
The way I see it :

• No need to automate cleaning (in the first place - maybe later - one thing at a time)
  
• BOSS adds notification
  
• Bashy reads notification from BOSS into an added field that can be user edited : Needs cleaning. Priority : User input > BOSS
  
• Bash adds a special project named "Clean Mods". Once an espm conflicts with an espm in "Clean Mods" no conflict is reported ("Clean Mods" must be near the end of the list naturally, user placed - should be allowed to be packaged to avoid CRC refresh lag). No need to package/unpackage. I find this more elegant than any other solution. Don't know how hard this will be internally (conflict system).
  
• Also mere presence of an espm in "Clean Mods" cleans the Needs cleaning field for this espm
  
• EDIT : on whitelists etc - the info on BOSS + user input will be enough I think - it is mainly a question of keeping track of things - user marks a mod as needing cleaning, informs BOSS thread (optional), cleans it at his leisure, dumps espm in "Clean Mods", done.

This won't keep any logs any place though - maybe a problem (or maybe not)
God I have to sleep

[FirDaus LOVe farhana]

Yep, that's the idea. I just need to double check that BOSS is giving the same CRCs as Bash does, we had a bit of trouble with that earlier.

You can send me your calculated CRC for, say Oblivion.esm (SI, 1.2.0.416) and I'll give you mine: 0x2FF840C5

hopefully they're the same *crosses fingers*

[Leilene Nessel]

You can send me your calculated CRC for, say Oblivion.esm (SI, 1.2.0.416) and I'll give you mine: 0x2FF840C5

hopefully they're the same *crosses fingers*

From the BOSSlog: 2FF840C5

Good news!

[George PUluse]

Alright, here's my what I've got so far:

I've got Wrye Bash scanning the masterlist, using this regex:

IF\s*\(\s*(.*?)\s*\|\s*[\"\'](.*?)[\'\"]\s*\)\s*DIRTY:\s*(.*)\s*$

which will give me the: crc, filename, message
Assuming the masterlist will be in the format of:

IF (xxxx|yyyy) DIRTY: zzzz

with
xxxx = file crc in hex
yyyy = file name, with or without quotes
zzzz = message
I then check to see if the dirty message begins with "Requires TES4Edit cleaning", ignoring case of course. If it does start with that, then I mark the mod as requiring cleaning.

Does that line up with your plans wrinklyninja? Should I assume the 'IF' and 'DIRTY' are case sensitive, or go ahead and ignore case?

After all that, I store the message for displaying. Since it takes a while for bash to calculate the crc's (upwards of 2 seconds for a large mod), I cache the crc's, only updating them if the file size or modified date change. I'm almost considering changing the check on modified time, since Wrye Bash just resets that field anyway, so it will have no meaning essentially, if you use lock times. Only issue with this method I can see is if the file is changed, but filesize isn't, and the modified timestamp isn't changed, Wrye Bash will still use the old crc. It's either that or have sluggish UI :shurg:. I can always add an option to force refresh the crc's.

Anywho, here's what the http://img858.imageshack.us/i/wb1.png/ looks like, with an http://img848.imageshack.us/i/wb2a.png/ I still need to add a per-mod option to mark it as already cleaned, so even if BOSS says it's dirty, you can mark it as being cleaned.