There is an issue with the TF2 mod loading from your custom folder, where for each folder you have, the game has to scan it individually. While intuitively this would seem to be the same speed as having one folder, in practice it isn't - the game actually seems to take exponentially longer to scan each individual mod directory you have stored inside of tf/custom. To resolve this, I will explain how mod merging works and how you can run over a hundred mods while maintaining fast loading times.
tf/custom and mod loading
To give some background on why this combining works the way it does let me explain how your custom directory and how vpk files work.
When TF2 loads, it has a priority queue of assets that it loads. It reads assets in this order to determine what to actually show in game. The directory tf/ is the directory inside of your Team Fortress 2 directory
-
tf/custom - The custom folder, where mods usually go
-
tf/ - The folder which has everything else inside of it, including the custom folder
-
tf/*.vpk - The base game vpk files, these include things like tf2_misc_000.vpk, tf2_sound_misc_dir.vpk, and tf2_textures_dir.vpk
When attempting to display any asset, for example a texture on a wall, in effect the game will first check your tf/custom directory to see if that texture exists, then display it. If it doesn't exist, it will look in your tf/ dir (tf/materials in the case of a texture). If it doesn't exist there, it will fall back to the base game textures stored in the vpk files, these are the bulk of the file size of the installed game and include all the regular sounds, materials, and models. If it doesn't exist there, then it will show up as a missing texture.
The tf/custom directory is intended for mods and custom content. From the explanation above you can see why - for example if you wanted a mod that replaces the model of a gun, if you put a set of files in your tf/custom which have the same name as the regular gun's model, the game would look in tf/custom, see that file, then load it first and then stop.
In reality, the game caches values as soon as it reads them, and this happens on start up as well as when you join a server (since the sv_pure setting of the server may require it to cache non-custom assets, it re-reads your assets whenever you connect). So every single time a texture is rendered it isn't doing this lookup, it only performs one big check at certain points. If you install a lot of mods, you'll notice that opening your game and joining servers tends to be slow, taking minutes at a time if you have many mods.
So how does tf/custom work? To understand this let's understand how TF2 loads content.
Ignoring all base game vpks and the custom, let's assume the TF2 structure was simply the tf/ folder and its contained folders, such as materials, models, and sounds.
The directory structure would look something like this
Code: Select all
tf
├──materials/
├──sound/
└──models/
So the structure for loading a "tf2 asset" is to have a parent folder (in this case named "tf"), then folders for asset type (such as "materials") then categorized further as needed inside these directories.
Now let's dive into how tf/custom works. If we had two mods, one that was a sound mod and another that replaced a weapon, we might have a tf/custom folder that looks something like this
Code: Select all
custom
├──mymod1
│ └──sound/
└──mymod2
├──materials/
└──models/
As you can see, the structure for tf/custom is that each folder inside mirrors the tf2 directory structure. In effect each "mod" is a mini tf2 directory, which only contains the files and paths you want to overwrite.
Lastly, we can cover vpks. A "vpk" file is like a .zip file except not compressed. It is simply a single file which contains a folder, in a format more easily read by the source engine.
If you open up any of the valve vpk files mentioned previously (such as tf2_textures_dir.vpk) in a vpk viewer such as vpkedit, you'll see it mirrors this exactly - there is a "root" folder and underneath are relevant folders such as materials.
vpks serve two purposes, firstly they are easier to pass around and deal with for downloads and modding, as moving one file is more efficient for your computer than moving a folder. Secondly, when doing things like making maps, files inside of vpks are not packed into the map. You will often see mod downloads available as either a folder or a vpk - there will be no functional difference for the most people once the mod has been downloaded, they are simply two formats to store the same information. As a bit of bonus information for curious readers, map files (.bsp) can have a vpk "packed" inside of them - this is how custom maps can have custom assets, the files are packed in this same format just within the .bsp.
The important thing to note here is that every folder and vpk inside of tf/custom is a separate directory which needs to be read by the game, and this is what makes mod loading take so long.
To resolve this, we can combine everything into one directory.
Mod combining
To start off with, as a disclaimer I am on linux and I have a set of scripts which largely automate this process for me, so you will likely have to do this manually. I have some tips as well for not making any blunders that will be irreversible later.
One tool you should familiarize yourself with is VPKEdit, a very powerful tool for dealing with packed materials. It can create, view, and extract vpk files, as well as the material contents of maps (if any exist within).
Setting up your directory structure
For starters, I would set up a "working directory" that you can use for your modding, outside of your custom dir. In my case, I do all my combining inside of a directory path tf/custom_mods.
My recommendations for subfolders would be:
-
enabled - A folder which stores the folder form of mods you have enabled
-
disabled - A folder which stores the folder form of mods you have disabled
-
to_extract - A folder for vpks that you have downloaded but haven't extracted into their folder form yet. Helpful if you are downloading a lot of mods at once and want to keep track, as well as for automation purposes with scripts
-
extracted - An output folder for any directories extracted from vpks. Optional but helpful for automation workflows
-
vpks - A folder for vpks you already extracted, in case you ever need them again
-
combined - A working folder which will contain the combined folders from all of your mods
Extracting vpks
As mentioned above I would recommend creating separate folders for every step of your process to make it easier. If your mod(s) are not in vpk form prior to combining, you can skip ahead to the next step.
With VPKEdit, you will want to open the vpk you want to extract, then right click the "root" folder at the top and click Extract All. In the case of my above example, I'd have a vpk I want to extract inside of my to_extract folder, I'd extract the internal directory into extracted, then I'd move the vpk into vpks and the extracted directory into enabled.
Combining mods
Assuming you've done the above, you should have a directory which contains a bunch of subdirectories, each being a mod. Note that at this stage you should not have a materials or sound folder exposed, those should be within these subfolders.
As per the previous example here is what your enabled folder might look like.
Code: Select all
enabled
├──mymod1
│ └──sound/
└──mymod2
├──materials/
└──models/
Disclaimer: When doing this manually, I do not know of a good way to perform "quick" testing and recombining. It is easy to do with scripts but much more difficult manually. I suggest you thoroughly test all mods you plan to use and see if you like them or not and if they work, as removing them will be harder later.
Next up, you will want to follow this algorithm to combine mods: for each mod folder, copy (not move) the contents (materials/sound) into a new folder (called "combined" for instance) such that every time you have a new mod being combined, its internal folders (materials) combine their contents with the other mods you have already copied.
The end result is you will have one folder that should have, at most, the following directories:
Code: Select all
combined
├──materials/
├──models/
├──sound/
├──scripts/
├──cfg/
└──particles/
Note that if you have anything extra here it won't negatively affect your game - if your directories are set up wrong the worst that will happen is the game will not read it. Extra files won't cause adverse effects.
Combining it all together
Once you have your combined folder, you will want to go into VPKEdit and select File > Create from Folder > VPK and select your combined folder.
This vpk file, which can be named whatever you like, can then be placed inside of tf/custom. Certain mods may benefit from not being combined, and having anything less than 10 mods will not have too much of an effect on your game. For example, I have my vpk for all my skin mods, I have a separate vpk for my no hats mod, and a separate folder for my HUD. At single digit numbers of folders inside of tf/custom you won't notice too many differences.
Removing mods
If you use scripts, I would advise you to setup something such that each time you build the vpk, it deletes your "combined" folder, recreates it, and recombines everything.
If you do not have scripts, you will need to either start from scratch with your combined mods, or you'll need to find the mod you want to remove, then look through it's folder and see what files it adds, and go and delete those. This is a tedious process which is why I'd recommend only combining mods you are sure you want to keep, or to set up some scripts to automate the whole process for you.
