Page 1 of 2

Need advice on milking as much memory resources.

Posted: Thu Jul 16, 2015 8:07 am
by TheDragonfly
Hi, I'm currently making quite a large mod and am at a stage where every asset that I can get rid of will hopefully make the difference.

Here are my thoughts;

For each map is it better, for memory usage, to have one script with all necessary functions within than multiple scripts?
Also does filling the remainder of the map with the 'void' tiles make any difference to memory usage?

Hopefully someone here will know the answers I seek as I'm sure its already been contemplated by someone else...

Re: Need advice on milking as much memory resources.

Posted: Thu Jul 16, 2015 9:15 am
by Isaac
The first thing you might want to look into, is removing the definitions of the official assets that you are not using.

*If you don't use the tomb tile set, you can remove the entire thing from the standard assets script, and it won't, load it.

Re: Need advice on milking as much memory resources.

Posted: Thu Jul 16, 2015 9:32 am
by minmay
TheDragonfly wrote:For each map is it better, for memory usage, to have one script with all necessary functions within than multiple scripts?
no significant difference
TheDragonfly wrote:Also does filling the remainder of the map with the 'void' tiles make any difference to memory usage?
No, the "wall" tiles don't actually place any entities.

You should focus on optimizing your models and textures, those are by far the largest files other than music (and there's not much you can do for music except reduce quality).
Grimrock 2's memory usage balloons when the game is saved. If your mod is really pushing the limits, you could try to reduce that by saving fewer entities and components. In particular, you should always make sure that objects have minimalSaveState when appropriate, i.e. when the object won't change in any way (including enabling/disabling components in the editor!)). The difference in saving performance is huge when the object is common like a basic wall or floor.

But really, you have nearly 2 GiB to work with. Did you make 1 GiB of assets yourself? If so, that's really impressive! But I'm guessing you didn't. In that case, it shouldn't be a significant setback to just not use some of the assets you didn't make; players/modders have already seen them after all. Your mod doesn't need to use all 30 monsters, have yet another tomb, and have yet another castle, we already know what those look like from playing the main game.

Re: Need advice on milking as much memory resources.

Posted: Thu Jul 16, 2015 9:36 am
by TheDragonfly
Isaac wrote:The first thing you might want to look into, is removing the definitions of the official assets that you are not using.

*If you don't use the tomb tile set, you can remove the entire thing from the standard assets script, and it won't, load it.
and what if I've pretty much used a bit of everything?

My mod has 36 maps, and is playable fine up until around the last few levels the game crashes sometimes when trying to save the game.
I've just gone through (so yet to be tested) and already deleted a lot of unused/unseen textures within each map so see if that would be enough so get by, but apart from that I was wondering if I can do anything further?

Re: Need advice on milking as much memory resources.

Posted: Thu Jul 16, 2015 9:45 am
by minmay
Just copy the material/object/etc definitions you need into your own mod's init files and don't import the standard ones.

Re: Need advice on milking as much memory resources.

Posted: Thu Jul 16, 2015 9:45 am
by TheDragonfly
minmay wrote:
TheDragonfly wrote:For each map is it better, for memory usage, to have one script with all necessary functions within than multiple scripts?
no significant difference
TheDragonfly wrote:Also does filling the remainder of the map with the 'void' tiles make any difference to memory usage?
No, the "wall" tiles don't actually place any entities.

You should focus on optimizing your models and textures, those are by far the largest files other than music (and there's not much you can do for music except reduce quality).
Grimrock 2's memory usage balloons when the game is saved. If your mod is really pushing the limits, you could try to reduce that by saving fewer entities and components. In particular, you should always make sure that objects have minimalSaveState when appropriate, i.e. when the object won't change in any way (including enabling/disabling components in the editor!)). The difference in saving performance is huge when the object is common like a basic wall or floor.

But really, you have nearly 2 GiB to work with. Did you make 1 GiB of assets yourself? If so, that's really impressive! But I'm guessing you didn't. In that case, it shouldn't be a significant setback to just not use some of the assets you didn't make; players/modders have already seen them after all. Your mod doesn't need to use all 30 monsters, have yet another tomb, and have yet another castle, we already know what those look like from playing the main game.
Ive created myself a litte open world style mod over 36 maps (not all fully utilized) with all existing assets. So it sounds like I should be able to do something with the 'minimalSaveState' thing. As im pretty new to the whole scripting/lua language, how would one go about doing that for each object?

Re: Need advice on milking as much memory resources.

Posted: Thu Jul 16, 2015 6:18 pm
by minmay
The standard assets already have minimalSaveState where appropriate. I am only talking about custom assets. Since you don't know what minimalSaveState is, I'm guessing you didn't actually define any objects yourself, in which case you don't need to add it to anything, unless you used faulty objects from someone else.

Re: Need advice on milking as much memory resources.

Posted: Sat Jul 18, 2015 10:00 pm
by Drakkan
I am still having problems. As my dungeon grew about 50 floors it starting to crash even with medium setting sometimes. :/

minmay could you please advice:

1. object like lanterns having castShadow = true and staticShadow = true ? will help set it to false ?
2. how much memory take "cloned" monsters ( default ones only with changed textures) ?
3. do objects like walls require minimalSaveState = true in case I am not going set its position somehow ? I noticed many default walls has it and I do not see reason for that
4.I am highly suspecting the town tilesets for many problems. I alreay set minimalsave false to all objects from this assets, still I wonder if some occluders etc are missing it could cause some memory issues

Re: Need advice on milking as much memory resources.

Posted: Sun Jul 19, 2015 12:02 am
by minmay
Drakkan wrote:I am still having problems. As my dungeon grew about 50 floors it starting to crash even with medium setting sometimes. :/
Because you used an excessive amount of textures, combined with a large number of objects and components.
Drakkan wrote:1. object like lanterns having castShadow = true and staticShadow = true ? will help set it to false ?
castShadow is whether a light source casts shadows (true) or doesn't (false). staticShadow is whether a model has static shadows (true) or dynamic ones (false). Models should have staticShadow set to true unless they are moving or animated in some way, since dynamic shadow casting is more expensive. The effect on system RAM usage is from this is insignificantly small, if it even exists at all; static shadows exist because casting dynamic shadows for every model would be unnecessary and horrible for rendering speed, and non-shadow-casting lights exist because giving shadows to every light source would be unnecessary and horrible for rendering speed.
Drakkan wrote:2. how much memory take "cloned" monsters ( default ones only with changed textures) ?
However much memory is taken up by the texture, plus a completely inconsequential amount of Lua memory. Object definitions don't use an appreciable amount of memory. Textures (which are loaded when you use them in a defineMaterial call, even if you don't actually place objects that use them), and to a lesser extent models, do. Memory usage will also balloon whenever the player saves; you can reduce this balloon by reducing the number of objects, and especially by reducing the number of objects and components on objects that lack minimalSaveState.
Drakkan wrote:3. do objects like walls require minimalSaveState = true in case I am not going set its position somehow ? I noticed many default walls has it and I do not see reason for that
Default walls have it because otherwise save games would be enormous: there are often hundreds of walls on one level.
Drakkan wrote:4.I am highly suspecting the town tilesets for many problems.
I have told you many times that this tileset conversion is awful and you shouldn't use it unless you are going to fix it. The only assets that were been converted to anything usable are the textures. The object definitions are essentially useless.
Drakkan wrote:I alreay set minimalsave false to all objects from this assets
What the hell, why would you do that? No wonder you're having memory usage problems if you're not using minimalSaveState. Without minimalSaveState, the entire state of an object and all its components must be saved. With minimalSaveState, the object is essentially just respawned: only its name, id, and map position (not world position) are saved. The difference in savegame size, and therefore memory usage from saving, is very large for objects that are used many times. If an object isn't going to change state, it should have minimalSaveState. If it is going to change state (a surface that could have items added to it, an item that could be picked up or moved, a door that could be opened, etc) then it shouldn't. Remember that disabling a component in the editor is also a state change, so, for example, you shouldn't disable (or open, or do anything else with) the DoorComponent on dungeon_wall_grating since it has minimalSaveState.
Drakkan wrote:still I wonder if some occluders etc are missing it could cause some memory issues
...how could occlusion possibly reduce RAM usage? I think you are very confused about some basic concepts here. Occlusion just prevents models/lights/etc from being rendered when they aren't visible, to improve rendering performance. It is absolutely inexcusable that there are tilesets released without occlusion support, and even more inexcusable that there are mods released using them, but that's because of the god-awful rendering performance that results from no occlusion, not because of memory.

Re: Need advice on milking as much memory resources.

Posted: Mon Jul 20, 2015 6:05 pm
by Drakkan
I started removing the city tileset...it will be quite a work. Meantime - what is happening is that while I run the game inside the editor and I am playing it, after aprox one minute of walking around it crash the editor because of lack of memory. I am walking on one location, therefore no loading of some new areas etc. Do you think this is still memory problem, or there could be some infinity script running which causing this problem ? Is there any way how to recognize this ?