Map Cache Fix

[KaYsEr]

Wow, it's crazy if some people have this problem and some people don't. I'll try on multiple computer as soon as I can.


(could also be linked to a specific version of windows, or if it's a 32/64 version etc, for example I have a 64bits windows... Who knows.)

 

[shazamham]

So I tried this out, just to let you guys know I know nothing about scripting and what not, and got an error.


TypeError


child.removeStageReference is not a function


I walked through 2 teleport events. Pretty much walked though one, then walked through another to go back to the previous map that I was previously in. I hoping that made sense, I personally do not know what this means. If you need anymore information just let me know.


Edit: I just ran it again this time only going through one teleport event and fought in a battle. I got the same error after the battle was over.


Forgot to mention I'm running it on android.

 

[Ashouse]

Im running win 8 64 bit. Using Kaus for the moment but may try bind to map to compare notes. Yeah, i get that phones are advancing faster than i am. My concern is for ppl like me, still using a g2, and happy about it. Since it isnt some major 3d engine it SHOULD (caps for emphasis) run fine, but doesnt. //rmwforums.s3.amazonaws.com/emoticons/default_sad.gif By my understandind its cuz its not native, but somewhere out there has got to be a solution because swim ran a dreamcast on this very device with no issues whatsoever... And THAT is saying something for the capability of this device. 


On a ram note: im using transparent tiles for my passability, could that cause a ram jump? I mean i edit the tile to be transparent, one is pass one is not... Then paste those over the parallax. Do you think using a plugin to handle this might boost performance. Btw im not getting much slow down, just heavy ram usage. I am running this plugin.

 

[KaYsEr]

On a ram note: im using transparent tiles for my passability, could that cause a ram jump?

I don't think it should be a problem.


And about overall performances, let's wait for the next update (with updated PIXI version) and we'll probably see a big jump in performances (+ maybe some major problems fixed).

Im running win 8 64 bit.

I tried on a WIN8 64bits and had the same problem of *RAM that doesn't flush until the game close itself* and now I tried on my GF's laptop (WIN10 64bits with only 4go of RAM) and it was the same problem too, it just happen quicker than on a big desktop with 32go like mine... But beside that, the game runs really great.)


Kaus overlays plugin is working flawlessly (the only bug I found is: auto-scrolling fog will restart when you get out the menu), even without this plugin Brandos posted. The real problem comes with the native parallax function or plugins using it (like the one from Galv).


On another hand, this "SpritesetMapCache" plugin is not functional yet, your "moving events" will stay at the last known position when you leave a map and come back, they will act like "ghosts" (you can walk through), but the original "solid moving event" will still be here moving around while being transparent (that's why it took me a while to realize that lol) like if the sprite and its collision were disassociated.


Maybe Brandos will ask for a fix. That would be nice since it's a great plugin, it even fix a bug with Mog Hunter's "Weather FX plugin" that spawn more particles when you get out the in-game menu.

 

[doranikofu]

I use parallax maps over 3000px so this is a big issue for me. The memory use is ~500mb for me just with map and char. That is with the plugin to clear cache, otherwise the memory builds up and crash at a certain point.


I also heard the maps are allocated memory based on their area (pixel size) instead of file size. So even if you compress your image files it won't save memory. I'm glad that more people are working on this and there might be a fix in the next updates.

 

[KaYsEr]

I use parallax maps over 3000px so this is a big issue for me. The memory use is ~500mb for me just with map and char. That is with the plugin to clear cache, otherwise the memory builds up and crash at a certain point.


I also heard the maps are allocated memory based on their area (pixel size) instead of file size. So even if you compress your image files it won't save memory. I'm glad that more people are working on this and there might be a fix in the next updates.

It's quite normal yes, big images will take some space in your RAM whatever you do. 500mo is OK for a modern game, most machines have at least 1go of free RAM available so we just need to make sure we don't go too close to 1go.


And, about image-size in RAM:


Take for example a 2048x2048 image with nothing much in it (let's say only a flat black color), if you save that image in a normal 32bits PNG to be compatible with MV (a lossless compression, but still a compression) you will have a very small file, since the redundancy is very strong and PNG files optimise that.


But this image will eventually be "converted" in your RAM as something way bigger (and indeed it will take the dimensions into account) and the size will be more like the size of a BMP version of your image, so if you want to have a good idea of the size of a PNG image once in the RAM, simply save it as a BMP and check the size. (or if you have photoshop you can check the information box)


Anyway, even a big map with 2 layers (because we need at least another layer on top of the ground one for stuff like trees etc) is not supposed to reach the RAM limits of an humble machine, but of course if it stays in RAM forever you only have to tranfer to several maps like this to kill your game. With a little time, this issue will probably be fixed.

 

[brandos]

Got very hyped to see the 1.1 news but disappointed that the problem is still here. Maybe its a problem that enterbrain can't fix since they are quite silent about it.

 

[Ashouse]

You might want to recheck the chocobo fix. They released a version 2, I'm running it and it solved the problem plus I'm sitting here looking at a random move route that is functioning normally. I switch maps and it does what it's supposed to.

 

[doranikofu]

It's quite normal yes, big images will take some space in your RAM whatever you do. 500mo is OK for a modern game, most machines have at least 1go of free RAM available so we just need to make sure we don't go too close to 1go.


And, about image-size in RAM:


Take for example a 2048x2048 image with nothing much in it (let's say only a flat black color), if you save that image in a normal 32bits PNG to be compatible with MV (a lossless compression, but still a compression) you will have a very small file, since the redundancy is very strong and PNG files optimise that.


But this image will eventually be "converted" in your RAM as something way bigger (and indeed it will take the dimensions into account) and the size will be more like the size of a BMP version of your image, so if you want to have a good idea of the size of a PNG image once in the RAM, simply save it as a BMP and check the size. (or if you have photoshop you can check the information box)


Anyway, even a big map with 2 layers (because we need at least another layer on top of the ground one for stuff like trees etc) is not supposed to reach the RAM limits of an humble machine, but of course if it stays in RAM forever you only have to tranfer to several maps like this to kill your game. With a little time, this issue will probably be fixed.

yeah looks like if the cache is cleared the memory is 600-700mb max even with maps over 4000px. It should be enough for most purposes but maybe a little hard for some mobile devices. The other way to cut down memory use is to use events to make the overlay layer, this would reduce the file size quite a bit for simple maps but just too tedious to make because you will need to align the sprites manually.


So I assume there won't be change in the way MV is handling the RAM when loading imaging files (i.e. it will always need RAM like the size of BMP)?

 

[brandos]

You might want to recheck the chocobo fix. They released a version 2, I'm running it and it solved the problem plus I'm sitting here looking at a random move route that is functioning normally. I switch maps and it does what it's supposed to.

Sadly version 2 also doesn't solve the problen. The engine still saves parallax maps and binded Picture in the cache until the RAM is full and the PC freezes or the game shut downs. Meanwhile I swapped back to VX Ace, I can't abstain from parallax maps, they are a must-have for me.

 

[Ashouse]

Sadly version 2 also doesn't solve the problen. The engine still saves parallax maps and binded Picture in the cache until the RAM is full and the PC freezes or the game shut downs. Meanwhile I swapped back to VX Ace, I can't abstain from parallax maps, they are a must-have for me.

They are for me too. I have considered switching to ace, but i dont own it and would have to pay. Especially since i decided to only aim at pc for now. I only had 4 maps to test on, so, i couldnt really confirm, sorry.

 

[Chocobo]

The engine still saves parallax maps and binded Picture in the cache until the RAM is full and the PC freezes or the game shut downs.

This is right - the RMMV loads bitmaps (parallaxes, pictures, ...) in such a way that they are cached. Thus, as you probably know, if you step onto a map with a parallax the first time, it takes a while before the parallax is loaded from your hard drive; but if you step onto it the second time, its displayed immediately, since it's still in the RAM and does not require loading. The problem is, of course, that this is no nice behavior for a game with big and/or many parallaxes.


It shouldn't be hard to create a plugin that works in a fashion that no parallax - or only the most recently used one/two/three/... - are cached. I just semi-successfully experimented with this; however, while this alleviates the memory issue and the RAM consumption is reduced, it does not fix the problem. This seems to be an issue with the underlying techniques of Bitmap.load. When I create an event that loads five bitmaps and instantly after this "forgets them", the RAM usage is still notably higher than before the loading. This memory leak could stem from PIXI.js, or from NW.js (and subsequently, more likely, from node.js or Chromium itself); a post from @Archeia makes me think it's a node.js issue (http://forums.rpgmakerweb.com/index.php?/topic/56679-possible-memory-leak-when-exiting-menu/&do=findComment&comment=554749).


The issue my plugin solves is another one; without it, transferring between parallax maps or opening/closing the menu on a parallax map would leak memory, depending on the frequency and parallax size, and eventually lead to freezing your computer. Or simplified: If you have a small number of big parallaxes, with the plugin, you can safely transfer between maps that display them and safely open/close the menu as many times as you like; without it, you will crash your computer after a certain time.

 

[Ashouse]

So if the game was made entirely of custom tilesets, do you think this would still be an issue? I dont HAVE to have parallax maps, really. My personal problem is default mv (and vxa) graphics in general. Since im no good at tilesets (and dont want to spend the extra months learning and designing them) parallaxing was a natural choice. Now I wonder if I can dredge the creative commons for something that will do it for me. The problem being if they ripped something and i get stuck with it... I do have commercial interests... Oh woe is me. Lol

 

[Chocobo]

@Ashouse:


At the current moment, with the bitmap loading memory leak issue, using parallaxes as maps would be like gambling. In my experiments (RMMV caching disabled), the memory leak appeared, at most, once for each bitmap. For example, at the start, my RAM usage was 4 GB. After loading some big images, the RAM usage peaked to 5,5 GB and then fell to 4,7 GB, although it should have fallen to 4 GB. But when I loaded the same images again, it again peaks at 5,5 GB and then drops to 4,7 GB (by the way, without disabling the RMMV caching, it would peak at 5,5 GB and stay there). So, if you use big parallaxes plus my plugin, in the worst case when the force is not with you, the player's computer would need enough RAM to store all parallaxes at the same time in it due to the memory leak - so the parallax approach would only be viable at the current RMMV state when you know that the player has tons of RAM or if you only use a few parallaxes, the smaller, the better. Thus I strongly advise against it at the current state //rmwforums.s3.amazonaws.com/emoticons/default_biggrin.gif


A parallax for each map means loading time at the start of each new map, the bigger the parallax, the longer the loading time. And every image needs space which makes your game bigger. Furthermore, you'd need pictures for the upper layers - if you want to walk behind the tree, this tree can not be on the parallax image. Again, more space and loading time.


The tileset approach does not have these disadvantages. The map drawing works in a way that the visible tiles are copied from the tileset bitmaps onto the screen. These bitmaps are kept in memory with caching, so they need to be loaded only once (when the player steps onto a map, its respective tilesets are loaded if they are not cached yet). Furthermore, the storage space needed is only that of the tilesets.

 

[KaYsEr]

Sadly version 2 also doesn't solve the problen. The engine still saves parallax maps and binded Picture in the cache until the RAM is full and the PC freezes or the game shut downs. Meanwhile I swapped back to VX Ace, I can't abstain from parallax maps, they are a must-have for me.

Same for me too, and spamming the main menu is still doing the same thing, filling the RAM until it's too much to handle.


This even happen with the default parallax function from "native MV" (even a new project without any plugin, simply using RTP-parallaxes).


Swapping back to VX ACE is kind of an "extreme solution" though, since this will likely be fixed before you finish any "big game" on VX ACE, this problem is so serious it's impossible to remain this way forever. (I heard it's a problem related to NODE.JS and that the next version will hopefully be ok, and indeed the message from @Archeia also goes in that way)


I think we can still create our games with the actual "broken MV"(as developpers we restart the game all the time, tbh I didn't pay attention to this problem, but it's also because my PC has a huge amount of RAM for 3D rendering) and be fine until the problem is fixed on the players side.


What is pretty strange is that some people said some fixes helped, but maybe they are talking about the canvas-mode. What I can confirm is that I have this problem on 3 different PC in Web-GL mode, the weakest one only has 2go of RAM and therefore crashes quickly after several transfers on a medium-sized 2 layers parallax-map (40x40).


I tried every "cache-fix plugin" available (even the japanese one or the cache manager from @Shaz) with no difference, the only one that was nearly OK is the one from this topic, but there is still a glitch with moving events so... Not usable in current state.

It shouldn't be hard to create a plugin that works in a fashion that no parallax - or only the most recently used one/two/three/... - are cached. I just semi-successfully experimented with this

That would be so great yes. Well, I just hope this issue will be a thing of the past soon.

 

[Krimer]

fix from @Chocobo for me works fine. I add huge parallax image (4142x3249 px, 38 Mb) and try to switch maps and abuse menu, no RAM leak for me. But after i turn off Chocobo fix and try to do the same things RAM died really fast.


 

 

[KaYsEr]

Are you in WebGL mode or Canvas (press F2 in-game to see). I'll try again on a fresh project with the fix from @Chocobo just in case. Maybe there is a conflict with another plugin.


(can you tell us about your computer/windows version... Remember you have to use the parallax function from MV or the plugin from @Galv for example, not the one from @Kaus which is OK with the RAM, even if after a long while I start to see some huge amount of data in RAM with that one too, but so far it never crashed any machine.)


It takes some time on my big computer to freeze (with regular parallax function), first I see the 3go of video memory in my GPU rising quickly (use a tool like GPU-Z to see it), then when it's full I can see my regular RAM rising until there is no more.

 

[Chocobo]

@KaYsEr: I just tried to set up a new project with no other plugins and move between two maps, each of them having a different circa 4000x4000 parallax on it. I can move between them and open/close the menu as often as I want, my PC does not freeze (although, I admit, when looking at the RAM consumption, I cannot always explain what is happening; there are some rises when there should be drops, but it never goes that far up that my machine crashes (8 GB, Win7, GeForce 640 GT,M i5 3210M, using WebGL btw)).


As I stated in my post before, it does not change the RMMV internal cache management (since there seems to be an issue, probably with the PIXI version and that would not make the fix worthwhile at the moment; you can see the discussion with Ashouse on my profile for more info).

 

[KaYsEr]

@Chocobo


I just tried on a new project without anything, not even the parallax-plugin from @Galv, and it worked.


So, it seems there is "simply" an incompatibility with this plugin from Galv. Should I see with him or can you add the "support" for it?


Or would it simply be better to wait for the new PIXI or NODE.JS maybe... Anyway, I'm grateful you are addressing the problem, even if it's just temporary.

 

[Chocobo]

Yeah, waiting for the new versions would probably be the best idea, as my fix wasn't really meant as a plugin – it was rather meant to be seen as "pointing the devs into the right direction", since I found out the little fix but do not have the time or capabilities to test it on a larger user base or to dig much deeper If the new versions are incorporated and the problem is not fixed, I'll probably have another shot at it. Apart from that, for the time being it would be a good idea to take this problem to @Galv since my fix is not really that complicated and should be easy to incorporate (although I do not know much about his plugin).