Frequently data loss - unable to open rpgmaker file(s)

[JoeRPGMAKER]

Constantly having the problem where a file or something can't open or doesn't work. Started with the trial version of rpgmaker mv a week ago. Have touched it every single day. Windows 10 new laptop. I have had to restore four times and lost hours of work. How do we fix this long term? I can't keep fighting this every couple of day. And having to backup an entire folder manually every day is just moronic and kind of second class for a production product. Is there something I am missing? The only other thing I do with the laptop is browse websites. No other apps not even office.

Typically the files that won't work or open start with Game.PRGMV
Then it varies but usually lots of .json files Mapinfos, Skills, States, Items, Enemies, just about anything... in the json directory
And in addition sometimes plugins.js

Any thoughts would be appreicate? Really like the program otherwise.

 

[bgillisp]

To avoid this issue, you *must* shut down windows properly. If you close RPGMaker then close the computer without doing a shutdown command, then you just turned off your computer in the middle of saving (as saving a database takes minutes and is done in the background). That is what causes the corrupt file issue.

I usually recommend closing RPGMaker about 15 minutes or so before I turn off the computer myself. That gives the OS plenty of time to finish saving the database, and avoids data loss.

The other possibility is a HD that is going bad. If the project keeps trying to save to sectors of the HD that are failing, that will cause data loss. That is usually more an issue though with older HD's, though HD's can fail in as little as 3 months of use (if they were badly designed or you just got unlucky).

Also, back up the project often.

 

[Andar]

This is not caused by the programs, but by something you do or something on your computer.
So to solve this, we need to analyse what goes wrong on your computer, because most other don't have that problem.

Basically there are three possible causes for this, and you need to check which one is the reason on your computer.

1) poweroff without completing shutdown.
If your harddrive looses power before all files are transfered from cache to magnetic storage, any open file will be damaged.
This can be caused by an already damaged Windows, by the user cutting oft power manually (including resetting) or by the harddrive being old and near end of its life cycle.

2) working on the project while it is stored on a cloud.
Forget that option if you do this and move the project data outside the cloudspace.
It doesn't matter which Internet cloud you use, the cloud Synchronisation will always interrupt the maker access and destroy the files you're working on.
That is because the RPG Makers need to access their data in the way a database handle this (permanently open for writing), not like regular office programs which only open on save.

3) a background program damaging your projects
This could be an AntiVirus falsely detecting them as viruses or a real virus trying to infect your project or something like that.


Most cases of constantly being damaged projects are cloudstorages however, so can you tell us the exact filepath where you store your projects?

 

[JoeRPGMAKER]

This is not caused by the programs, but by something you do or something on your computer.
So to solve this, we need to analyse what goes wrong on your computer, because most other don't have that problem.

Basically there are three possible causes for this, and you need to check which one is the reason on your computer.

1) poweroff without completing shutdown.
If your harddrive looses power before all files are transfered from cache to magnetic storage, any open file will be damaged.
This can be caused by an already damaged Windows, by the user cutting oft power manually (including resetting) or by the harddrive being old and near end of its life cycle.

2) working on the project while it is stored on a cloud.
Forget that option if you do this and move the project data outside the cloudspace.
It doesn't matter which Internet cloud you use, the cloud Synchronisation will always interrupt the maker access and destroy the files you're working on.
That is because the RPG Makers need to access their data in the way a database handle this (permanently open for writing), not like regular office programs which only open on save.

3) a background program damaging your projects
This could be an AntiVirus falsely detecting them as viruses or a real virus trying to infect your project or something like that.


Most cases of constantly being damaged projects are cloudstorages however, so can you tell us the exact filepath where you store your projects?

#1 I am not sure is an issue. I have altered the power saving settings to never shutdown. Just because I thought that could be a problem at first.
#2 seems very unlikely. Unless there is some new default Microsoft cloud storage I don't know about.
#3 I am not sure I know a way to monitor this. Is there some software I could download to monitor this. Or something I could do. It could be the issue, but was wondering if you saw a way to do this. I guess I can start pouring through task manager, but any input from you would be great! Since the machine is brand new, there could certainly be something like that.

The difficult with diagnosis is it work well 3 our of 4 uses.

 

[JoeRPGMAKER]

To avoid this issue, you *must* shut down windows properly. If you close RPGMaker then close the computer without doing a shutdown command, then you just turned off your computer in the middle of saving (as saving a database takes minutes and is done in the background). That is what causes the corrupt file issue.

I usually recommend closing RPGMaker about 15 minutes or so before I turn off the computer myself. That gives the OS plenty of time to finish saving the database, and avoids data loss.

The other possibility is a HD that is going bad. If the project keeps trying to save to sectors of the HD that are failing, that will cause data loss. That is usually more an issue though with older HD's, though HD's can fail in as little as 3 months of use (if they were badly designed or you just got unlucky).

Also, back up the project often.

I appreciate this. But shutting down and leaving windows open for an additional 15 minutes seems unlikely to be a solution. I can't ever recall having a problem with any other document editing sofware that required this. And why is the OS saving the database, 15 minutes later... when you hit save project what happens?

 

[bgillisp]

Actually, it's ALWAYS been a problem, just rarely shows up. I had this issue back in DOS in 1993, so it has existed for ages. The issue is saves are not automatic, they are queued in the system, and done when the OS has some free time. If saves were done automatically the engine would appear to freeze for minutes every time you hit save (and this would be true for video games too. Imagine every autosave hanging the game for minutes in your favorite video game), hence why it is done this way.

In DOS there was a command to force all disk write operations in the system to complete now. Windows is supposed to do this when you hit shutdown, but windows 10 has that fast shutdown deal which is probably going too fast for the disk save. Maybe there's an OS command to force all saves to finish, but the fact you haven't hit it yet means you are VERY lucky, as I've even had it hit word documents and had to restore the automatic save.

Edit: Note the 15 minutes to wait is just a recommended time by me. Disk writes shouldn't take that long unless you got a ton going on in the background.

 

[Andar]

I can't ever recall having a problem with any other document editing sofware that required this. And why is the OS saving the database, 15 minutes later... when you hit save project what happens?

1) but as I said above, the RPG Makers are NOT document editing softwares, they are databasing softwares. You HAVE to shutdown properly, you can't ignore the shutdown procedures.

2) when you hit save in all programs, no matter which, the data is transferred into the HDD cache (which is fast, but like all RAM erased on poweroff) with a command to transfer that data to the magnetic disc (which takes a lot more time than cache access).
Part of the Windows shutdown procedure is for Windows to wait until it received the all-finished from the HDD before cutting power. Ignoring that and cutting power without regular shutdown will cause the cache data to be lost.

You don't need fifteen minutes for that unless you have a really old HDD, but depending on the amount of data (and RM projects can be large) it can take one or two minutes even on the fastest and most modern HDD.
This is especially the case if the HDD was improved with a large cache for fast direct operations, because those HDDs will need more time for shutdown, that is the price they pay for faster direct speed.

So if you disabled Windows shutdown to speed up the power off procedures, that is the reason why your RM projects are offen damaged.

 

[JoeRPGMAKER]

Actually, it's ALWAYS been a problem, just rarely shows up. I had this issue back in DOS in 1993, so it has existed for ages. The issue is saves are not automatic, they are queued in the system, and done when the OS has some free time. If saves were done automatically the engine would appear to freeze for minutes every time you hit save (and this would be true for video games too. Imagine every autosave hanging the game for minutes in your favorite video game), hence why it is done this way.

In DOS there was a command to force all disk write operations in the system to complete now. Windows is supposed to do this when you hit shutdown, but windows 10 has that fast shutdown deal which is probably going too fast for the disk save. Maybe there's an OS command to force all saves to finish, but the fact you haven't hit it yet means you are VERY lucky, as I've even had it hit word documents and had to restore the automatic save.

Edit: Note the 15 minutes to wait is just a recommended time by me. Disk writes shouldn't take that long unless you got a ton going on in the background.

Appreciate your help. But I have been programming and editing for that long as well. And I really can't ever recall that kind of problem consistently except maybe across a network with SAMBA or something like that. And I have worked in programs that pause a little when saving. Again problem more of a network thing. So I am guessing it has something to do with windows 10 some sort of default or 3rd party add on, that deletes or removes files. Could there be a setting maybe that forces windows 10 to save before shutting down?

 

[JoeRPGMAKER]

1) but as I said above, the RPG Makers are NOT document editing softwares, they are databasing softwares. You HAVE to shutdown properly, you can't ignore the shutdown procedures.

2) when you hit save in all programs, no matter which, the data is transferred into the HDD cache (which is fast, but like all RAM erased on poweroff) with a command to transfer that data to the magnetic disc (which takes a lot more time than cache access).
Part of the Windows shutdown procedure is for Windows to wait until it received the all-finished from the HDD before cutting power. Ignoring that and cutting power without regular shutdown will cause the cache data to be lost.

You don't need fifteen minutes for that unless you have a really old HDD, but depending on the amount of data (and RM projects can be large) it can take one or two minutes even on the fastest and most modern HDD.
This is especially the case if the HDD was improved with a large cache for fast direct operations, because those HDDs will need more time for shutdown, that is the price they pay for faster direct speed.

So if you disabled Windows shutdown to speed up the power off procedures, that is the reason why your RM projects are offen damaged.

Unless Windows is set out of the box to do something like not save on shut down, I can't see how it is ever shutting down that quick. Hmmm....

 

[gstv87]

the only critical failure I've ever got (the first one) happened when the computer shut down due to overheating while compiling a new game.... and that process usually takes 10 seconds!
I adjusted the temperature controls, and haven't had that problem anymore.
I'm running a Pentium 4 from 2006.
you can't possibly tell us it takes you fifteen minutes to save a project with a modern windows 10 computer!

my money is in an antivirus program locking the files for analysis, since they're .js and they could be blacklisted.

 

[bgillisp]

Sheesh everyone, 15 minutes is just a rule of thumb I use. And you're all missing the point! Windows does NOT save instantly. That's the point. I've hit this on word documents when I've closed word and then shutdown even. It's been around forever, but it's not often hit as usually most people don't save a critical file then shutdown right away.

And yes, I mean forever. Back in the DOS days you used to when you closed a program have to wait for the floppy disk light to go off to safely remove the disk. Fail to do that and you can get a corrupted file. And sometimes that took a little time once you closed the program before it was safe to do so, how long depended on your floppy drive and how big the file was that was still being saved.

PS: If you wish to see what I mean...get a flash drive and copy a few files to it. Even copying my project to it takes a minute or two. Or you can do a 6 GB file...that was about 1 - 2 hours. Sure, it is slowed down by the USB port (and who would save 6 GB at once?), but you'll get the idea. And, if you remove it too soon...bye bye file (do not actually do this though unless you have a flash drive you don't care if it never works again).

 

[JoeRPGMAKER]

@bgillisp @Andar Sometimes RPG is left open on the counter. And ignored for a little bit. I wonder if somehow that could be a problem. Would it be advisable to always save and close immediately? A lingering open window could be problematic, but not sure how. Any recommendation on shut down process? Or maybe what do you would be a guide?

 

[bgillisp]

Hmmm....no idea honestly. I used to leave it open all the time with no issues, but I also have my laptop set to never shutdown unless it loses power (as I leave mine plugged in). I'm still learning more about this Windows 10 fast shutdown thing myself too, as I just got a Windows 10 system myself about 3 months ago.

The only thing I could think of that could cause an issue is if you made a change, didn't hit save, then left it open and ignored it. Then it wouldn't send a save command through and the database would be open (due to the edits) when the computer goes to sleep. And honestly I have no idea how windows handles that...though I'm guessing not well as that is how I got a corrupted Word Document back in Windows 7.

 

[Shaz]

What is the path to your project?

If you store it anywhere that syncs - dropbox, steam, anywhere else, that is pretty famous for corrupting files.

If you store it in My Documents or any other part of the Windows universe, that is also dodgy. I have had windows updates revert my files back to older versions and have lost days of work in the past.

I suggest you create a new folder in the root directory of whatever drive you're using and keep your projects in that. Don't even store them in the MV install folder.

I didn't see if you answered the question about your hard drive - when's the last time you did a scan?

The fact that hundreds of people here are using MV and you are the only one who currently has this problem indicates it is something to do with your setup - either your hardware or your software.

 

[JoeRPGMAKER]

What is the path to your project?

.

My Documents/Game... which I think is the default. I suspect people that have this problem in trials mostly just bug off. But I agree there must be a plethora of people out there who use it all the time without a problem, so there has to be a way to do it. There must be just some small stupid windows new installation thing wrong or some small way we use it that is wrong. It just works more often then not, but losing the time is a killer!

 

[Shaz]

There is no difference between the trial and the full version. If people have this issue with the trial, they will also have it with the full version.

I suggest you move your project somewhere else. Even though it shouldn't, Windows messes with files in My Documents. I'd put it far, far away from any folder that has anything to do with the OS. Found that one out the hard way myself.

I don't think that's your entire problem, though. Maybe once or twice, but not every day.

 

[JoeRPGMAKER]

@Shaz will keep trying. I am going to keep tinkering and hope to get this solved for me and other users with similar problems.

 

[Shaz]

Please do move it to another location. You may find all your issues suddenly stop. Leaving it where it is is inviting trouble.

I still didn't see if you answered the questions about your hard drive. How old is it? How full is it? When is the last time you did a scan or a CHKDSK? To be losing data every day there is something going on there.

 

[bgillisp]

Here's details on how to do a chkdsk in windows 10 if you need it. I ahd to do it via the command prompt myself, but you might have better luck:

https://windowsinstructed.com/run-chkdsk-windows/

And yeah, move it from my documents too. Personally, I never used it but I come from the pre-windows days where you had everything backed up in triplicate if you cared about it.