Forum deleted 99% of my post...

[SilverDash]

http://forums.rpgmakerweb.com/index.php?/topic/51307-silvers-advanced-minimap/


It's all gone.. Any chance for restoring a backup? 'Forum outtage' caused data-loss here.


I kinda need the original links to the old versions as well as the screenshots and the list of features at the very least.


Oh wait I can use Google Cache to restore it myself

 

[Andar]

I't neither deleted nor gone - the topic is still there and viewable.

It's just that the forum server/host is instabil yesterday and today, and until the problem is fixed your computer has problems loading the entire topic

EDIT: unless you're talking about the edits you made yesterday to the first post.

In that case, I would try the wayback machine (it's a site that automatically stores old websites by date - search for it, I don't have the link and don't know if there are limitations, but I know others have used it for forum posts here)

 

[SilverDash]

Yes the forum kinda 'crashed' on me while posting an edit and thus it only saved 2 sentences and deleted everything else (for everyone). Luckily using Google cache I was able to restore everything.

I kinda expected some kind of transaction-design in the forums that simply wouldn't finalize the edit until everything is processed/received. But it seems not. The forums just delete everything if something goes wrong while posting an edit. This could use some improvement.

Thanks though.

 

[Andar]

As I said in my edit, search the internet for the wayback machine, it might be able to help

 

[SilverDash]

I did read it and I also already fixed it. But it's still a bug in the forums nonetheless.

 

[Andar]

No, it's not a bug in the forum software - it happened because the transmission was broken due to the server problems.

Forum software with version history would make the forum basically either unusable or require a server that has a hundred times the storage capacity - only wikis and similiar sites with low-frequency of changes can store versions

 

[SilverDash]

Yes it can be considered a bug. When the connection is lost while committing data. I expect the server to cancel the process. But instead it just applies it blindly... Causing bugs like the one I experienced. It may also be related to this bug that also caused my to duplicate my posts through editing (I edited in 2 different windows, 2 different posts and committed at the same time and for some reason one of the commits duplicated the other post into the other topic. Perhaps it also does not keep track what data belongs to what commit).

What I said is that this forum has no transaction-design (maybe the term I'm using is not accurate). But you basically only commit data to the database AFTER you received ALL of the data. And that can be checked with an extra piece of data that marks the end of a commit. But this forum does not use anything like that nor any other failsafe protection. And thus if for any reason the connection is interrupted during the commit, the post may corrupt or be partially deleted because the forum then supposes that it must be the end of the commit and blindly applies it to the database. And then you can only hope for the Google Cache or the TimeMachine to get it back for you. Which is just.... Bad? I consider this design of just blindly committing the data to the database flawed/bugged.

There will always be connection problems for many reasons. Saying it's not a bug, because it's not the forum's fault that the connection may be unstable sounds weird to me. Maybe IP.Board should look into this?

 

[Nathanial]

No, it's not a bug in the forum software. Your connection was lost to the forum and all the data wasn't captured for whatever reason. The *server* was losing connection in and out due to the DDOS. We don't have it (it's not in the database) and it's not the forums fault.


IP.Board isn't going to "fix" it even if you think it's a flaw in its design. They're working on a completely new version of the software now that I've been prepping to upgrade us to for awhile now (there's a lot of custom stuff we need to function before we can make the upgrade).


What you described ("transaction-design") isn't flawless/guaranteed either... The entire post very well COULD have been captured, but a drop in the DATABASE connection (on the servers end, again due to the DDOS) could have lost the data. There's no bullet-proof way to prevent issues like this during a network attack, period.


In the future, especially during an attack on the datacenter (not us, but Linode), backup your post in Notepad before posting. I suggest this for ALL sites, and have experienced lost posts on multiple sites (even because I didn't submit a captcha correctly for crying out loud...) and have adapted this, and so have many others. Live and learn. Sorry about the inconvenience, but there's nothing that can be done about it.

 

[SilverDash]

Ugh it seems the DDOS just launched again. I hope this is the last of it.

Anyway I found the term: "atomic transaction processing". But I see, so it's not the connection between my PC and the forum-server. It's the connection (or DDOS) problem between the forum software and the database OR purely on the database-server itself. Possibly the database got killed/crashed while it was applying my change to the database. Yeah okay then there is not so much one can do indeed. The design could be improved still to counter this but it's indeed not worth the effort when I can just back it up to notepad next time.

But I always thought that a server, even when crashing mid-transaction, was always able to rollback the changes made upon next restart. I always thought that transactions were like 99.9999% safe. I never heard a story about a transaction committing data when it shouldn't just because of a lost connection, crash, power outage or whatever. Probably because that's the reason we use transactions, to prevent partial-data from being committed to the database(s). Even banks rely on it to keep their money safe. It must be good.

So if the forum uses transactions, I do still not understand how this is even possible.

 

[Nathanial]

I never heard a story about a transaction committing data when it shouldn't just because of a lost connection, crash, power outage or whatever.

I know dozens (if not hundreds) of instances. Maybe that's just because I've done a lot of web stuff over the years.


Again, the database server and the forum server probably lost connection mid-posting which is why it happened. It's hard to say for sure because our host was going through a network attack. Weird things happen. If this happened normally, and/or was continuously happening then I would look into it a bit more/be worried about it. I don't think it's worth bugging IPS over.


You're worrying about something rather trivial. This is the first instance I've seen of a post being partially committed like this in the 3+ years of running the forum, and it was during a network DDoS, so I'm not too incredibly worried about it. Glitches happen. If anything, I'm glad this is the only issue reported so far because of it (other than the slowness/downtime which is... well, what a DDoS is intended to do).


Glad we both agree that it's not worth the effort over just copying and pasting in notepad. It's a good idea to do so anyway

 

[Kane Hart]

Why this site not on a cloud proxy anyways for the access to it? Change the IP's and get it thrown onto something like cloudflare.

 

[Nathanial]

Why this site not on a cloud proxy anyways for the access to it? Change the IP's and get it thrown onto something like cloudflare.

CloudFlare can only do so much, it would have probably been useless for the scale of attack that was happening in Atlanta and wasn't targeted at us directly. It took the entire datacenter down. I've attempted to use it in the past but ran into some difficulties related to the control of the domain.


I want[ed] to set up some load balancer's in the future though.

 

[Kane Hart]

CloudFlare can only do so much, it would have probably been useless for the scale of attack that was happening in Atlanta. I've attempted to use it in the past but ran into some difficulties related to the control of the domain.

I want[ed] to set up some load balancer's in the future though.

Yeah Load balancer's would help. With a lot of experience though Cloudflare does help a lot as long you don't have any nasty scripts going on the site for people to abuse etc. Without shorter times outs. 

You might also want to adjust this to 15 minutes or 30 even:

http://forums.rpgmakerweb.com/index.php?app=members&module=online&sort_order=desc

These tend to be quite taxing overall and your just bleeding cpu time on this. I know we use to use quite a silly large number and just be bleeding away the cpu time on it. Without a proper caching system for it and IPB/Xenforo has I think a small one but not a decent one they tend to just eat up the CPU time like candy. 

 

[Nathanial]

Yeah Load balancer's would help. With a lot of experience though Cloudflare does help a lot as long you don't have any nasty scripts going on the site for people to abuse etc. Without shorter times outs. 


You might also want to adjust this to 15 minutes or 30 even:


http://forums.rpgmakerweb.com/index.php?app=members&module=online&sort_order=desc


These tend to be quite taxing overall and your just bleeding cpu time on this. I know we use to use quite a silly large number and just be bleeding away the cpu time on it. Without a proper caching system for it and IPB/Xenforo has I think a small one but not a decent one they tend to just eat up the CPU time like candy.

I have that set to 500 minutes so it's less taxing than 15 or even 30 minutes. It's less SQL queries that way and just caches the data. At least from what it was explained to me, it's better to keep that number if you're really trying to keep resource usage down


With that said, however, resource usage isn't a super huge concern as it was in the past when we had pretty weak hardware to work with, so I could probably shorten it.


The load balancer is nice for when the site's super busy, but where it would really shine is in a situation like the Atlanta datacenter having an outage... users would just instead bounce to a server setup in a different datacenter such as Texas... or California, or even Tokyo. I have a "balancer plan" made already, I just need to get it approved and then get around to doing it... I would probably also upgrade the forums to V4 at the same time... which speaking of...


I'm DYING to upgrade, but there's been so much custom stuff I've had to do until we're ready for the upgrade. Hopefully it will be soon, as V4 kicks a lot of butt in general and has much better caching.