Asset/Plugin Repository?

[deusprogrammer]

Hey guys. My name is Mike, but my friend call me Deus. I am a professional developer and I have loved toying around with RPG Maker VX for years. I just got RPG Maker MV and I have already written a couple scripts for it, but I really want to give something back to this community (other than my scripts that are probably very niche). One thing I have noticed is there isn't really a tool out there for managing Plugins, Tilesets, Sound FX and other Assets.

I am going to be writing this tool, but I wanted to gauge interest in it before I waste several months writing it.

The idea is to create some similar to Maven for RPG Maker plugins. Maven; if you are not familiar, is a tool that programmers use to handle builds and dependencies for projects. Essentially you just tell Maven "I need library x", and it retrieves it and adds it to your project for you. And the real kicker is if library x for example requires libraries a, b and c to work, it will also retrieve those as well as any dependencies that libraries a, b and c have.

Basically what I want to give the community is a separate application (available for free) that would allow this kind of functionality for RPG Maker Plugins as well as actually telling you if two plugins might conflict with one another. The other portion of the project would be a web service that would sit in front of the repository and allow Plugin Developers to upload new plugins and manage existing ones. A REST API would be provided for other developers to make use of it however they please.

As a bonus it would serve as a repository for all kinds of other assets including tilesets, sfx, bgm, artwork, and full projects. The full projects would be presented as a web page (since that's all the RPG Maker MV projects are anyways), so all of your games would be playable from this one location. And then if I got some other developers to help out, we could build an entire community around it.

All I need is a little bit of interest from the community and I will get started immediately. For the early versions of this project I would be hosting them myself on my small home server farm. Once it took off we would have to find a permanent hosting solution and probably start collecting donations.

Anyways, who's with me?

Update: the preliminary web service project is at https://github.com/rpg-maker-repo/rmmv-api.  Currently I just have the functionality there for storing/managing plugins and their dependencies.  The next step (other than adding security and finishing the other features), is to deploy this on my home server farm and to start writing the client app and the UI for the web service.

 

[GaryCXJk]

I think, if executed correctly, this could be good, and, seeing as RPG Maker MV plugins use a certain format for their information (description and help) it's easy to implement said information without requiring the scripters to enter all this information manually.

You could possibly even implement your own data for specific information for the repository, such as license and contact information, which can just be added to the regular block, as unsupported parameters are ignored.

 

[deusprogrammer]

That's a fantastic idea!  And parsing out those comment annotations would be a cinch!  I actually just wrote a jQuery consumer for my web service now, so soon I will have a UI for the web service and I will then get it deployed so people can play with it.  After the UI I will write the JAX-RS consumer jar for my web service and start working on the Project Manager.

 

[deusprogrammer]

The REST API is mostly done with the exception of security.  And I have built a temporary angularjs ui around it as you can see below.  Next I will work on the fun part of making the project manager.  And work on writing a jsdoc parser so users can just upload their file and it will automatically fill in the details.

 

[deusprogrammer]

Just another image to show the dependency list, since I had the script showing last time.

 

[PeaceBrigade]

I think the most useful tool would be akin to Curse Client or some such.

If you're not familiar with it it's a addon manager for games, you can just browse and click "Install" to install addons for games, or if there are updates you can just hit the update button to update to the latest version of the addon.

This would be great plugin wise as you could just update all your plugins at once, see updates and changelogs etc.

This is something I'd be willing to either work on, or at minimum provide hosting and all that jazz for. 

I guess another program to compare it to would be like NexusModManager or something like that. A useful website in combo with a plugin manager utility would be a dream. Let me know if you're interested or wanna shoot ideas around. At minimum though like I said I can completely provide hosting, I have a couple VPS servers currently laying around doing nothing.

 

[deusprogrammer]

What you are describing is exactly what I'm aiming for =).  I would love to have some assistance if you would like to help.  I already have all three projects started (REST API, JS client, and Java Client).  I have also started working on the Plugin Manager.  But so far I haven't really sat down and planned the UI for either.

Are you good with UI work PeaceBrigade =)?  I could certainly use someone to help me design wise, or with whatever else you would like to help with.  I can code just about anything.  Also feel free to get the source code from my github.  I can help you get a workspace set up.

 

[deusprogrammer]

As for a server, would we be able to get Wildfly 8.2.1 and a MySQL database running on your virtual machine?

 

[deusprogrammer]

Organization on GitHub is here:

https://github.com/rpg-maker-repo

 

[deusprogrammer]

Started experimenting with the REST Java Client.  Still need to plan the UI for both the Project Manager and the Web Interface for uploading plugins and managing dependencies and such.  If anyone wants to help with UI design, that would help a lot.

 

[PeaceBrigade]

As for a server, would we be able to get Wildfly 8.2.1 and a MySQL database running on your virtual machine?

Yes. Already have databases and all that, can grab and set up Wildfly in the morning. (It's 2am here hence the slow reply and me about to go to bed.)

Go ahead and hit me up on a private message. If you have Skype or something similar we could chat a bit about it, or IRC etc. (I run an IRC server on a decent sized network)

I'd love to jump in and get started helping tonight, sadly was a long halloween night and I'm completely tired out. I'll certainly help out and we can get things cracking and all that, like I said hit me up in a private message and we can figure out some of the logistics and requirements/setup server wise and all that.

(I just noticed your steam name in your sig, I added you there.)

 

[xhh2a]

I sure hope it is more like npm rather than maven.

I can probably hack together something nice within like 2 days if I wasn't working on a custom battle engine, a startup for work, and my own startup on the side....hm.....

I would recommend heroku + couchbase + node.js instead of MySQL. NoSQL can scale much better than SQL, and given plugins are in javascript, NoSQL is much better suited for JSON objects.

You can make a very nice express server with jade templating and stylus css, along with a great REST interface.

 

[deusprogrammer]

I wouldn't mind making a package manager like pip or npm, but I would rather have a nice UI that even non-developers would feel comfortable using.  Besides that, I have plans to allow for other asset management too, like tilesets and sounds.  And I feel like a GUI would help that more.  I could certainly wrap the REST api with a CLI though to give developers a more streamlined experience.  The REST API is already done with the exception of a few database modifications, so if you wanted to help out by writing the CLI, we could certainly use that.

The back end is written using Java EE 7 using JAX-RS for the RESTful Web API and Hibernate/JPA for the ORM...I know that NoSQL can work with JPA...but isn't NoSQL a non-relational database?  The database is configurable as a datasource on the Application Server.  I wouldn't mind using Node.js, but I have already started writing it in Java...because that's what I know best.

Would you be open to helping even if we continue down the path we are heading down.

 

[deusprogrammer]

Honestly I have never had the opportunity to use NoSQL in my career.  From what I understand though, I can see how couchbase would lend it self to storing scripts themselves.  But due to my lack of understanding of NoSQL I'm not sure how I would be able to manage the kind of searching and object hierarchy I am imagining.  I apologize for my ignorance in this matter.

Personally, I don't see our project drawing that kind of volume...the REST service on it's own would be plenty flexible and scalable.  If we needed more nodes they could share one data layer.  But if need be we could spread it out across multiple servers...not as easily as NoSQL I understand.  I'm no database guru though.  Just a motivated programmer.  

I would love to chat though if you want to.  I would love to hear your ideas.  From what I understand, JPA can work with NoSQL.  I would just need to figure out how it would affect my ability to search.  Even though, right now my searches are relatively basic (searching by author, plugin name, search, tags, file hash, etc).

 

[xhh2a]

Java is the WRONG language for anything REST. Custom JSON serialization and deserialization logic required for Java is all kinds of pain (yes it's doable, I have to maintain an Android app in Java).

NoSQL just means anything non-SQL. It could be a graph database like Neo4j, it can be a document store (perfect for JSON) like Mongo, it can be a document store with mem cache on top like Couchbase, it can be a quick k,v store like Redis. I haven't used Couchbase myself, would be a great time for me to learn it.

I'm not very good with bash (CLI) writing so I probably won't be able to work on that. I'm more than happy to work on a UI written in node (writing frontend in Java is also hilariously bad).

I have servers in Azure and Heroku I can contribute (or we can just use some free Heroku servers), Heroku has lots of free database providers.

 

[deusprogrammer]

I would have to disagree.  Java is great for REST services.  Serialization and Deserialization of data is simple with JAX-RS.  You pass a POJO wit the JAX-RS client and you get a POJO with the JAX-RS server.  The only benefit you would get from Node.js is the fact that Javascript isn't strictly object oriented, meaning that objects can easily add and remove fields from an object without having a strict structure to follow like in Java.  I don't believe that makes Java WRONG as you put it.

I originally went into a rant here asking why you were being so passive aggressive and essentially calling my design ****.  I have since calmed down.  Let's try this again.

I mean, man...I'm not trying to be a dick here.  And I love learning more than most people.  But you are just coming off as extremely abrasive.  Tell me why you think something is bad, so I can possibly learn from it.  Passive aggressive insults don't help me grow as a programmer...they make me never want to ask for help or opinions ever again.  So man...help me here.  Don't be so judgmental.  I have been a Java Web Developer for 10 years.  I am using the tools I am most familiar with.

So starting over...let's be nice to one another.  Tell me why implementing a UI in Java using Swing/AWT is such a bad idea.  I wasn't aware that node.js was capable of UI.  The reason I chose Java is because it is portable and I wasn't aware of how portable Node.js could be, since my experience in it is limited.  When you say that Java is a terrible choice for REST, are you referring to consuming REST or serving RESTful resources?  I think it does a great job of acting as a RESTful web application server.  Client wise, sure...JAX-RS isn't great...but it does well enough.  Let me know your thoughts.  Let's have a conversation.  Not an argument.

 

[deusprogrammer]

And as a show of good faith, I am going to experiment with Couchbase.  Because I designed the REST service with SOA, I can easily swap out the data layer with something different.  Such as a NoSQL document based strategy.

 

[deusprogrammer]

Messed around with Couchbase.  That's pretty slick.  I love how easy it is to cluster.  I only wish there was a JPA driver for it.  I might just rewrite my data layer to use it.  Document based NoSQL really does make sense for this task.  I can also see why you would use Node.js for the server since it natively uses JSON as Javascript does.  But I am still going to use Java.  I need the power and reusability that I can only get with an application container, so I will stick with Java.

Looking into using Node.js for a UI doesn't seem like a terrible idea since it essentially uses a Chromium browser to display the UI.  This doesn't seem all that portable to me, so I am going to stick with the Java Swing Thick Client.  The API is written in such a way that one could easily use any language to write a thick client for it.  My Swing implementation is just going to be the first one I do.

I have started writing the code to populate the local side of the plugin tree using the plugins.js and a directory scan.  What I could use, if anyone is up to it, is to find a good library for Java and Javascript for reading JSDoc into a map so we can use it to allow batch uploads of scripts (basically just autofill in information based on the JSDoc info), and to autofill the form on the web interface based on the JSDoc in the uploaded file.

 

[deusprogrammer]

Still working on it.  I just updated the database schema, web service and ui to account for versioning of plugins.

 

[deusprogrammer]

Sorry for the pause between updates guys.  I was out on vacation in Virginia.  I did a little work though and refactored a bit.  I now have the thick client able to scan the project for plugins (which is able to identify them by their hash on the REST service).  I also updated the UI and Web UI to handle versioning.

The items in red are the ones that were scanned, but the service couldn't identify them based on their hash.

This just shows the ability to look at different versions of a plugin (keep in mind this web ui is just temporary...it is just a test harness...the real web ui will look much better).

This is just an image of the test harness for creating a new plugin and adding dependencies.

This is just showing the functionality of creating a new version of a plugin.  It will show all of the dependencies for a plugin as you select different base plugin values.  Once again...just a test harness.  So...who wants to help design a UI for the web client and the thick client?