External Scripts Framework

[Galenmereth]

External Scripts Framework

For RM: VX Ace (RGSS3)

Version: 1.1.0

Creator: Galenmereth

Latest changes:

• 1.1.1 – Implemented the :rest symbol feature in order arrays
• 1.1.0 – Introduced the exclude option, and the load_recursive method
• 1.0.2 – Actually works properly with encrypted archives
• 1.0.1 – Works with encrypted archives

Introduction:

The Plugin Framework is a small combination of scripts for handling external script files in Ace in an intelligent and modular fashion. Through use of a simple hash to specify the folder structure, load order, and even exclusion of individual files for debugging, it's much easier to manage vast number of scripts using this framework. And through use of a few simple methods, a plugin can register its own id and check for the existence of other plugins, making inter-plugin scripting much easier.

For example, this system allows for the grouping of scripts by "vendor" (a scripter) in plugin folders, and letting users drop all extensions to a script inside a folder and have it loaded automatically. An example of this would be Yanfly's core script being a plugin folder with a subfolder of extensions, where a user puts all other Yanfly scripts. While this might not sound like an improvement of just putting scripts in Ace's script editor, it allows for much greater flexibility for scripters, as they can ensure specific order in their own script collections, check for existence of other scripts to make compatibility easier, and overall allow for more structure in bigger script collections.

Features:

• Modular plugin folder structure
• A Plugin module with simple methods for requiring files in a given path, and in a given order if specified
• Documented code
• Works with encrypted game archives

How to use:

• Look at the linked demo project for an example that includes a Plugins folder with many example plugins that show the different methods. The scripts are documented as well.
• The Plugins folder can be found in Data/Plugins
• Make sure you test your project in the maker after you make changes to script and before you make an encrypted archive; this is not only a smart thing to do in general, but is required for the script package file to be up-to-date with any changes.

Demo:

Demo does not include RTP files and comes in a zip archive; it's 1,3mb in size.

https://github.com/TorD/rgss3-plugin-script-framework/archive/master.zip

Installation:

Detailed instructions for installation can be found in the GitHub repository readme file:

https://github.com/TorD/rgss3-plugin-script-framework

License

This script is free to use in commercial and non-commercial products with credits given. See below for whom to credit; credits may be just names or also mention what is being credited for.

Credit and Thanks

– Tsukihime for the "External Script Loader" script

– Galenmereth for the Plugin Framework

Author's Notes:

I posted this in the development subforum because I want feedback on this, and I'll be changing it a lot based on the feedback I get. Since this is something that can be useful for anyone who scripts a lot (I'm using it myself for my game), its use would be much greater if more scripters where to use it for their own plugins.

Feeling generous?

If you would like to give me a small tip to help me spend even more time making and maintaining these free scripts, do consider supporting me on Gratipay. Any and all support will be greatly appreciated.

 

[Tsukihime]

Works with encrypted game archives (but does not encrypt the plugins themselves; I will implement this as an optional feature very soon)

How are you planning to do this?

 

[Galenmereth]

How are you planning to do this?

I'm not entirely sure, so I rephraced the sentence I'm looking at a few different approaches, and I'll get back to that. So for now, plugins will be exposed in encrypted archives, which limits the use somewhat, especially if people use licensed scripts for commercial games that can't be exposed to the public.

 

[Tsukihime]

Just move your Plugins folder into the Data folder and it'll get encrypted automatically.


So I would just have the user enter the path to the Plugins folder (so if they don't want to encrypt it, they can just leave it outside of the Data folder)

 

[Galenmereth]

Just move your Plugins folder into the Data folder and it'll get encrypted automatically.

So I would just have the user enter the path to the Plugins folder (so if they don't want to encrypt it, they can just leave it outside of the Data folder)

That's what I ended up doing, but to get it to work, I had to use the load_script method instead of require, as require can't find the files at runtime when called in external scripts inside encrypted archives  Due to the bootstrap files, this ended up causing an infinite loop of files loading themselves for some reason, so I had to put in some checks for this. But now it works with encrypted archives, and I've updated the demo file with the latest changes.

 

[Tsukihime]

You can alias the require method to try load_script before failing. That's also an option if you want to transparently handle scripts inside and outside the archive without ever having to worry about where your files might be.


Though, I don't know how to get it to automatically search the encrypted archive.

 

[Galenmereth]

Since this works with encrypted archives, and I'm using Dir.glob to go through files in subfolders, I'm assuming it retains the map structure within the rgss3a file. I tested this on a proper project with actual classes and modules, by the way, and it works beautifully so far

 

[Tsukihime]

I'm pretty sure Dir.glob doesn't actually search the encrypted archive though. Even though the maintains the exact same folder structure as your project, you need special methods to actually read or search through them.


The only functions available to us exposed to the ruby API are load_data and Bitmap.new


There may be functions available in the RGSS DLL, but I don't know anything about that.

 

[Galenmereth]

Mm, it might also just eval these scripts while archiving, making classes and modules available in a single file as they're parsed. Which is fine too, really, since as you see in the demo file everything is executed in the correct order But I'm going to debug some more by moving all my current scripts to this format in my game project and see if things break when archived using "real" scripts.

Does this concept interest you yourself, by the way?

 

[Tsukihime]

I would like to see a plugin framework for RM. We could probably add a GUI on top as well later down the road. Then your bootstrap file can be managed automatically by the system and users can add/remove scripts using an intuitive front-end.

 

[Galenmereth]

That's what I'd like to do (GUI) if there's interest. I may do it regardless for the practice In the meantime I'll make sure to make it as effortless as possible to do manually; it's very light on boilerplate already, and I'll be sure to keep it like that if/when I add new functionality as requirements come up.

Edit: Seems like it does not work properly with class inheritance in encrypted archives. Working on it.

 

[Tsukihime]

Seems like it does not work properly with class inheritance in encrypted archives.

What do you mean?

My interpretation of the problem is, given two files as follows

script1.rb

class Testend

script2.rb

class Test2 < Testend

It works normally, but not inside the archive?

 

[Galenmereth]

What do you mean?

My interpretation of the problem is, given two files as follows

script1.rb

class Testendscript2.rb

class Test2 < Testend

It works normally, but not inside the archive?

It seems like no classes or methods actually work when encrypted; it doesn't whine about not getting to load the scripts, but once you require a class or method in an event somewhere, it can't find it. So I'm thinking it doesn't actually load the scripts at all, or it doesn't load them at the right time. The classes and methods are not available to the game. When I used require it would output an error saying it couldn't find the file, which it doesn't do when using load_script.

 

[Tsukihime]

Yes, your scripts are not being loaded at all.

If you're doing this

Dir.glob("#{root_path}/#{path}/#{bootstrap_file}") do |bootstrap| next if loaded.include?(bootstrap.chomp(bootstrap_file)) self.require_bootstrap(bootstrap)endIt's not going to find anything in your encrypted archive, because ruby doesn't know how to search it (and no error messages will appear because you get an empty array anyways)I don't know if there is any way to search the encrypted archive yourself without writing a function to decrypt the data and retrieve a file table that you can then iterate over. That's the approach I'm taking with the patcher/DLC system I'm writing (well, I build a file table, but I don't let you iterate it)

For security purposes, you do not want to allow people to search for full paths in your archive, because then you could just inject some code to have the program dump out all of the data, which pretty much defeats the point of the encryption.

May need to re-consider the structure...the load_data function requires you to pass in the full path to the file you want. If you want to stick with the current structure, someone can provide a DLL that will parse the archive's file table and return that to you so you can perform searches on the entries.

 

[Galenmereth]

I solved it in a different manner; plugins are now "packaged" into a single scripts.rb file after the plugin module is done going through everything, placing it in the plugins folder. This scripts file is then loaded and eval'ed using load_script afterwards. This works with encrypted archives, and since the load order determines each class and method's placement in the scripts.rb file, everything gets executed in the correct order too.

The packaging part is skipped when the game is launched from an encrypted archive, so there's no overhead in that regard either. I haven't tested it with a huge amount of scripts yet, though, so I'm not 100% how and if it will impact performance when launching a game.

 

[Tsukihime]

I'm not sure I understand how you have solved the "file search" problem.


Can you explain how the plugin loader figures out what folders there are to search?


My impression is that you have a bootstrap.rb file in each folder, and then you explicitly load that file to determine what to load from the plugin.


However, then the question becomes...how do you know what the folder names are?


It looks like when I start the game (as a real, extracted-from-compressed-archive game), it prints out a bunch of lines, but then when I commented out the stuff in `scripts.rb`, nothing was printed out anymore.


I looked each plugin folder and see a module that will print something out when it gets loaded, but those aren't being printed out.


I thought maybe you had manually typed in the names of the plugin folders to search, but I didn't see that anywhere in any of the bootstrap files, so now I'm curious how you managed to find those folders.

 

[Galenmereth]

If you take a look in the setup.rb file, you'll see that I call Plugins.load_plugins with the ath param set to "*", which means it will now look for subfolders of root_path with bootstrap.rb files in them. It then executes the bootstrap.rb files using the Plugins.load_bootstrap method. This sets the Plugins.current_path variable to the path where bootstrap.rb file resides, then executes the bootstrap.rb file using load_script. The bootstrap.rb files call Plugins.load_files, which adds the path to each file into an array called @@scripts. At the end, in setup.rb, I call Plugins.package, which loops through the @@scripts array, reads the file contents of each scripts file in order into a string, which is then written to scripts.rb. This scripts.rb file is then eval'ed at the very end.

In an encrypted archive, it can't write a new scripts.rb file, so it is skipped. For a completed game you could call scripts.rb directly instead of setup.rb from within the game, and delete all other content from the Plugins folder to save a bit of space if you've got a ton of scripts. This is a process that there should be some kind of safe automation for eventually, I believe.

As long as you test your project after you make changes and before you encrypt it (which I hope everyone does ), the scripts.rb file will always be up-to-date for an encrypted version. I found out that in an encrypted archive, you can reference a script by path, but you can't use Dir.glob to find files and load them. That's why I write all the scripts into a single scripts.rb file, and reference that at the end of setup.rb using a direct path, since setup.rb is called just fine from within an encrypted archive.

Edit: I migrated my own game completely to the new structure, adding all current scripts into its own plugin folder called "Legacy Scripts", and it took no time at all (a few minutes of copypasta). Works like a charm, and the folder structure currently looks like this:

 

[Tsukihime]

Oh, I see how it works now.


I misunderstood what the scripts.rb file was actually for. That makes sense.


Ya, as long as you explicitly tell users to testplay (or provide a stand-alone tool that will perform the same operations), it would work well.

 

[Galenmereth]

Yeah, I need to write that up. I'm going to make some nice documentation when things are set more in stone. I'll also make an open github repository so people can fork and request merges if they fix stuff or want new features in the core repo

I made sure to make this modular, so you can for example call Plugins.load_plugins from within a plugin folder, and have separate folders with bootstrap files within these, too. Plugins.current_path is available for this very reason.

I'm also going to implement functionality that lets you order scripts/plugins even if they're not present (so that a script/plugin writer can control the order of files if the user has added them) in a transparent fashion, and I also want some sort of "ensure appended/prepended" functionality. For example, a bootstrap file can call a method to ensure that if another plugin is present, then it loads itself before the other one. But it needs to be simple and easy to use, so I'll spend some time figuring out the best approach. You do have this kind of control right now by specifying the rder param, but it requires editing of some files by the end user, which isn't optimal

 

[Shaz]

Moved to Ace Scripts at OP's request