Zip vs Jars, Plugin nameing conventions, upgrades, and Maven repositories

Consolidating a few thoughts from other threads that I posted so I will be deleting them shortly..

I am concerned about how we will be installing/managing zips can be dificult to maintain and distribute. So I'm going to give some specific suggestions. This is all stuff I do for a living so I'm going to speak with some authority on the issue. I'll also go back and delete my posts in other threads since I'd like to keep this discussion in one place.

1. If we use zip files then everyone has to come up with their own way to manage and distribute their plugins.
2. We need a way to track versions and be able to see incremental builds or development (snapshot) builds and distribute them easily.

I suggest we consider using Maven to build and distribute our plugins. I just purchased a standalone copy of RW to confirm this but it looks like the standalone uses a helper application as Minecraft does so awesome bbq sauce! Now we need something simlar for plugins and possibly eventually distribution of the server jar with maybe a helper application (or script as well)

I personally manage a a few different maven repos at my job. Sonatype Nexus OSS is a free version that could possibly be integrated into rising-world.net. Not sure how it would link up with the current user database but it might be possible to make it a repository for our community only. I'm currently experimenting with Sonatype on my personal web server so I'll folow up on this when I can.

Now, Netbeans supports Maven-based projects. Here's a quick-start with screenshots. Essentially this allows you to build a jar following the versioning standard that Maven uses and I'll sumarize below:
https://platform.netbeans.org/…-maven-quickstart.html#01

So a few terms:
Artifact: anything that is built. A JAR would be an exmaple of an artifact
Release build: this is where you build an official release of a jar. This can represent one plugin. This is only one release artifact per version : e.g ... areaProtection-1.0.0.jar
Snapshot build: this is the development version that uses the same version as the intended release build. for example, areaProtection-1.0.0-SNAPSHOT.jar

each snapshot is built, deployed to maven, and available for testing. anyone who wants to examine the jar deployed will always get the latest 1.0.0-SNAPSHOT as snapshots contain many builds, each with an internal timestamp

About snapshots: The thing nice about snapshots is that they are incremental but keep the same version. The API jar that Red51 shared with us is a perfect candidate for maven. It has already been updated a few times but each time it is, we need to download and update our local copy.. If this was built with maven, we would essentially point our Netbeans to the maven repository and request something like:
maven.rising-world.net/central
maven.rising-world.net/snapshots

once inside the each directory, you browse it the way you brows files on any typical webserver. here is an example of a common maven repo:
https://repo1.maven.org/maven2/

the namespaces inside can match whatever we want: e.g. for example, we might navigate to the API like this: net/jiwgames/risingworld/api/1.0 or if i wanted to add my own plugin I can use my own domain space and mine would be in us/zork-foxden/myFabulousPlugin/1.0.0-SNAPSHOT

Whenever Red51 updates a pre-release of his API and deploys it as a SNAPSHOT, we all now can pull the latest by referencing 1.0-SNAPSHOT rather than having to manually download and install. Then when us modders distribute our plugins, we deploy them into this maven repo and its available for anyone. Then someone has to write a plugin for RisingWorld (or Red can integrate it into the application) to help us manage versions. So lets say that in the game, we want to try out Zork's myFabulousPlugin-1.0. In RisingWorld, you look at the mods you are using, the system seens that you are on 1.0 but wait, there's a new version in the works... 1.1-SNAPSHOT. Now that means the snapshot 1.1 is still in the works and might be still a bit buggy but yuu can choose to filter out snapshots and just wait for the next release which would make the most recent 1.1-SNAPSHOT become 1.1. Maven allows you to filter releases and snapshots so its quite convenient.

------------------------

Regarding installations of zips.....
The plugins folder should allow either zip or jar. If a developer wants to not use maven, they can make their own zip and come up with their own versioning scheme, or a make their own jar and provide install instructions to maually download the zip or jar and drop it into the plugins folder.

I've been thinking about this for a while and there should not be any extraction process on the zip or jar. Java always runs a jar without extracting it. The only reason we would need subfolders is if there might be user custom configuration so for example:

Scenario1: For simple plugins that do not have options for users to customize. file is not extracted as the contents will not be modified:

• plugins
  
  
  • zorkPlugin-1.0.jar

--OR--

• plugins
  
  
  • zorksPlugin-1.0.zip

Scenario2: for plugins that might have config files within a folder created contains any generated and modified files (config files, databases, etc):

• plugins
  
  
  • zorkPlugin-1.0.jar
  • zorkPlugin
    
    
    • libs (any additional libraries downloaded that werent in the jar)
    • registeredUsers.db (custom user data that the plugin uses)
    • configFile.txt (some config file)

  
  

In scenario 2, the directory matches the plugin name and is in the same folder level under plugins. The content in the directory is up to the plugin to establish. Not sure if the API should include something to create userdata folder and limit any file changes to that directory -- to prevent the plugin from accessing the rest of the filesystem). Version is explicitly not included so that if you upgrade your plugin, the config file remains in the same location. Not sure if this standard should be implemented in the game itself or if its up to us to figure this out our own.

I understand the neuron situation my synapses have been firing like mad since that API came out. Its distracting me from my day job.

Yes. A snapshot is essentially a pre-release version. I do agree with you that it is wise to limit who can see them otherwise you have end users compaining about your plugin busting their game. Multiple repositories in maven could be set up, one per developer. The repos are accessible via virtual repos that join all repos together. Virtual repos are made available to the outside world and require authentication if linked to the rw user database) so developers decide in their own repos who can access their snapshots and releases or we can just agree to make a single repo of snapshots and limit access to trusted developers and testers in this community. The release repos can be made available to the public and eventually be linked to the Steam Workshop ( i still know nothing about SW but it sounds like something to manage plugins in versions so I assume it has maven support).

But yes, disclaimers galore must be placed with developer contact info published. This can be written directly into a jar which contains lots of meta data in addition to the code. A jar is essentially a smart zip file. Infact they are compressed exactly the same way! Every possible precaution must be made to ensure that plugins are installed as is and Red51 is not responsible if we break the game! ;-p

Regarding the elevator scenario, i still think single jar is best approach. If you provide an elevator model, it is inside the jar. The plugin works out of the box. Your instructions can say you can optionally make the subfolder and have the folder contain the config file and user provided model. Your code in the jar just has to be writen to ovverride the model a config file and defined model name are in that optional subfolder.

But regardless of how you implement it, the subdirectory matching the jar name is essentially a place to modify the mod or maybe where the jar can create a temp file or something. It would be up to the developer on how to use the folder.

Placing the jar outside a subfolder makes it a bit easier to update jars (all jar files immediately under the plugins directory) or a way to quickly restore a plugin behavior (wipe the subfolder).

The way red described keeping all assets in one jar is very common practice for making a jar. If you build a simple jar, all the included classes (libs) should get put into the jar. Same with images or 3d models. If you study how the asset jars in rising world are designed then you should see what I am talking about.

Minecraft has a very messy approach of creating a disguisting and deeply nested directory structure. I do not understand the reason for this. Rising World on the other hand composes of about a dozen or so jars with some config files. Very very well organized.

And here is my grande future vision of things:

The only reason i suggested integrating into the rising-world.net domain is to make it easy for us to control access by rw usernames. Same thing with some other ideas that I've ran by Red a while ago... granting developers git repositories to centrally manage and share plugin source code. I also had an idea of sharing blueprints through git as well... I envision that someone might someday write a plugin to share blueprints. They can be versioned as well and sent directly to a user or shared publically hence the need for a repository. Public blueprints might also come into play if someone was to write a plugin to generate custom villages or dungeons with different themes. Red51 is writing the dungeon generator, we would just need a way to override dungeon rooms with custom blueprints for each room (this is still very speculative as we still do not have dungeons yet nor API calls to customize dungeon generation but I just picture lots of stuff working togethrt if we make repositories to share our plugins and blueprints.

The above would be made easy if we had an option to centrally store and distribute. I do not like the way the minecraft community distributes plugins. Its a very manual process. But I suppose that in itself discourages players from misusing mods. But isn't that essentially what my idea or the Steam Workshop opening the door to?

oh... i just thought of something.. plugin bundling would be a cakewalk with maven. Arcane Desmond mentioned to me how he used to test minecraft mod bundles (something like Tekkit would be a great example). The process he described to me seemed very manual and tedious. When I set up new maven projects at work, I have to write a pom file to describe all the various jars i want to include. Im sure Netbeans offers a way to generate this file. Essentially you would be building a list of all plugins to include. Netbeans can compile either the sources or the built jars and make a massive jar.

A jar is essentially a collection of classes and additional asset files. Even with the additional libraries that might be needed, your Netbeans should be consolidating all those classes (your and the additional libraries) into a single jar. So combining multiple jars into a new massive jar (the bundle of mods) can be made, tested, and distributed easily as well.

Quote from Miwarre

Indeed! But it only works for assets distributed together with the .jar and which do not change over time, it does not work for assets which do change over time or are added at user side.

[/quote]
If your assets distributed change over time, you need to bump your version number whenever you release a change to your code or assets. If you want to have multiple variations of your distribution with the same variation then you can give each package a different name variation. The idea of an optional subfolder is to allow users to add to the mod content to override whats in the mod, for example if they choose to use a different elevator model. Essentially the jar works as its told to internally unless folder container configuration info. The jar just contains the default configuration

[quote='Miwarre','https://forum.rising-world.net/index.php/Thread/4806-Zip-vs-Jars-Plugin-nameing-conventions-upgrades-and
Oh no! Nobody reads instructions (except you, me and some other on this forum, of course... ), Windows users never pay attention to file/directory name case, everybody can make mistake... The less steps are required by the user and the most obvious each operation is, the smaller the probability of errors and problems. The simpler, the better; and "one plug-in, one directory" is hard to beat as simplicity and understandability.
[/quote]
I think its reasonable to expect that if the person using the mod just wants to use it out of the box then they can just install it. The directory creation and whatnot should not be done unless the player actually wants to mod it like you said with extra elevator models. That person is going to have to read instuctions otherwise. Or you can design the elevator mod with a gui that will allow the player to browse to a 3d model on their hard drive and then the mod will update the config file and subdirectory by itself. If you design software to be user friendly then the instructions should not be too complicated or in many cases the process intuitive. Most of the minecraft mods' I've seen are NOT drop-in friendly so I hope that with Rising World we get a chance to do modding better and smarter

Quote from Miwarre

(incidentally, the requirement you seem to propose of .jar file name matching sub-directory name is another thing which can easily go wrong).

Depends on how Red51 implements the plugin installation process. Perhaps when RW boots up, it scans the plugins and automatically creates the subfolders based on the jar names. Its up to the jar to use the folder space as it needs but it could definately be optional. This would remove any requirement for a player to be renaming folders properly. We have to view installing mods almost like operating a toaster appliance. You buy the think and plug it in and it just works.

Quote from Miwarre

B) Maven & NetBeans: I have never used either, as I'm in the Eclipse / Ant congregation

Maven is a much smarter built tool than Ant. In ant, steps are defined manually. In maven for the most part just define the jars and their version numbers you want to include and Maven does the rest. There's much less configuration necessary and I'd imagine that Netbeans or Eclipse will do most of that work for you. Incidentally, Ant is used by default in Netbeans as well. if you swith to a maven java project then it changes.

Quote from Miwarre

How the repository backing this list is organised, whether some quality control is assured or whether some requirements are made to appear in this list (and which ones), I believe are questions better left to the dev team to decide, according to the general application structure, dev resources and priorities, whether Steam (and Steam Workshop, in case) make requirements or assumptions and so on.

i agree that its ultimately up to the JIW team how to manage all this, Perhaps the modder community can manage something like what I'm suggesting seperately. Its just an idea. The convenience of having it part of rising-world.net would mainly be to give all users access without having to register on a standalone site. However, for the API it should be easy for us to get to. Most likely the API will be bundled directly with all future releases so using maven to access the API will be not-necessary. In that case the maven repository idea would be more of a community effort to distribute mods.

Quote from Miwarre

C2: From the plug-in developer perspective, a general repo for plug-in source code dev would surely be nice to have.
*) This can be as simple as a public github repo (which are free) or more structured (I don't think github public repos allows to restrict access to individual sub-directories).

The problem with putting everyting on github is I imagine we would have lots of mods in the works. That cannot be put under a single git repository. Subversion approach puts all projects under one repo. Git is one repo per project. A git server dedicated to the entire community (whether it be paid for by the community or by JIW) could be more cost effective. Each mod writer has their own repo or repos. And they can control write accesss to their repo

Quote from ArcaneDesmond

Yes very quite similar. Whenever Forge is installed in Minecraft it generates a folder called mods and in addition to all the libraries and functions it requires to launch the game. It also adds various other folders for screenshots,texture packs etc.. Right now Rising World is in it's infancy and to create a mod loader for it without the full source code in Java yet is just going to become a nightmare. Example: Right now the RW directory seems organized but once you start adding new libraries and mod folders you need a place for these to go without creating a disaster in the plugins folder. The plugins is a mods folder in itself external from that you would need another folder for additional libraries that connect to both the launcher and game itself otherwise the game won't run. Instead it's going to continually crash. This is why Forge creates it's own sub-directory and folders for which the game can launch.

A properly packaged jar file will have all its depdenencies included inside he jar. This is the purpose of the Maven build tool.
So if you want to build a bundle plugin that contains multiple unique plugins written seperately, you'd still be distributing one jar which is a consolidation and repackaging of all the other plugins.

When a jar is executed by JVM, it does not extract itself. the JVM loads what it needs from within the jar so no mess should be made.

I took a look at feed the beast and direwolf. Looks like their idea of a bundle is a zip file containing folders containing jars. I guess I don't understand well enough how plugins work in Minecraft but it looks extremely sloppy. If Minecraft is written in Java as the plugins are, then there should not be a mess of folders

[quote='ArcaneDesmond','https://forum.rising-world.net/index.php/Thread/4806-Zip-vs-Jars-Plugin-nameing-conventions-upgrades-and-Maven-repositories/?postID=38625#post38625']
Think of it like http://www.feed-the-beast.com Maven would need to have an independent launcher or a dependency written into the game code eta "Paulscode" and added to the games source-code for this to work properly. Don't get me wrong Maven sounds like a great idea and one that we could all use as we rock back and forth on the edge of our seats in excitement for real mods to enter the game. If I'm misunderstanding Maven or what you mean by this then I am sorry.
[/quote]
Could you help me understand what a launcher is? I have a feeling that the way plugins work in Minecraft is the result of how the game was hacked and modded in the first place. it worked but it is not clean. Red51 is wanting to make modding easy and support as much as the mod community desires. I get the feeling that he wants to follow all the normal jar packaging standards which should help us write clean and simply packaged mods. No hacky stuff!

Maven is a standard building tool used by software developers around the world. This is what we use at work to bundle our applications. Our applications are deployed into our Maven repository and pushed into production into our Fuse Fabric which grabs our jar files and makes them active on our application servers.... No messy file structures.. Just a bunch of jars.

Arcane,
I did a bit more research into Forge and confirmed my suspicions. Forge is a hack. It modifies the original jars. Therefore, yes it needs a standalone installer. If anyone wants to come up with hacks for Rising World then I highly doubt they will be supported by the JIW team.
The API provided to us requires writing normal java applications. They interact with the game using the official API. We are referring to them as mods bit they modify the game through an officially supported means. No installing or hacking necessary

Miwarre,

If we get a subdirectory dedicated to the plugin, you could essentialy write a class to handle the configuration file. It would contain a method to generate a config file off of a template inside the jar. The config file is placed inside the folder and can have as many comments as you want. The class would also need methods in it to check the file for proper syntax, handling errors, and your standard get/set methods for managing the values. If you want to have external variables the i suppose it would make more sense to only read off the external file but its largely up to you how to handle it. I would program default values in the code regardless so incase a user comments out a line, it won't cause an undefined variable error.

I do agree with you that it adds to complexity. So I guess at this point it all boils down to whether or not Red51 wants to allow the game to support drop in zip files which would expand and let you modify the contents anyway you want. I do not like that approach personally but I don't know specifically what the harm would be to support that approach in addition to what I propose.