Plug-in manager, part 2: downloader / installer

    Collecting and organising thoughts and hints dispersed in recent threads, this thread is to discuss a specific feature set of a future plug-in manager (or a separate form of it), namely:


    *) accessing a list of published plug-ins from one (or more?) well-known source(s)
    *) accessing a list of plug-ins updated since installation ('@Jaob' suggestion)
    *) downloading selected plug-ins
    *) installing them into the local / dedicated server
    *) uninstalling / removing


    A) This requires some pieces of infrastructure / code:


    A1) a repository where to store published plug-ins (plenty of examples/resources for this, but a selection has to be made)
    A2) a protocol to access data / files from the repo (again, plenty of existing choices and a selection has to be made)
    A3) a way for plug-in authors to publish/upload their plug-ins into the repo
    A4) a UI for final users to select the required plug-in(s) for downloading / installing


    B) Also, a number of policy decisions are required:


    B1) If the publishing / uploading is open to anyone and/or if some registration / authentication is required
    B2) If the "system" (i.e. ultimately JIW staff) enforces some kind of quality control or not
    B3) If there are controls on who is downloading what or not (and, in case, which kinds of controls)
    B4) If the plug-in manager is the only supported/documented way of installing plug-ins (restricting manual addition of files to a well-known directory will be difficult, but installation might require additional steps than simply dropping files; or not!)


    And eventually, the grand decision:


    B0) Does this make sense at all?


    In particular, point A4 above might be of some import in particular for remote, dedicated servers, for which console access (via SSH or other) is not necessarily available. A general solution will require implementing some kind of UI for managing a remote dedicated server (for plug-ins, but also for other purposes, of course), which is not a trivial task.


    As plug-ins can be used on local installations, but make most of their sense on dedicated multi-player servers, whether implementing this whole machinery is worth the trouble or not requires careful evaluation.


    Awaiting comments, integrations, suggestions...

    Hey Miwarre :)


    Quote from Miwarre

    *) accessing a list of published plug-ins from one (or more?) well-known source(s)
    *) downloading selected plug-ins
    *) installing them into the local / dedicated server
    *) uninstalling / removing


    I would add the possibility to search for plug-in updates (so a repository with version control is needed). One annoying task for server admins is to update installed plugins manually, so a list which plug-ins need an update with the choice to update them automatically would be nice.


    Quote from Miwarre

    As plug-ins can be used on local installations, but make most of their sense on dedicated multi-player servers, whether implementing this whole machinery is worth the trouble or not requires careful evaluation.

    In my opinion this is the most important question at the moment :D
    I'am not sure right now, if its worth the trouble.

    • A plug-in manager just makes sense at all if there are enough plug-ins to manage. If someone rewrite the existing lua scripts there are maybe four or five plug-ins. I'am not sure if there will be enough new plug-ins in future for a plugin-manager to makes sense.
    • Most plug-ins will be developed for servers (you actually could use them in singleplayer but e.g. "areaprotection" doesn't make sense at all). Player who would like to play on a server with plug-ins don't have to download the plug-ins (unlike Minecraft which has several Modmanager), so in my opinion a plug-in manager would be used mainly by server owners. Most server owner knows how to install plug-ins manually and because there aren't enough plug-ins it's not complicated to update them (e.g. with a little .sh script).
    Quote from Jaob

    I would add the possibility to search for plug-in updates (so a repository with version control is needed). One annoying task for server admins is to update installed plugins manually, so a list which plug-ins need an update with the choice to update them automatically would be nice.

    Good point. OP updated.

    Quote

    In my opinion this is the most important question at the moment :D I'am not sure right now, if its worth the trouble.
    A plug-in manager just makes sense at all if there are enough plug-ins to manage. If someone rewrite the existing lua scripts there are maybe four or five plug-ins. I'am not sure if there will be enough new plug-ins in future for a plugin-manager to makes sense.

    The number of plug-ins is surely limited today. However, I would not be surprised if it should increase significantly in the near future. The more the API will be able to do, the more ideas will come to plug-in authors and the more plug-ins we will have.


    Of course, it also greatly depends on how widely used RW will be; but plug-ins may help even in this aspect, originating a kind of 'virtuous loop'.


    So, I am aware that right today a plug-in manager is not a priority, not only because there are not so may plug-ins, but also because there are more important aspects of the game to bring forward.


    Still, I believe it is useful to start thinking about it.

    Quote

    Most plug-ins will be developed for servers (you actually could use them in singleplayer but e.g. "areaprotection" doesn't make sense at all). Player who would like to play on a server with plug-ins don't have to download the plug-ins (unlike Minecraft which has several Modmanager), so in my opinion a plug-in manager would be used mainly by server owners. Most server owner knows how to install plug-ins manually and because there aren't enough plug-ins it's not complicated to update them (e.g. with a little .sh script).

    Yes and no: many ready-to-run game servers provided by companies out there have no actual access to a console shell from where you can launch a .sh file.


    Also, any automatable (spelling?) task still requires a well-known repo to access and some known protocol to query it.

    Whatever plugin manager comes to RW must have an option to run from the command line for. Commands from the game console is fine too but many of us running servers do management by shutting down the server and manipulating files from command line.


    I stand by my last posts about using Maven as a the best repo option for distributing plugins. It provides a simply web-based directory view that is easy to use browse or grab plugins via text-based web browser or curl or wget commands. It offers version control as well.
    Maven is browsable via HTTP and easy to view in a console web-browser which does not support graphics, flash, css, and other fancy stuff that is not text-based friendly. With maven, we could easily use the lynx browser or curl or wget to grab whatever plugin we need from the repository and install it ourselves. A plugin manager would automate all this but at minum if we have a Maven repo we atleast centralize and simplify access to the plugins


    If you havent yet looked at Maven, check out this link below. It drops you into the namespace com. All .com companies offering java programs to the public are found here. Ours would be greatly simplified but these public maven repos are used for application developers that develop EVERYTHING so literlly everything you can imagine is out there. Ours would be much much smaller (plugins only). So we can have our net/jiw/rising-world path along with others like my us/zork-foxden for my plugins and then namespaces for any other developer contributing, com/miwware or com/yahgiggle among others... All this can also be easily browsed by command line.


    https://repo1.maven.org/maven2/com/


    for a specific example... here's a library for android development. You can how each artifact is organized by: toplevel/company/artifactName/version.... Several past versions are available and each version contains only one jar file.


    https://repo1.maven.org/maven2…/aws-android-sdk-cognito/


    You can also see how javadoc (like our new API documentation) is distributed easily this way. You can also make any additional README text file and distribute that too if you want that to be dropped in to the directory too. Maven is not limited to jar files only. Javadocs and various xml files are also optionally distributed along with the jar.


    I'd love to know what other repositories are out there that are useful for distributing java applications and java libraries. I can't think of anything better to use than Maven which is pretty much the defacto standard

    Quote from zfoxfire

    Whatever plugin manager comes to RW must have an option to run from the command line for.

    Not sure to understand. For what?

    Quote

    Commands from the game console is fine too but many of us running servers do management by shutting down the server and manipulating files from command line.

    It seems to me that some initial consensus has been reached here (including @red51!) that in-game commands affecting the set-up of a dedicated server are too risky.


    Additionally, not all ready-to-run RW servers which can be rented by final users provide a command line; many only provide FTP access or a web-based equivalent.


    As the server itself, as far as I can see, currently allows no user input (it only has stout/stderr output which are printed to the console only if the server does not run in background), a way has to be added to allow it, either by an external manager console or by extending RCON or by whatever other mean deemed appropriate.


    Note that this does not need to be limited to plug-in management; it may also (or primarily!) be used for configuration / tuning of other server aspects as well. In any case, the server itself should be modified to react to this input.


    I understand quite well that this is not a priority now; however, I believe a user-friendly way of managing a dedicated server would make RW to stand a part from the most obvious competitors and would help enlarging the installed base.

    Quote

    I stand by my last posts about using Maven as a the best repo option for distributing plugins[...]

    I looked at both links you quoted and, sorry, but this is not my idea of a complete plug-in downloader / installer. This might very well be the back-end of it, then the real downloader/installer should come.


    You know how to make use of it; I also can surely quickly learn how; a number of current and potential RW users will also be surely at ease. However, I expect many users would look at those web pages and ask: "Well, now what?" No disrespect intended, mind; I am not saying that they are dumb or whatever, not by any mean; they might very well be much better than me at modelling or building or 'surviving' or whatever, they simply have different priorities, different experiences and expertises.


    My idea of a plug-in downloader / installer is a tool which:
    *) displays a list of installable plug-ins (possibly accessing a Maven-driven back end, I have no objections on this, but the decision is not mine), checking which plug-ins are already installed in the server and possibly the version of each;
    *) if a GUI is possible, the user clicks on a plug-in or, if only a text interface is possible, plug-ins are IDentified in some way and the user enters the number (or ID of whatever sort) of the required plug-in;
    *) the tool accesses the back end, downloads the appropriate file(s) into the appropriate server directory, performing any additionally required step, if any;
    *) if everything goes ok, it tells the server to enable the plug-in, otherwise displays an error message.

    Miware,


    I should have responded in-line but was lazy. Let me clarify..
    Maven would address only these following points you brought up.


    • A1) a repository where to store published plug-ins (plenty of examples/resources for this, but a selection has to be made)
    • A2) a protocol to access data / files from the repo (again, plenty of existing choices and a selection has to be made)
    • A3) a way for plug-in authors to publish/upload their plug-ins into the repo


    A plugin manager/UI is the last part of the puzzle. I I think we are just using our terminology differenty a "gui" is something launched from a desktop interface such as Windows that is graphical. A UI is the "gui" of Rising World where you play normally. A gui like you launch from a desktop is out of the question since not all servers offer a gui. Windows always has a gui but dedicated Linux servers offer only a command line.


    Like you said, some rented game servers do not offer a console option. This immediately rules out the option of a command-line tool


    It seems like RCON is the way to go. I tried it just now on my personal server and I like the tool However it needs a few improvements.
    [*]Offer a standard RW console in RCON to run any console command remotely
    [*]Offer option to runas <player> the above
    [*]Add a panel to implement the plugin manager

    • The panel should have a tool to point to a specified repo and list plugins available and select available version numbers.

    [*]Assuming loading/unloading plugins is dangerous while users are logged in and world is modifiable, offer a "maintenance mode" which will ensure game is read-only and any outstanding database calls are completed, prevent any user change from saving.
    [/list]I still like the idea of implementing a tool in the UI of RW itself. The maiin reason is that players in single-player mode may want to mod too. So a UI should be available which can actually benefit single and multi games.
    [*]Single Player - a config screen can be placed under Settings to manage plugins for the client only
    [*]In multiplayer - a console command to launch the same UI as above but this version only manages the server plugins only
    [/list]

    Quote from Miwarre

    B) Also, a number of policy decisions are required:


    B1) If the publishing / uploading is open to anyone and/or if some registration / authentication is required
    B2) If the "system" (i.e. ultimately JIW staff) enforces some kind of quality control or not
    B3) If there are controls on who is downloading what or not (and, in case, which kinds of controls)
    B4) If the plug-in manager is the only supported/documented way of installing plug-ins (restricting manual addition of files to a well-known directory will be difficult, but installation might require additional steps than simply dropping files; or not!)

    Here is one section I should address a bit clearer:


    B1)
    Whoever manages the repo must decide who is allowed to upload to it. I was thinking the well known modders here would specifically request it (or they can host their mods on their own which is why the plugin manager has to allow for manually installing into the plugins folder).
    I do not know how RW authenticates internally but if it should be interfaced by some kind of standard protocol such as LDAP. The Administrator of the repository is given access to look up users via LDAP and grant write access accordingly.


    B2)
    So regarding quality control, I would assume that JIW is not going to offer quality control. Otherwise we would need to add a build system to grab the source code from a developer's source repository, build it, scan for bugs and malicious code and then upload the build into the repo.
    The Maven repos act as mirrors to other repos. They accumulate jars from many different companies they mirror. There is no assumption of QA on the person running that repo. Only the original author who built and deployed the jar is the one who is assumed to have done QA.
    Same thing over in the minecraft world we also assume the authors of plugins are not writing malicious code but without looking at source code we cannot be sure.
    However, there's also the Steam Workshop which would in a way offer something similar to the plugin manager we are talking about. Now does the Steam Workshop ensure that players are safe from malicious plugins? This is also a consideration because Steam Workshop support has been requested in the past. I do not know enough about how this works or how its supposed to work.


    B3)
    I'm not following you on this. I interpret it to mean we should limit who the mods are distributed to. My believe is that the limitation is only on who can upload, not who can download.


    B4)
    I think that the plugin manager is just a tool to automate the manual process. I may want to test various versions of a plugin or test different combinations and for me it might be simpler just to manipulate a few files on the command line on my server then restart RW. I see no benefit to limiting manual intervention. The plugin will read the configuration of the plugins installed by reading each jar and the Manfiest file in each jar.

    I'm glad we are eventually reaching some common ground! :D (possibly the usual excuse about native language can be brought in again: English is not my mother tongue and, IIRC, is not yours either...)

    Quote from zfoxfire

    Maven would address only these following points you brought up[...]


    A plugin manager/UI is the last part of the puzzle.

    Indeed!

    Quote

    I think we are just using our terminology differenty a "gui" is something launched from a desktop interface such as Windows that is graphical. A UI is the "gui" of Rising World where you play normally.

    I assume there are formal definitions out there, but for me, informally, a UI is a User Interface, i.e. any mean allowing an interaction between the user and a device/application/.... Possibly, even the hands of a clock are a user interface too, but I prefer to think that a UI should allow both output to AND input from the user, so the hands are not enough to qualify as a UI, buttons/knobs to set the clock time (or to control other clock functions, like the alarm) are also needed.


    A GUI is a Graphical UI, i.e. a UI to a software application allowing output/input of more kinds than purely textual.

    Quote

    A gui like you launch from a desktop is out of the question since not all servers offer a gui. Windows always has a gui but dedicated Linux servers offer only a command line.


    Like you said, some rented game servers do not offer a console option. This immediately rules out the option of a command-line tool

    Possibly some trick could be devised to circumvent this limitation, but surely it risks to be hacky and to raise more problems than it would solve.

    Quote

    It seems like RCON is the way to go.[...] However it needs a few improvements.
    [...]

    This was my impression too. That RCON could benefit from improvements is understandable, as it is not a really finished product yet. We can expect it will grow and improve with the growing needs.

    Quote

    Assuming loading/unloading plugins is dangerous while users are logged in and world is modifiable, offer a "maintenance mode" [...]

    Single-player is a lesser problem; a plug-in manager runnable from the first screen, when no world is loaded (and no 'server' is actually running) could be perfectly acceptable for users and would avoid any issue from world access concurrency.


    Dedicated server is more tricky. As you said, plug-ins may (well, in the near future) change the world in many ways and altering it 'under the feet' of the users may have collateral effects. OTOH, plug-in update / turn over is not done every day and a responsible server owner can alert the players beforehand; server down time is always negative for the users; should RW allow at least some kinds of maintenance / set-up / fine tuning without necessarily turning off the server, this would be a big plus comparing with competitors.


    It is a point to keep in mind.

    Quote

    B2)
    So regarding quality control, I would assume that JIW is not going to offer quality control.[..]

    I do not expect JIW to assume any responsibility about this. Still some form of control could be automated, looking for used classed and methods, and may stop automatic acceptance for plug-ins which, just to make some example, try to 'phone home' or to export info or whatever. It would be far from fool-proof, but it might be a point to keep in mind.

    Quote

    B3)
    I'm not following you on this. I interpret it to mean we should limit who the mods are distributed to. My believe is that the limitation is only on who can upload, not who can download.

    There might be statistical purposes and there might be reasons to limit the download only to registered users or something similar.

    Miwarre,


    Thankyou for clarifying B3. Yes, i did not consider the possibility of excessive downloads or even DOS attacks. I was originally considering authentication on downloads to only apply to snapshots (pre-release builds).


    And English is my primary and only language but If I become relaxed and use too much technical jargon, a translator will not work as well. I also rush my posts a little because on complex issues like this, I could spend hours writing and re-writing one post and I'm usually too busy to be doing that. But I will try to be a bit more strict on my grammar in the future. It is especially important in a international community like this.


    For the most part I think we are on the same page with everything we've discussed in the past, maybe just not completely understanding on some of the finer details but that's ok :)

Participate now!

Don’t have an account yet? Create a new account now and be part of our community!

Create Account Login