Mod Storage & Distribution Suggestion

Discussion in 'Mod Discussions' started by Pawz, December 5, 2013.

  1. Pawz

    Pawz Active Member

    Messages:
    951
    Likes Received:
    161
    Hi Guys,

    The recent discussion on where to host the UI Debugger toolkit prompted me to think about a better way to distribute and maintain a collection of mods and mod tools that could potentially be fairly large in size, and also changes regularly.

    The answer, I believe, is to use a distributed file sharing platform - namely, BTSync.

    It's basically DropBox without any associated costs, handles massive files with ease, and shares the distribution load among each person connected to the network.

    It works based on what they call a 'secret' - a unique identifier that links your devices together

    It's extremely simple to use - as a modder (or UberEnt), you simply set up a distribution folder and create a read-only secret that you publish, and everyone who adds the secret to their btsync client will be automatically kept up to date.

    It IS a little bit more intensive than a simple file download, and the process would have to be refined a little with some testing, but I believe that we should be able to come together as a modding community to share our bandwidth in such a way that huge mods and tools and libraries are available to all.


    I've created a sample setup for public consumption, using Cola_Colin's hotbuild as the guinea pig (it was at the top of the forum at the time :)) This is more intended for people to understand how BTSync works - I'm thinking that integrating this somehow with the mod manager might make for a fantastic way to distribute and keep mods up to date - ultimately you'd want to be able to hide the fiddly bits from the end user.

    In the meantime, I think that modders should be able to work together to collate the tools available into one easily accessed location.

    Here is the sample of how it would work:

    Read-only secrets for Planetary Annihilation Mods:

    Entire List:
    BSARM5NT364DTXP7SF4JYTDEGL6QAARCC

    Cola_Colin's Mods:
    BVNUCJZURV2WQIE77TVXUIR5XCPR7TBH2


    To Use:

    1. Install BtSync
    2. Click 'Add Sync Folder'
    3. Paste Secret into the Folder Secret
    4. Select where you want to store the mods
    5. Hit ok. Win!


    Please post your comments / ideas / concerns below!
  2. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,324
  3. cola_colin

    cola_colin Moderator Alumni

    Messages:
    12,074
    Likes Received:
    16,221
    What problems do you see with how the mod manager currently works?
  4. SXX

    SXX Post Master General

    Messages:
    6,896
    Likes Received:
    1,812
    I don't see any advantage of having dependency on closed source BtSync for such small files like most of mods will be. Even if there ever will be large mods there is no problem to host files traditional way.
  5. Pawz

    Pawz Active Member

    Messages:
    951
    Likes Received:
    161
    Currently yes there is not a huge need for large file distribution,but when I see things like uber thinking that an 80mb file is large, and struggling with posting the file to a shared drive, then I wonder if some kind of distribution system would be useful.

    Yes currently all mods are ui mods and literally tiny but that is not a situation that will last beyond the release of the server when full to mods etc become available. If you look at the state of mine craft the constant ad.fly links are a pain in the ***, and keeping things like the texture packs up to date between lots of contributors is hard as well.

    The mod manager is pretty slick but will we want to github a 1gb mod? (I mean that honestly took I don't have enough experience with it to know if it will scale or if a certain file size is going to start incurring server costs)
  6. Raevn

    Raevn Moderator Alumni

    Messages:
    4,226
    Likes Received:
    4,324
    The mods don't strictly need to be hosted there, ultimately it makes generating the mod list convenient, but the actual location of the mod is specified in a URL, so they can exist anywhere.

    I also don't know what the limitations of GitHub are with regards to file sizes etc.
  7. cwarner7264

    cwarner7264 Moderator Alumni

    Messages:
    4,460
    Likes Received:
    5,390
    Are we talking about storage of the source code for collaborative projects here, or a place where users can download the latest version of mods?
  8. Pawz

    Pawz Active Member

    Messages:
    951
    Likes Received:
    161
    I don't think it would well for storage of source code, since things like Github etc are explicitly designed to supplement that. It would, however, be perfect for modder / user downloads that are larger than normal. I was more envisioning that you could have something like a 'modders folder' that contains ALL your modding tools / downloads / sdks / dev environments / Photoshop actions / tutorials etc, so someone interested in getting started with mods can download the whole lot in one go and doesn't have to chase around trying to find this version of the importer and that version of the exporter and this Coherent UI debugger, etc etc.

    Aside from that, as the size of mods grow (and potential mod packs etc) then it would be an easy platform to distribute and keep your mods up to date.
  9. sebs303

    sebs303 New Member

    Messages:
    22
    Likes Received:
    6
    Another way could be developing a system like node.js'es NPM or Bower (Gem in Ruby or Composer for PHP) . Its well known infrastructure and there is a minimum set of controls that could be maintained on distribution.

    Additionally there is a codebase for this tools already and stuff has not to be re-invented (NIH - DRY ;)

Share This Page