Reformat your json files~

Discussion in 'Mod Discussions' started by shadowmint, June 10, 2013.

  1. stuart98

    stuart98 Post Master General

    Messages:
    6,009
    Likes Received:
    3,888
    Actually need it without the sort keys now that I think about it...
  2. stuart98

    stuart98 Post Master General

    Messages:
    6,009
    Likes Received:
    3,888
    The reformatter breaks .pfx files. The order of the data cannot change, all that should happen is whitespace being added. Essentially make it into an automated jsonlint.
  3. mysticalfists

    mysticalfists Member

    Messages:
    92
    Likes Received:
    40
    Fancy script you made there Stuart! Cuttin my workload down by a ton!~
  4. stuart98

    stuart98 Post Master General

    Messages:
    6,009
    Likes Received:
    3,888
    I didn't make it, the OP of the thread did.
  5. mysticalfists

    mysticalfists Member

    Messages:
    92
    Likes Received:
    40
    ROFLMAO!! Hahaha, I need to sleep, I'm starting to see things :p
    stuart98 likes this.
  6. stylisticsagittarius

    stylisticsagittarius Active Member

    Messages:
    172
    Likes Received:
    57
    Probably a stupid question, but all of these json links ask for a url with no option to put a base file in there, how can i reformat all the pa files into the json format?
  7. zx0

    zx0 Well-Known Member

    Messages:
    295
    Likes Received:
    319
    You only reformat .json files from sinlge-line into easy readable. You can open json files with notepad or some better text editor that supports highlighting.
  8. LordFraggington

    LordFraggington New Member

    Messages:
    1
    Likes Received:
    0
    Just a quick update for anyone looking for an alternate way of formatting your JSON file - try the Atom editor. It's cross-platform, free, and most importantly, hackable. Sales pitch aside, what is neat is that it ships with JSON language support as well, and there is a nice package that you can install to make the code more readable.

    The package I have been using is called Pretty JSON, and from Atom you just need to follow these steps to set it up:
    File->Settings->Install->Enter "pretty-json" in the search packages field and Press Enter -> click the "Install" button of the package (a.k.a. pretty-json).

    Once installed, to use the package, open a json file then click Packages->Pretty JSON->Prettify. Viola!

    I hope this helps anyone looking to easily get started but is intimidated by using Python (not that you should be, Python is neat!), or working on other platforms (a lot of people associated with modding probably know a few good editors/IDEs, so here's hoping at least newbie might find this mostly "out-of-the-box" solution useful).

    Disclaimer: I'm not the guy who made the package, so I'm not going to promise it will work, but it's largely been unchanged despite a year of updates to Atom and doesn't throw any errors so you "should" be alright.

    Edit: Clean-up, clarification
    Last edited: March 1, 2015
  9. nixtempestas

    nixtempestas Post Master General

    Messages:
    1,216
    Likes Received:
    746
    When I really need to look at jsons I use the notepad++ plugin, though most of what I'm doing lately is just mod maintenance, which is all done by a java program I wrote to keep myself sane, which doesn't give a dam about formatting.
    robintick likes this.
  10. exterminans

    exterminans Post Master General

    Messages:
    1,881
    Likes Received:
    986
    Can't we just get a more robust json loader in the PA binary?

    One which can cope with whitespaces, tabs, linebreaks, and favorably even with comments (not an official part of JSON syntax, but sure helpful!)?

    It's not like that should impair performance as it's only a one-time cost when loading the JSON, and considering that the content is shipped compressed, whitespaces and alike shouldn't even increase download times.

    While the gains in return are significant, mostly because it becomes possible to document stuff inline which helps all mod creators.
    bsergent and lokiCML like this.
  11. cola_colin

    cola_colin Moderator Alumni

    Messages:
    12,074
    Likes Received:
    16,221
    The current one can do that? I use the script from this thread with a tiny modification to not stop on some invalid files:
    Code:
    import os
    import json
    import string
    from os.path import join
    
    for root, dirnames, filenames in os.walk('.'):
      for f in filenames:
        f = f.lower()
        if f[-5:] == ".json":
          try:
            fullpath = join(root, f)
            print("%s" % fullpath)
            fp = open(fullpath)
            data = json.load(fp)
            fp.close()
            fp = open(fullpath, 'w')
            fp.write(json.dumps(data, indent=2, sort_keys=True))
            fp.close()
          except:
            print "Error processing some file"
    
    I let this run over my whole PA directory directly after patching. Never saw issues that could be tracked down to the pretty json files.
  12. robintick

    robintick Well-Known Member

    Messages:
    325
    Likes Received:
    291
    Thanks for mentioning that plugin :) certainly makes things easier.
  13. Quitch

    Quitch Post Master General

    Messages:
    5,672
    Likes Received:
    6,032
    This is what a properly sorted condition in an AI file should look like:

    Code:
    {
             "name":"Basic Metal Extractor - cdr",
             "to_build":"BasicMetalExtractor",
             "instance_count":1,
             "shared_instance_count":"BasicMetal",
             "priority":264,
             "base_sort":"FromBuilder",
             "builders":[
                "Commander"
             ],
             "build_conditions":[
                [
                   {
                      "test_type":"PotentialMetalEfficiency",
                      "compare0":"<",
                      "value0":1.0
                   },
                   {
                      "test_type":"CanFindMetalSpotToBuildBasic",
                      "boolean":true
                   },
                   {
                      "test_type":"UnitCountOnPlanet",
                      "unit_type_string0":"Fabber - Orbital",
                      "compare0":"<",
                      "value0":1,
                      "boolean":true
                   }
                ],
                [
                   {
                      "test_type":"PotentialMetalEfficiency",
                      "compare0":"<",
                      "value0":1.0
                   },
                   {
                      "test_type":"CanFindMetalSpotToBuildBasic",
                      "boolean":true
                   },
                   {
                      "test_type":"CanDeployNavalFromBase",
                      "boolean":true
                   },
                   {
                      "test_type":"UnitCountOnPlanet",
                      "unit_type_string0":"Fabber & (Air | Naval)",
                      "compare0":"<",
                      "value0":1,
                      "boolean":true
                   }
                ]
             ],
             "placement_rules":{
                "placement_type":"FromMainBaseCenter",
                "threat":{
                   "influence_type":"AntiSurface",
                   "compare_type":"<",
                   "radius":10,
                   "value":50
                }
             }
          }
    This is what it looks like with this script

    Code:
    {
          "base_sort": "FromBuilder",
          "build_conditions": [
            [
              {
                "compare0": "<",
                "test_type": "PotentialMetalEfficiency",
                "value0": 1.0
              },
              {
                "boolean": true,
                "test_type": "CanFindMetalSpotToBuildBasic"
              },
              {
                "boolean": true,
                "compare0": "<",
                "test_type": "UnitCountOnPlanet",
                "unit_type_string0": "Fabber - Orbital",
                "value0": 1
              }
            ],
            [
              {
                "compare0": "<",
                "test_type": "PotentialMetalEfficiency",
                "value0": 1.0
              },
              {
                "boolean": true,
                "test_type": "CanFindMetalSpotToBuildBasic"
              },
              {
                "boolean": true,
                "test_type": "CanDeployNavalFromBase"
              },
              {
                "boolean": true,
                "compare0": "<",
                "test_type": "UnitCountOnPlanet",
                "unit_type_string0": "Fabber & (Air | Naval)",
                "value0": 1
              }
            ]
          ],
          "builders": [
            "Commander"
          ],
          "instance_count": 1,
          "name": "Basic Metal Extractor - cdr",
          "placement_rules": {
            "placement_type": "FromMainBaseCenter",
            "threat": {
              "compare_type": "<",
              "influence_type": "AntiSurface",
              "radius": 10,
              "value": 50
            }
          },
          "priority": 264,
          "shared_instance_count": "BasicMetal",
          "to_build": "BasicMetalExtractor"
        }
    Anyone have a modification that stops it alphabetizing the order of a block?
  14. dom314

    dom314 Post Master General

    Messages:
    894
    Likes Received:
    1,193
    @Quitch
    Code:
    import os
    import json
    import string
    import collections
    from os.path import join
    
    for root, dirnames, filenames in os.walk('.'):
      for f in filenames:
        f = f.lower()
        if f[-5:] == ".json":
          try:
            fullpath = join(root, f)
            print("%s" % fullpath)
            fp = open(fullpath)
            data = json.load(fp, object_pairs_hook=collections.OrderedDict)
            fp.close()
            fp = open(fullpath, 'w')
            fp.write(json.dumps(data, indent=2))
            fp.close()
          except:
            print "Error processing some file"
    The secret is object_pairs_hook :)
    Last edited: June 16, 2015
    Quitch likes this.
  15. Quitch

    Quitch Post Master General

    Messages:
    5,672
    Likes Received:
    6,032
    Whenever I run a version with the print command I get the following:

    File "C:\Python34\reformat.py", line 21
    print "Error processing some file"
    ^
    SyntaxError: Missing parentheses in call to 'print'


    The original script works fine.
  16. dom314

    dom314 Post Master General

    Messages:
    894
    Likes Received:
    1,193
    Oopsie, I just copied the code from cola colin and modified the relevant bits. The problem is that you have python 3, which means print is a function not a statement.
    Replace that line with this:
    print ("Error processing some file")
    Quitch likes this.
  17. dom314

    dom314 Post Master General

    Messages:
    894
    Likes Received:
    1,193
    For those who want the original version of the script that does not reorder (or scramble) the keys:
    Code:
    import os
    import json
    import string
    import collections
    
    from os.path import join
    
    for root, dirnames, filenames in os.walk('.'):
      for f in filenames:
        f = f.lower()
        if f[-5:] == ".json":
          fullpath = join(root, f)
          print("%s" % fullpath)
          fp = open(fullpath)
          data = json.load(fp, object_pairs_hook=collections.OrderedDict)
          fp.close()
          fp = open(fullpath, 'w')
          fp.write(json.dumps(data, indent=2))
          fp.close()
  18. Quitch

    Quitch Post Master General

    Messages:
    5,672
    Likes Received:
    6,032
    [​IMG]
    dom314 likes this.
  19. mysticalfists

    mysticalfists Member

    Messages:
    92
    Likes Received:
    40
    I had been using the original script posted by @shadowmint for reiterations of my mods; however, I recently updated to Windows 10 and I'm now having issues. Not sure if it was Windows 10 related or the version of Python so after receiving errors with Python 3.5.1 I downgraded back to Python 3.4, which had worked fine for me in the past. With either, I get an error now:


    Code:
    File "C:\Python34\lib\json\decoder.py", line 361, in raw_decode
      raise ValueError(errmsg("Expecting value", s, err.value)) from None
    ValueError: Expecting value: line 1 column 1 (char 0)
    Any thoughts on how to correct this? Thanks a ton. For now I'm looking into the other options posted.

    EDIT:
    I previously had Notepad++ as well, but had to reinstall it just now after having updated to Windows 10. Apparently Notepad++ reformats js and json script without the need for a plugin now. All is well!~
  20. Killerkiwijuice

    Killerkiwijuice Post Master General

    Messages:
    3,879
    Likes Received:
    3,597
    Are you sure the json you're trying to format is valid?

Share This Page