Skip to content

Specifying nested file trees without IDs in the API #1073

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task done
andrewn opened this issue May 13, 2019 · 2 comments
Closed
1 task done

Specifying nested file trees without IDs in the API #1073

andrewn opened this issue May 13, 2019 · 2 comments

Comments

@andrewn
Copy link
Member

andrewn commented May 13, 2019

Nature of issue?

  • Existing feature enhancement

Feature enhancement details:

As part of the Sketch upload API work, we want a nicer way of specifying file trees.

Below, "file" means file or directory, as they're mostly the same for this discussion.

Current approach

The current API shape closely mirrors the shape of the data stored in mongodb:

  • requires BSON-object ids to be generated by clients
  • files are specified as a flat array of all files
  • hierarchy/nesting is defined by the file IDs in a children array on the relevant object
Example
{
  "name": "Summer accordion",
  "updatedAt": "",
  "isSaving": false,
  "files": [
    {
      "name": "root",
      "id": "5cd98108d498ade50d793853",
      "_id": "5cd98108d498ade50d793853",
      "children": [
        "5cd98108d498ade50d793850",
        "5cd98108d498ade50d793851",
        "5cd98108d498ade50d793852",
        "5cd9810dd498ade50d793854"
      ],
      "fileType": "folder",
      "content": "",
      "isSelectedFile": false
    },
    {
      "name": "sketch.js",
      "content": "function setup() {\n  createCanvas(400, 400);\n}\n\nfunction draw() {\n  background(220);\n}",
      "id": "5cd98108d498ade50d793850",
      "_id": "5cd98108d498ade50d793850",
      "isSelectedFile": false,
      "fileType": "file",
      "children": []
    },
    {
      "name": "index.html",
      "content": "<!DOCTYPE html>\n<html>\n  <head>\n    <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/p5.min.js\"></script>\n    <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.dom.min.js\"></script>\n    <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.sound.min.js\"></script>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"style.css\">\n    <meta charset=\"utf-8\" />\n\n  </head>\n  <body>\n    <script src=\"sketch.js\"></script>\n  </body>\n</html>\n",
      "id": "5cd98108d498ade50d793851",
      "_id": "5cd98108d498ade50d793851",
      "fileType": "file",
      "children": [],
      "isSelectedFile": false
    },
    {
      "name": "style.css",
      "content": "html, body {\n  margin: 0;\n  padding: 0;\n}\ncanvas {\n  display: block;\n}\n",
      "id": "5cd98108d498ade50d793852",
      "_id": "5cd98108d498ade50d793852",
      "fileType": "file",
      "children": [],
      "isSelectedFile": false
    },
    {
      "name": "my-dir",
      "id": "5cd9810dd498ade50d793854",
      "_id": "5cd9810dd498ade50d793854",
      "content": "",
      "children": ["5cd98167d498ade50d793858"],
      "fileType": "folder",
      "isFolderClosed": false,
      "isSelectedFile": true
    },
    {
      "name": "something.js",
      "id": "5cd98167d498ade50d793858",
      "_id": "5cd98167d498ade50d793858",
      "content": "var answer = 42;",
      "children": [],
      "fileType": "file"
    }
  ]
}

New approach

Files can be specified as a JSON object, without IDs using just the filename. The name is used to enforce the uniqueness of a file in a directory, and directories support nesting.

The server would be responsible for creating IDs and converting to the structure required by mongodb.

Example
{
  "name": "Summer accordion",
  "files": {
    "sketch.js": {
      "content": "function setup() {\n  createCanvas(400, 400);\n}\n\nfunction draw() {\n  background(220);\n}"
    },
    "index.html": {
      "content": "<!DOCTYPE html>\n<html>\n  <head>\n    <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/p5.min.js\"></script>\n    <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.dom.min.js\"></script>\n    <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.sound.min.js\"></script>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"style.css\">\n    <meta charset=\"utf-8\" />\n\n  </head>\n  <body>\n    <script src=\"sketch.js\"></script>\n  </body>\n</html>\n"
    },
    "style.css": {
      "content": "html, body {\n  margin: 0;\n  padding: 0;\n}\ncanvas {\n  display: block;\n}\n"
    },
    "my-dir": {
      "children": {
        "something.js": {
          "content": "var answer = 42;"
        }
      }
    }
  }
}

Open questions

  • Do we need to limit how nested the files can be?
  • How do binary files get uploaded? I think this format doesn't handle it, and there's an endpoint to upload individual binary files.
@catarak
Copy link
Member

catarak commented May 15, 2019

Overall, this looks go to me! To answer your questions:

Do we need to limit how nested the files can be?

Right now, there is no limit; however, one data point for this is that the Sidebar styling breaks if a file is more than four levels nested, see _sidebar.scss. Given that this hasn't come up, ever, I think it makes sense for the sake of simplicity to limit the level of nesting. For the relatively small sketches that folks will upload to the web editor, I don't think they'll need more than four or five levels of nesting, anyway.

How do binary files get uploaded? I think this format doesn't handle it, and there's an endpoint to upload individual binary files.

My instinct is that this should be a separate endpoint—a user could upload a file or a folder of files, and get back the S3 URL, or some other ID, and then use to build the sketch JSON. We could also allow users to use their own media hosting, e.g. they can use whatever media url they want as long as it's properly CORS configured. For the p5.js examples uploaded to the editor, the media files are hosted on GitHub and using jsdelivr.

@andrewn
Copy link
Member Author

andrewn commented Jun 6, 2019

I'm going to close this issue, as the results of the discussion have been incorporated into the Public API proposal (#1076).

@andrewn andrewn closed this as completed Jun 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants