Data.post

[GregoryTravis]

This is a preliminary plan based on a quick conversation with Radek.

The post method follows the example of the fetch method, but with a body parameter:

post : (URI | Text) -> Text | File | Request_Body | Any -> Text | Nothing -> Vector (Header | Pair Text Text) -> Boolean -> Any
post uri body mime_type=Nothing headers=[] try_auto_parse=True =

The body parameter can be any Request_Body value, but it can also be a Text or an arbitrary object. These are handled as follows:

• If a Request_Body, it is passed along to HTTP.request unchanged
• If a Text, it is assumed to be a valid JSON string, and is sent as a Request_Body.Json value
• If a File, it is uploaded with Request_Body.Form
• If another Enso object, its to_json method is used to convert it to string, and it is sent as a Request_Body.Json value

Other ideas considered:

• A simple method for the common cases, and a more advanced one closer to the contents of a Request

Questions:

• Does post need a method argument?

[radeusgd]

Didn't we want to drive the interpretation of body by the mime-type argument? I thought that was our conclusion from the discussion.

[radeusgd]

It had the nice feature of disambiguating the Text case where we do not know if it's a raw string or a JSON representation already - with mime type we know how to interpret it.

[radeusgd]

Also, is the parameter to be named body or data? The proposal seems to mix these names, but we need to pick one.

[GregoryTravis]

I was planning to use the mime type only in the more 'advanced' method, but decided to have just one method. The thing is, it currently only uses the mime type for one specific case, to choose between Request_Body.Text and Request_Body.Json when the body is a Text. The underlying HTTP.request handles the content type for most cases.

[radeusgd]

IMHO that specific case is quite important, as otherwise there is no 'simple' way to send a plain text request. But I guess JSON is much more common, so maybe that's ok.

[radeusgd]

• If a File, it is uploaded with Request_Body.Form

I don't think I understand the rationale for this mapping.

Indeed, we have Part_Value.File in Form.enso, but:

1. If we are sending the file as a form data - what key will we use for this file? What key is valid depends on particular APIs, there is no good default here.
2. As far as I can see in our internal APIs, the file is sent as plain text. How will this handle non-plain text files, like binary, or even text files using different encodings?

IMO there's too many uncertainties here, especially (1 - the key name) - shouldn't we require the user to build the Form object themselves? Otherwise we really cannot guess what the key name should be. Also even with file upload APIs, they will often accept more than just a file (e.g. also a separate text_field for the file name and possibly other metadata), so I don't think such a default can ever work in practice.

[GregoryTravis]

Is the alternative to send the file simply by reading it and sending it as application/octet-stream?

[radeusgd]

Probably. I guess it depends if any APIs expect such an input. Maybe as part of this task we should do a small survey of what are expected formats in some common APIs (I assume most will be JSON, but we could look into what is expected in examples like file upload), and this could guide us what are the best defaults.

All-in-all, regardless of defaults, we should be able to support both approaches anyway. So we should be able to construct a form request for file upload expecting a multipart/form-data, and we should be able to send raw bytes as application/octet-stream too.

[GregoryTravis]

I changed file upload to send the file without a form, and added post_file_as_form.
Draft PR: #7700

[radeusgd]

IMHO post_file_as_form is not a good idea.

I don't think that many APIs expect a form consisting of only a file field and nothing else. Most of the time, there will be other fields that are to be set too (e.g. name, username, other metadata).

This function is so specific that I don't know if it will ever be useful. I may be wrong - do you have examples where such API format is expected? Are they relatively common?

[radeusgd]

My point is though, the post_file_as_form is too specific. It solves a very specific problem which I'm not even sure if occurrs in practice.

Instead, I think we should ensure that our Post API allows to nicely construct a form-data request of any 'shape', filling in various fields - containing file data or simple raw text content.

We already have a stub for this in the Form type (with file_field and text_field methods). I think that is the good direction and we just need to ensure that it is adapted to be user-friendly and work nicely in the GUI.

[GregoryTravis]

I based this on what https://api.cloudmersive.com/image/convert/to/gif requires, which is a form with a single file field. I don't know how common this is in practice.

I'm not sure a single post method will work. It would have parameters (like mime_type in the current one) that is only used in certain cases, and which must be omitted in other cases. Adding more features to this one method might make this worse.

We already have a method that is fully general -- HTTP.request. If Data.post handles all kinds of posts, but with a different protocol for the arguments, it's not really adding much, but just creates another, different API to learn.

[GregoryTravis]

It might make sense to just try out this code with other APIs, and find out what is commonly used by them.

[jdunkerley]

Data.fetch url method headers try_auto_parse

• GET / DELETE / OPTIONS / HEAD

Data.post url body method=HTTP_Method.POST headers=[] try_auto_parse_response=True

• POST / PUT / PATCH
• Behind the Output context - return a clear error message saying that the output context is not enabled click to enable.

Request_Body.Text <Encoding=UTF8> <Content_Type="text/plain">

• Auto conversion of Text to Request_Body.Text _ Encoding.UTF8
• Content type should have a suggested drop down set (text, json, xml, html, and any other that feel useful...)

Request_Body.JSON

• Calls to_json and then makes a Text request.
• Assume UTF-8, user can use Text and to_json if they really need non UTF8.

Request_Body.Binary

• POST an octet stream (binary data) --> application/octet-stream
• Could support conversion from a file... (File -> Request_Body.Binary)

Request_Body.Form_Data <Map Key:Text Value:(Text | File)>

• POST from a file submit button --> x-www-form-urlencoded
• File is an Octet Stream should have the filename in content-disposition header.

Request_Body.Form_URLEncoded

• The form data is encoded as URL parameters --> x-www-form-urlencoded
• Sent as Text with UTF8 encoding in the body.

Ideally lets do these as
Request_Body.from (that:Text)
Request_Body.from (that:File)

and try the auto conversion support.

Few questions later on more edge cases...

• follow later on Binary Vector (we probably need Byte and Byte_Vector type).
• we need to revisit authorization (both Fetch and Post).
• Need to contemplate cookies and sessions.
• Is there a sensible dry run option?
• Can we get the planned request in anyway? (for debugging)