Python plugin and response encoding from RestAPI Get/Post/Put

I’ve been working with the python plugin and find that certain Get/Post operations return byte strings while others return json or text.

For example, querying a remote PACS recently with

orthanc.RestApiPost(/modalities/REMOTEPACS/query, json.dumps(query))

I discovered the results came back as a byte string rather than json. I had to decode (assuming utf-8) before converting via json.loads() into a python dict.

I also received a byte string when asking for /metadata from a series on the Orthanc.

Most of the time, I receive json strings in response to various Get/Post/etc.

Can I do anything to pre-determine the encoding of the response? Are there times I can consistently expect byte encoding (and decode with utf-8?) and other times I can consistently expect a json string?

Thanks,
John.

Hello,

This is again a consequence of the fact that the “orthanc” Python module is automatically generated from the C SDK of Orthanc, whose source code can be found at:
https://hg.orthanc-server.com/orthanc/file/Orthanc-1.9.6/OrthancServer/Plugins/Include/orthanc/OrthancCPlugin.h

In a nutshell, in this file, the “char*” type is mapped to Python strings, whereas the “void*” and “OrthancPluginMemoryBuffer*” types are mapped to binary buffers.

If you are interested in how this concretely works, check out the source code of the Python plugin:
https://hg.orthanc-server.com/orthanc-python/file/OrthancPython-3.2/CodeAnalysis/ParseOrthancSDK.py#l391
https://hg.orthanc-server.com/orthanc-python/file/OrthancPython-3.2/Sources/Autogenerated

In the case of “orthanc.RestApiPost()”, the corresponding function is “OrthancPluginRestApiPost()”, that uses “OrthancPluginMemoryBuffer*” as its output, and that is ultimately mapped to a Python binary buffer using “PyBytes_FromStringAndSize()”:
https://hg.orthanc-server.com/orthanc-python/file/OrthancPython-3.2/Sources/Autogenerated/sdk_GlobalFunctions.impl.h#l190

HTH,
Sébastien-

Thanks, Sebastien.

Scanning through the source, it looks like an OrthancPluginMemoryBuffer* type is used for Get, Put, and Post calls. So I should always expect a binary buffer response?

I’m probably not reading the source correctly or putting it all together in my head.

Until I ran into my recent problem with a binary buffer response, I had lots of code where I treated the response from orthanc.RestApiGet like it was a string and passed it along to json.loads() to convert to a python dict. This was similar to my Lua scripts.

Assuming the python response is always a binary buffer, would that mean that sometimes json.loads() is able to handle the binary response as though it were a string and other times not? Because it gets confused by the byte encoding?

I just went and looked at a few examples on the Orthanc Python Plugin page and it seems most cases send the response straight to json.loads() without worrying about decoding the binary buffer (unless the response is an image blob or something like that).

In my case, json.loads() returns an error with a very particular PACS query response from orthanc.RestApiPost() unless I forcefully decode the response with response.decode(‘utf-8’) first before feeding it to json.loads().

I guess I need to figure out whether this encoding issue is a peculiarity of the particular PACS I am querying or a misunderstanding on my part as to when I need to decode responses prior to feeding to json.loads() and when I don’t need to decode them.

John.

Hello,

Scanning through the source, it looks like an OrthancPluginMemoryBuffer* type is used for Get, Put, and Post calls. So I should always expect a binary buffer response?

You should always expect a binary buffer response (i.e. of “bytes” type).

However, if this buffer contains only ASCII or UTF-8 characters, “json.loads()” should work transparently on it: UTF-8 is indeed both the encoding that is always used by Orthanc, and the default encoding for “json.loads()”.

You might possibly face an issue because of different versions of Python, as “json.loads()” has changed its specification over time (it looks like “json.loads()” doesn’t work on the “bytes” type between Python 3.0 and Python 3.5) :
https://docs.python.org/2/library/json.html#json.loads

For instance, in the integration tests of Orthanc, I had to always use the “.decode()” function of “bytes” object to convert the binary buffer to a string before applying “json.loads()” on Python 3.x:

https://hg.orthanc-server.com/orthanc-tests/file/Orthanc-1.9.6/Tests/Toolbox.py#l59

You might also be dealing with strings that contain invalid UTF-8 characters.

However, if you don’t provide a minimal working example (sample Python script + DICOM instances + version of Python), I cannot provide any definitive answer:
https://book.orthanc-server.com/users/support.html#discussing-a-minimal-working-example

Regards,
Sébastien-