Viewer Integration Needs

Chris_Hafey · August 30, 2014, 4:27pm

Hello,

I am working on integrating an image viewer with Orthanc and need all the DICOM fields for each SOP Instance in a study so I can sort/organize the images correctly. Right now I am iterating over each SOP Instance and getting the DICOM fields using the simplified-tags endpoint for each one but this can take several seconds to do when there are many SOP Instances in a study. Is there “batch” interface that would allow me to request all DICOM fields for all instances in a series or a study in JSON format?

I know WADO-RS is in the roadmap, any idea on when it will be coming and how? Will it be provided as a plug-in or as a native API?

Another useful feature would be to obtain the pixel data for an image frame as it is stored in the DICOM P10 instance (e.g. compressed) as well as one that returns the pixel data uncompressed. I am aware of the image-uint16 endpoint which returns a PNG and that is close, but I would like a bit more control. Something like:

/instances/{id}/frames/{frameNumber}/raw- Returns the raw pixel data for the frame as it is stored in the SOP Instance with MIME type application/octet-stream

/instances/{id}/frames/{frameNumber}/uncompressed - Returns the raw uncompressed pixel data for the frame as it is stored in the SOP Instance with MIME type application/octet-stream

A related question is what happens when you request a image-uint16 and the underlying image frame is signed 16 bit data? Does it apply some kind of offset?

Orthanc is a great project and I would be willing to help implement these features if you will take the help. Thanks!

Chris

PS - This is the viewer I am integrating: http://chafey.github.io/cornerstoneDemo/

jodogne · August 31, 2014, 7:28am

Hello Chris,

Thanks for your interest in Orthanc!

I am working on integrating an image viewer with Orthanc and need all the DICOM fields for each SOP Instance in a study so I can sort/organize the images correctly. Right now I am iterating over each SOP Instance and getting the DICOM fields using the simplified-tags endpoint for each one but this can take several seconds to do when there are many SOP Instances in a study. Is there “batch” interface that would allow me to request all DICOM fields for all instances in a series or a study in JSON format?

Yes. You can use the following URIs:
/patients/{id}/module

/studies/{id}/module

/series/{id}/module

These URIs will respectively give you the DICOM tags of the patient, study and series levels, according to the so-called “modules” of the DICOM standard. This avoids you to go down to the instance level using the REST API.

I know WADO-RS is in the roadmap, any idea on when it will be coming and how? Will it be provided as a plug-in or as a native API?

For the time being, our focus is on the release of Orthanc 1.0.0 (scheduled for Q4 2014). Because of the huge lack of public funding in Belgium for open-source products, my team at the University Hospital of Liège does not enough man-power to work on WADO-RS right now, at least until the release of Orthanc 1.0.0. This is the reason why the plugin mechanism was introduced: To avoid bottleneck on the Orthanc core.

It would be great if someone in the Orthanc community could work on a user-contributed plugin to support WADO-RS. I think that all the primitives are available in the plugin SDK [1], but do not hesitate to contact me if something is missing.

Another useful feature would be to obtain the pixel data for an image frame as it is stored in the DICOM P10 instance (e.g. compressed) as well as one that returns the pixel data uncompressed. I am aware of the image-uint16 endpoint which returns a PNG and that is close, but I would like a bit more control. Something like:

/instances/{id}/frames/{frameNumber}/raw- Returns the raw pixel data for the frame as it is stored in the SOP Instance with MIME type application/octet-stream

Orthanc already allows you to access the raw “Pixel Data” (0x7fe0, 0x0010) values:
/instances/{id}/content/7fe0-0010/{block}

In the case of uncompressed images, there is a single data block, so “block” will always be “0”. In the case of compressed images (JPEG, JPEG-LS…), there may be more than one data block. This explains the “block” parameter.

/instances/{id}/frames/{frameNumber}/uncompressed - Returns the raw uncompressed pixel data for the frame as it is stored in the SOP Instance with MIME type application/octet-stream

I do not understand this request: Please would you kindly explain the difference between it and the already-existing “image-uint16” and “content/7fe0-0010” URIs? For instance, in the case of a JPEG image, your “uncompressed” URIs would correspond to the result of “image-uint16”.

A related question is what happens when you request a image-uint16 and the underlying image frame is signed 16 bit data? Does it apply some kind of offset?

No offset is applied: The integer value stored in the DICOM image is cropped to the “uint16” data range (0->65535). For instance, “-42” would be set to “0”.

The same mechanism applies to “image-uint8” (0->255) and “image-int16” (-32768->32767). On the other hand, the “preview” URI would compute the minimum/maximum values and shift/rescale this range to the 0->255 range.

Please also note that the “RescaleSlope” and “RescaleIntercept” tags are never taken into account. The image extraction mechanism of Orthanc transmits the values that are directly stored in the “PixelData” tag.

Orthanc is a great project and I would be willing to help implement these features if you will take the help. Thanks!

Great news! I you are willing to contribute to extending Orthanc, I definitely suggest you to create user-contributed plugins, using the plugin SDK [1]. Once again, do not hesitate to contact me if some primitive is missing for your needs.

Thanks,
Sébastien-

[1] http://www.codeproject.com/Articles/797118/Implementing-a-WADO-Server-using-Orthanc

Chris_Hafey · August 31, 2014, 2:49pm

Hi Sébastien,

Thanks for the quick response. I did try the module endpoint, but none of them returned the SOP Instance level fields (e.g. InstanceNumber, Rows, Columns, etc). The series/{id}/module endpoint returns series level tags only. Is there some other parameter I need to specify to get the SOP Instance level tags? Also - the modules endpoint is not listed in the Orthanc REST API Spreadsheet (it appears to have been changed recently - it may have been in the older version but I don’t know if that is still available).

In terms of my request for accessing raw uncompressed pixel data, my hope was that Orthanc could handle the decompression of the various transfer syntaxes (e.g. JPEG2000) so I wouldn’t have to deal with that on my viewer side. Assuming Orthanc could do this and return the uncompressed image frame in a PNG I should be able to make it work. It would be more convenient for me if I could just get the uncompressed image frame as a raw byte stream (no PNG packing) though. I think this could be easily done by adding supporting for the application/octet-sequence MIME type to the image* endpoints. I did just test this with a JPEG2000 image and Orthanc returns a generic “UNSUPPORTED” PNG for /preview and /image-uint16. Does (or will) Orthanc support decompression of JPEG2000 images when generating PNGs? If not, is there someway to modify the list of transfer syntaxes accepted by the SCP so I can prevent storage of compressed transfer syntaxes? Thanks

Thanks

Chris

Qaler · September 5, 2014, 9:15am

How do you integrate this viewer with Orthanc? Does it simply pass the raw dicom data like xml or json to the viewer? Any chance to integrate with Orthanc explorer?

Chris_Hafey · September 5, 2014, 10:56am

The integration consists of a study list as well as a study viewer. The study viewer takes the Orthanc study uuid as input and can run in its own browser window. Selecting a study in the study list displays that study by creating a new browser tab and loading the study viewer with the study uuid The integration requires server side support for three types of calls:

Study List. This is a JSON document with fields from the patient, study and series level. This is used by the study list only
Study Document. This is a JSON document various fields from the patient, study, series and instance level. This is used by the study viewer only.
SOP Instance as a DICOM P10 byte stream. This is used by the study viewer only.

Orthanc has endpoints that provide data for all three above. Calling #1 and #2 from the browser would be too slow so some server code is needed to speed things up. I decided to build an Orthanc C++ plugin that implements #1 and #2. The browser calls the existing Orthanc /instance/{id}/file endpoint for #3.

The cornerstone javascript code is very modular so it would be straightforward to add it to Orthanc Explorer in a variety of ways. Orthanc would need to provide some kind of batch interface similar to #2 for fast study loading though. A batch interface would generally be useful so I hope it is something that can be added to Orthanc in the future.

I have the initial integration as described above implemented and it works very well. Large studies still take too long to load though so I am going to enhance #2 to prebuild the study document and store it in Orthanc. Doing so will allow the viewer to load any study no matter how many images in under a second. In fact, did a quick prototype of this already and was able to load a 18,000 image functional MRI study in 250ms!

Chris

jodogne · September 6, 2014, 8:59am

Dear Chris and Qaler,

I am really looking forward to see the interfacing between Orthanc and Cornerstone

Regarding the integration of a more advanced Web GUI than Orthanc Explorer to browse/view the DICOM store, I am convinced that the plugin SDK is a very nice fit [1]. Thanks to plugins, you can transform Orthanc into a programmable mini-Web server: Any URI (or any regular expression on URI) can be associated with a callback that is programmed in C/C++. Given this ability, you can statically serve JavaScript/HTML/CSS/… files, possibly with a dynamic templating engine such as Plustache [2], hereby bypassing the cross-origin problem in AJAX [3].

I am definitely convinced that this allows the development of an advanced GUI for Orthanc, e.g. with BootStrap and jQuery. I think that the only missing thing is the possibility to set the HTTP headers from the plugins (e.g. for cookie-based authentication), but this will be fixed in the next few days [4]. Do not hesitate to contact me if some other primitive should be implemented.

Regarding the “batch retrieving” of a set of URIs, this is indeed an excellent idea. I have just added a task on the roadmap [5]. I propose to add the URI “/tools/batch”, accessible with the POST method, that takes as input a JSON array containing the URIs to be fetched. The output is a JSON array (of the same size as the input array) that contains, for each input URIs, the JSON answer for this URI.

If the answer is not a JSON object, it is converted as a Base64 string. If the fetching of some input URI fails, the answer is set to the “null” value of JSON. I think this way of doing things is rather generic, yet it should solve the aforementioned performance problems.

Please let me know if you have any other suggestion.

Thanks,
Sébastien-

[1] http://www.codeproject.com/Articles/797118/Implementing-a-WADO-Server-using-Orthanc
[2] https://github.com/mrtazz/plustache
[3] http://en.wikipedia.org/wiki/Same-origin_policy
[4] https://trello.com/c/v3dXXMue

[5] https://trello.com/c/NVCD7rVm

Chris_Hafey · September 7, 2014, 3:53am

Hi Sébastien,

I agree with your thinking and what you are describing is exactly how the cornerstone integration is working today (it even has a static file handler to serve the js/css/html files to avoid cross origin issues). When it comes to the batch interface, I am thinking something slightly different. I don’t need the flexibility to specify which instances I want in batch, I just need to get information about all the instances in a study in one request. Currently it takes too long to build this on the fly for large studies. Users are impatient and will give up if it takes more than a few seconds to load a study regardless of how big the study is. My goal is therefore to load all studies no matter how big in under a second. Doing this requires prebuilding the information I need and storing it on disk so it can be returned to the client quickly. My plugin is currently addressing this by monitoring the /changes endpoint in a dedicated thread for StableStudy events. When one is detected, it builds the “study doc” and saves it back into Orthanc as a custom metadata item.

If you like my approach, I would be happy to help get this implemented in Orthanc core rather than as a plugin. Returning all sop instance fields for a study in one request is a key feature in WADO-RS and requires some preprocessing like I described to do it in a performant way.

Chris

PS - I initially implemented this using jsoncpp but found it to be quite slow. I have recently switched to using RapidJSON and it resulted in over 10x speed improvement. The RapidJSON interface is much more difficult to use than jsoncpp’s, but that is one of the tradeoffs that it made to achieve its speed. Overall Orthanc seems a bit sluggish when it comes to receiving DICOM and I suspect the use of jsoncpp may be part of the cause (storing all the tags in the DB is probably the other) so it might be something to consider.

Chris_Hafey · September 9, 2014, 2:08am

Hi Sébastien,
Based on your reply to AT VR bug today, I now realize that /content/{7fe0-0010} takes into account the transfer syntax and will decompress the image frame as I am requesting. I didn’t quite understand this from your prior reply so I apologize for confusing things. This is great news as it is exactly what I was looking for. I am curious if there is any way to access a given pixel frame without taking into account the transfer syntax. I don’t have an immediate need for this, but it could some day. Thanks

Chris

Qaler · September 9, 2014, 12:44pm

The cornerstoneDemo works perfectly on Firefox 31 and Chrome 37, however, it fails on MS IE 11 (which is also the latest IE version). On IE explorer, only the Invert and Play/stop clips tools function properly, other tools like ww/wc just make no response. I don’t know if it is an exceptional case, hope you can test it too.
FYI, my OS is Windows 8.1 with a default IE11( it means I did’t change any settings of IE).

在 2014年8月31日星期日UTC+8上午12时27分49秒，Chris Hafey写道：

Chris_Hafey · September 9, 2014, 2:56pm

Yes, I am aware of the bugs on IE 11, it has to do with the custom event mechanism. I expect to fix this int the coming weeks.

Chris

jodogne · September 11, 2014, 3:20pm

Hi Chris,

When it comes to the batch interface, I am thinking something slightly different. I don’t need the flexibility to specify which instances I want in batch, I just need to get information about all the instances in a study in one request. Currently it takes too long to build this on the fly for large studies.

Thanks for these good ideas! I have followed them for Orthanc 0.8.3. Now, you can find the 6 following URIs in the REST API:

/patients/{id}/studies
/patients/{id}/series
/patients/{id}/instances
/studies/{id}/series
/studies/{id}/instances
/series/{id}/instances
Using these URIs, you can download all the child studies/series/instances in just one REST call.

Returning all sop instance fields for a study in one request is a key feature in WADO-RS and requires some preprocessing like I described to do it in a performant way.

Have a look at “/studies/{id}/instances”. I guess that its performance should be quite good.

PS - I initially implemented this using jsoncpp but found it to be quite slow. I have recently switched to using RapidJSON and it resulted in over 10x speed improvement. The RapidJSON interface is much more difficult to use than jsoncpp’s, but that is one of the tradeoffs that it made to achieve its speed. Overall Orthanc seems a bit sluggish when it comes to receiving DICOM and I suspect the use of jsoncpp may be part of the cause (storing all the tags in the DB is probably the other) so it might be something to consider.

I have added a task to investigate with RapidJSON, but this is a rather long-term goal:
https://trello.com/c/a5r8s5lx

HTH,
Sébastien-

jodogne · September 11, 2014, 3:34pm

Hi again Chris,

Looks as if I missed a part of your question

I did try the module endpoint, but none of them returned the SOP Instance level fields (e.g. InstanceNumber, Rows, Columns, etc). The series/{id}/module endpoint returns series level tags only. Is there some other parameter I need to specify to get the SOP Instance level tags?

Have you had a look at “/series/{id}/shared-tags?simplify”?

This URI will loop across all the instances of the series, and will return all the DICOM tags whose value is constant across all the DICOM instances. It will allow you to reliably determine the “Rows” and the “Columns” of your image, since these values should be shared by all the instances of the series.

The “InstanceNumber” information can be found using the newly-introduced “/series/{id}/instances” (Orthanc 0.8.3).

Also - the modules endpoint is not listed in the Orthanc REST API Spreadsheet (it appears to have been changed recently - it may have been in the older version but I don’t know if that is still available).

I have just checked, and the modules are correctly listed in the link “Quick reference of the REST API for the latest release” of the following page:
http://www.orthanc-server.com/resources.php

Pay attention to the fact that after each release, this URL changes on the Web site.

Cheers,
Sébastien-

Chris_Hafey · September 11, 2014, 5:37pm

Hi Sébastien,

I did try the series/shared-tags but I need both the shared and unique tags. Since I have to go to the instance level, there is no benefit for me to call /shared-tags (well other than it can normalize for me, but its not hard to do once you have the data). My goal is to open a study in under a second and the only way that is possible for large studies is by precomputing the information I need (which is what I am doing). I did have a plugin method that iterate through the images and return everything I need but it was just too slow. The slowness came from having to open and parse so many files. I do think this precomputed version would be useful to others so it is something to consider adding to Orthanc core. This precomputed study document is part of the cornerstone plugin so others can of course use it if Orthanc doesn’t adopt this capability itself.

Thanks for clarifying the link to the REST API. I was looking at the .4 version, not sure how I got there but there must be a stale link somewhere. I’ll let you know if I come across the stale link again…

Chris

jodogne · September 11, 2014, 8:10pm

Hi,

I did try the series/shared-tags but I need both the shared and unique tags. Since I have to go to the instance level, there is no benefit for me to call /shared-tags (well other than it can normalize for me, but its not hard to do once you have the data). My goal is to open a study in under a second and the only way that is possible for large studies is by precomputing the information I need (which is what I am doing).

If I understand correctly and if I neglect the performance question, you would like to have an URI such as “/series/{id}/instances-tags” that would return an array that is the concatenation of all the “/series/{id}/tags”, for each instance in the study of interest?

Don’t get me wrong, I would just like to better understand your needs to provide the most generic implementation as possible

Sébastien-

Chris_Hafey · September 12, 2014, 1:34am

Yes, I think you understand but I suspect you mistyped "concatenation of all the “/series/{id}/tags”. Here are the two proposed calls:

/studies/{id}/all-instances - Returns an array of containing a JSON object matching what is returned from “/instances/{id}/tags” for each instance in the study. This would be similar in purpose to the WADO-RS RetrieveMetadata call

/series/{id}/all-instances - Returns an array of containing a JSON object matching what is returned from “/instances/{id}/tags” for each instance in the series.

Chris

PS - I found the stale link to the old rest api, it is on the bottom of this page:
https://code.google.com/p/orthanc/wiki/RestContent

Chris_Hafey · September 12, 2014, 2:40am

I just created a blog entry describing WADO-RS which you might find useful:

http://chafey.blogspot.com/2014/09/wado-rs-overview.html

Emsy_Chan · September 15, 2014, 2:38pm

Oh THANK YOU THANK YOU THANK YOU!!! Using PHP on Orthanc 0.8.2, I needed 35 seconds to parse through the instances to get the instanceNumbers; now I can get the instanceNumbers in just a couple of seconds!

jodogne · September 19, 2014, 9:49am

Hi Chris,

This is implemented in Orthanc 0.8.4 (that has just been released). Here are the 3 URIs of interest to you: