Plugins not using DICOM cache

I encountered a major performance regression when upgrading from DICOMweb plugin 1.6->1.8, specifically when making multiple requests for image frames within a series. After some investigation it appears that the entire DICOM file is now being re-retrieved for each request- since we’re using S3 and can have some large files, the slowdown can be ~30x or more, making it unusable.

It appears the regression for DICOMweb occurs in changeset 505: https://hg.orthanc-server.com/orthanc-dicomweb/rev/76bb4f169072

I also found that using the GDCM plugin causes the same issue, regardless of accessing with the Orthanc REST or DICOMweb API- the whole file is re-retrieved.

Any ideas on how we can configure these plugins to make use of the cached data?

Hi Martin,

I confirm I can reproduce the issue with this call:
curl http://localhost:8044/dicom-web/studies/1.2.276.0.7230010.3.1.2.2344313775.14992.1458058363.6979/series/1.2.276.0.7230010.3.1.3.2344313775.14992.1458058363.6980/instances/1.2.276.0.7230010.3.1.4.2344313775.14992.1458058365.7015/frames/1/rendered -H “accept: image/png”

I had planned to work on the DicomWeb plugin today so I’ll have a look at it.

Best regards,

Alain

Hi Martin,

The problem was actually in the Orthanc StorageCache, not the dicom-web plugin itself. We had introduced this cache specifically to improve the /rendered routes of DicomWeb but we actually broke it while cleaning up the code before the initial release in 1.10.0: https://hg.orthanc-server.com/orthanc/rev/ea5f1c6ed07e.

This is now fixed in https://hg.orthanc-server.com/orthanc/rev/2b3b0ab88c1d.

I’m planning to release a new version of Orthanc and the DicomWeb plugin anyway “soon”.

Best regards,

Alain.

Wow, that’s great- thanks for the rapid response and fix! I can confirm this appears to fix caching for both DicomWeb and GDCM in a minimal setup (haven’t yet retested with ObjectStorage).

I benchmarked in a minimal local setup for a baseline, and found that DicomWeb requests are still a few times slower than Orthanc REST- they’re at about parity for DicomWeb 1.6 / Orthanc 1.11. Orthanc REST appears maybe a couple of milliseconds slower as well. These are for JPEG frames.

Using
Orthanc https://hg.orthanc-server.com/orthanc/rev/8c9a1cce076e

DicomWeb https://hg.orthanc-server.com/orthanc-dicomweb/rev/5620b09635c8

GET http://10.211.55.6:8042/dicom-web/studies/1.2.840.113619.2.199.32640.9756.20674.1645084972.90.8/series/1.2.840.113619.2.199.32640.9756.20674.1645084972.90.9/instances/1.2.840.113619.2.199.32640.9756.20674.1645087918.91.87/frames/1/rendered
DICOMWEB First: 0.295614
DICOMWEB Retrieved 30 frames in 7.025007000000001 seconds
Average: 0.23416690000000004
Min: 0.195881
Max: 0.379724

GET http://10.211.55.6:8042/instances/32a83460-78925937-008a1236-d1620df5-38dedefb/frames/0/rendered
ORTHANC First: 0.177552
ORTHANC Retrieved 30 frames in 2.064561 seconds
Average: 0.0688187
Min: 0.043821
Max: 0.080454

Hi Martin,

I just performed a quick test here on my system with the nightly builds. In my case, the DicomWeb returns smaller and faster rendered images.
The test imag is in cache from the start. The image decompression does not require GDCM

REST API
size: 116037, elapsed = 0.06019115447998047
size: 114740, elapsed = 0.05525064468383789

Dicom WEB API
size: 41630, elapsed = 0.022785425186157227
size: 40725, elapsed = 0.02229785919189453

I have attached my test script for the records and here’s how orthanc is started:
docker run -p 8044:8042 --env VERBOSE_ENABLED=true --env DICOM_WEB_PLUGIN_ENABLED=true --env ORTHANC__AUTHENTICATION_ENABLED=false osimis/orthanc:master-unstable

You might have another TransferSyntax that might explain some difference.

I’ll very likely release the DicomWeb plugin as is and, if you provide more information about your test images/setup, I might investigate performance issues later (really later :slight_smile: )

Best regards,

Alain.

test_dicom_web.py (945 Bytes)

Hi Alain,

I tested with your data (thank you!) and got similar results. Here is some sample data where Dicom WEB API performs much slower: https://drive.google.com/file/d/1eBBNEw8FHbsH9dDjg9vxBAT0cP7emeIl/view?usp=sharing
The TransferSyntax is the same (should just retrieve the underlying JPEG frames), so perhaps the difference is explained by the larger file size.

In DICOMweb 1.6, there was a fallback to the /rendered API when no customization was required. I find with this fallback in place, performance is much faster for both our datasets. It’s also faster than calling the /rendered API directly, and correctly doesn’t re-encode the JPEGs to a larger file size like a direct call (wonder why that is?)

Here’s my results for your data with current DICOMweb 1.9:

GET http://10.211.55.7:8042/instances/6da5698c-e0786c6a-4da190c4-f9c397da-b000aac5/frames/0/rendered
ORTHANC First: 0.299436
ORTHANC Retrieved 30 frames in 5.2794170000000005 seconds
Average: 0.17598056666666667
Min: 0.158508
Max: 0.204529
GET http://10.211.55.7:8042/dicom-web/studies/2.2.156.112536.1.2116.116254072074014219.13869477570.1/series/2.2.156.112536.1.2116.116254072074014219.13869477570.2/instances/2.2.156.112536.1.2116.116254072074014219.13869482390.39/frames/1/rendered
DICOMWEB First: 0.056248
DICOMWEB Retrieved 30 frames in 1.4525129999999997 seconds
Average: 0.04841709999999999
Min: 0.033337
Max: 0.074367

And results for your data with fallback applied:

GET http://10.211.55.7:8042/instances/6da5698c-e0786c6a-4da190c4-f9c397da-b000aac5/frames/0/rendered
ORTHANC First: 0.261207
ORTHANC Retrieved 30 frames in 4.803484 seconds
Average: 0.16011613333333333
Min: 0.143848
Max: 0.199825
GET http://10.211.55.7:8042/dicom-web/studies/2.2.156.112536.1.2116.116254072074014219.13869477570.1/series/2.2.156.112536.1.2116.116254072074014219.13869477570.2/instances/2.2.156.112536.1.2116.116254072074014219.13869482390.39/frames/1/rendered
DICOMWEB First: 0.02479
DICOMWEB Retrieved 30 frames in 0.5223550000000001 seconds
Average: 0.017411833333333338
Min: 0.016414
Max: 0.020122

The difference is even more dramatic with my dataset, which has more frames but smaller frame sizes:

DICOMweb 1.9:

GET http://10.211.55.7:8042/instances/32a83460-78925937-008a1236-d1620df5-38dedefb/frames/0/rendered
ORTHANC First: 0.421173
ORTHANC Retrieved 30 frames in 2.4859219999999995 seconds
Average: 0.08286406666666665
Min: 0.058674
Max: 0.180662
GET http://10.211.55.7:8042/dicom-web/studies/1.2.840.113619.2.199.32640.9756.20674.1645084972.90.8/series/1.2.840.113619.2.199.32640.9756.20674.1645084972.90.9/instances/1.2.840.113619.2.199.32640.9756.20674.1645087918.91.87/frames/1/rendered
DICOMWEB First: 0.184218
DICOMWEB Retrieved 30 frames in 5.805720999999999 seconds
Average: 0.19352403333333332
Min: 0.16563
Max: 0.283516

With fallback applied:

GET http://10.211.55.7:8042/instances/32a83460-78925937-008a1236-d1620df5-38dedefb/frames/0/rendered
ORTHANC First: 0.362587
ORTHANC Retrieved 30 frames in 1.6274589999999998 seconds
Average: 0.054248633333333324
Min: 0.037944
Max: 0.063809
GET http://10.211.55.7:8042/dicom-web/studies/1.2.840.113619.2.199.32640.9756.20674.1645084972.90.8/series/1.2.840.113619.2.199.32640.9756.20674.1645084972.90.9/instances/1.2.840.113619.2.199.32640.9756.20674.1645087918.91.87/frames/1/rendered
DICOMWEB First: 0.007981
DICOMWEB Retrieved 30 frames in 0.21028499999999994 seconds
Average: 0.007009499999999998
Min: 0.006321
Max: 0.010919

Here is the patch I applied to restore the fallback:

HG changeset patch

User Martin

Date 1658435397 25200

Thu Jul 21 13:29:57 2022 -0700

Branch dicomweb-fallback

Node ID 2d116fa435c7f340c344117e968dc13f47ada496

Parent ff823e1c31401866d042167cca201ae8d3d8b646

Restore DicomWeb fallback when no customization is required

diff -r ff823e1c3140 -r 2d116fa435c7 Plugin/WadoRsRetrieveRendered.cpp
— a/Plugin/WadoRsRetrieveRendered.cpp Wed Jul 13 13:19:48 2022 +0200
+++ b/Plugin/WadoRsRetrieveRendered.cpp Thu Jul 21 13:29:57 2022 -0700
@@ -328,6 +328,10 @@
}
}

  • bool HasCustomization() const
  • {
  • return (hasViewport_ || hasQuality_ || hasWindowing_);
  • }

unsigned int GetTargetWidth(unsigned int sourceWidth) const
{
@@ -864,6 +868,36 @@
throw Orthanc::OrthancException(Orthanc::ErrorCode_ParameterOutOfRange,
"Inexistent frame index in this image: " + boost::lexical_caststd::string(frame));
}

  • else if (!parameters.HasCustomization())
  • {
  • // No customization of the rendering. Return the default
  • // preview of Orthanc.
  • std::map<std::string, std::string> headers;
  • headers[“Accept”] = Orthanc::EnumerationToString(mime);