Possible Memory Leak with Multiframe DICOM (Orthanc+OHIF)

Hello Everyone.

I’ve been trying to optimize multiframe DICOM (US) viewing with Orthanc+OHIF Plugin for some time now. I was able to “fix” YBR to RGB issue by modifying OHIF metadata provider source code to interpret all images with PI containing “YBR” and modality matching “US” as RGB until a permanent fix is available.

I am facing two other major issues though:

  1. When Orthanc ingest transcoding is not defined, it takes a while to load the frames initially (probably due to transcoding) and certain images (e.g. last frame - 94 of the first sample series) are permanently stuck on “Loading…”, this makes it impossible to view all images and play the series in Cine mode.

  2. To minimize initial load time and dodge the YBR/RGB issue mentioned earlier, I’ve experimented with setting the ingest transcoding to “1.2.840.10008.1.2.1”. Unfortunately, this leads to exceptionally high CPU/Memory load during both - ingesting the study and viewing it.

It seems like Orthanc is actually attempting to transcode to the same transfer syntax.

Please also note that 3gb memory usage spike is very unusual as the DICOM files themselves are around 150mb.

to replicate my issues:

version: '3'

services:
  orthanc:
    image: osimis/orthanc:master-full
    ports:
      - "80:8042"
    environment:
      - VERBOSE_ENABLED=true
      - VERBOSE_STARTUP=true
      - OHIF_PLUGIN_ENABLED=true
      - ORTHANC__OHIF__DATA_SOURCE="dicom-web"
      - DICOM_WEB_PLUGIN_ENABLED=true
      #- ORTHANC__INGEST_TRANSCODING="1.2.840.10008.1.2.1"
      - ORTHANC__ORTHANC_EXPLORER_2__IS_DEFAULT_ORTHANC_UI=true

Issue 1:

Upload my sample study to Orthanc running with above docker-compose and try to view in OHIF.

Issue 2:

Delete the orthanc volume, Uncomment ingest transcoding env var, re-upload the sample study into Orthanc and try to view in OHIF.

Sample US File (taken with Philips iE33)

Thank you for your work, hopefully someone can help.

Hi,

For Issue 1:
I’m not sure what version you are actually using. I tried with osimis/orthanc:23.9.2 and docker stats does not show memory usage higher than 600-700 MB that goes down to 200-300 MB after loading. Frames 94 opens correctly

For Issue 2:
Indeed, I observe high cpu load when ingesting but this is expected since transcoding occurs.
When viewing, there is a high CPU indeed while the HTTP threads are serving the frames and extracting them from the multi-frame files. The messages about transcoding are misleading as explained in this thread.

The memory usage goes up to 2.5 GB which is actually the size of the study once stored uncompressed. I tried limiting the container allocated memory to 4 GB and reload the OHIF viewer many times and its not growing and sometimes going back to 1 GB so I don’t think there is a leak here.
Feel free to check the implementation notes about memory consumption for more details.

So, nothing to worry about so far. But feel free to correct me if I’m wrong.

Best regards,

Alain.

Hi Alain,

Thank you for your response. I’ve decided to conduct a more comprehensive test with the following parameters:

  • Device: I’m using an M1 MacBook Pro with 16 GB of RAM. 6GB and 4 CPU cores are dedicated to Docker.
  • Samples:
    • US Study: 48mb in original transfer syntax and around 730mb uncompressed.
    • CT Study: Approximately 650mb, both in native transfer syntax and transcoded.
  • Viewers: I tested using both Stone and OHIF.

Test Procedure:

  1. Start the logging/resource monitoring script and name the test.
  2. Go to localhost and upload the study.
  3. For the US, open the first 101-frame series (or 320-frame series for CT) and attempt to play it in CINE mode.
  4. Stop the monitoring script when either: a) The series has played from start to finish or b) The viewer seems unable to complete the series playback.

Results:

  1. US Study (native compression) with OHIF:
  • Resource usage is consistent and low. Some frames load slowly, but after a certain point, the viewer/CINE playback will hang permanently.
  1. US Study (native compression) with Stone:
  • This is the best-case. Resource consumption is minimal, with almost instant display/playback.
  1. US Study (1.2.840.10008.1.2.1 ingest transcoding) with OHIF:
  • Both CPU and RAM usage are very high. RAM peaks at 3.8GiB, but I’ve had Orthanc OOM killed once, even with a 6GB budget. 95% of frames load quickly, but some take longer. On average, full CINE playback is achieved in about 2 minutes.
  1. US Study (1.2.840.10008.1.2.1 ingest transcoding) with Stone:
  • RAM peaks around 2GiB, with an average resource load. The viewing performance is terrible; in several attempts, I’ve only seen 10 frames at most loaded in the series. I’ve had cases where the viewport went black and viewer became unresponsive.

I’ve attached the full test and sample data in a zip for reference. If you’d like to test it yourself, delete the CSVs and run “bash test.sh TestName”. The script will launch docker-compose and record logs/resource data in CSVs. To finish the test, use ctrl+c in the terminal and answer the question y/n.

I’ll also ask OHIF maintainers to take a look and give us input. I think both this, and YBR/RGB issue can be resolved in one of two ways:

  1. Server prioritizes responding to the viewer request in the transfer syntax that the file is stored in, unless another TS is explicitly requested.
  2. Viewer explicitly requests compressed transfer syntax for US studies

It would be great if you could discuss and decide on a course of action.

Apologies for a long post, Best Wishes.

Charts from all my recorded tests:








Might solve the issue

Thanks Alireza!

I’ll test with new OHIF build and give results here.

I’m guessing this won’t address the fundamental issue of Orthanc transcoding compressed images to respond to OHIF. Would it be possible to add config options to OHIF that will allow us to specify transfer syntax in accept headers according to modality/filetype?

from my earlier testlogs:

USNativeCompressionOHIF I1019 5:59:47 DcmtkTranscoder.cpp 306 DCMTK transcoding from 1.2.840.10008.1.2.4.50 to one of: 1.2.840.10008.1.2.1
USNativeCompressionOHIF I1019 5:59:47 StorageCache.cpp 101 Read attachment fdf99ad3-e9ac-490b-8e67-de711da6040b with content type 1 from cache
USNativeCompressionOHIF I1019 5:59:47 PluginsManager.cpp 161 (plugins) DICOMweb RetrieveFrames on 5bd140f9-050453b0-08cfcc84-0dab9558-531778f2, frames: 51
USNativeCompressionOHIF I1019 5:59:47 PluginsManager.cpp 161 (plugins) DICOMweb RetrieveFrames: Transcoding instance 5bd140f9-050453b0-08cfcc84-0dab9558-531778f2 to transfer syntax 1.2.840.10008.1.2.1

Test results:

1) US Original Transcoding

  • OHIF: 3.8.0-beta.3
  • Metadata Patch: (YBR > RGB) Not applied
    • Images are green and unusable.

2) US Ingest Transcoded to 1.2.840.10008.1.2.1

  • OHIF: 3.8.0-beta.3
  • Metadata Patch: No
    • Differences from OHIF Version used in Orthanc plugin:
      1. Framerate is automatically set to 50 instead of older (default) 24, I’m assuming this is the framerate specified in DICOM files.
      2. Slightly lower (-100-300MiB) RAM consumption on some ticks, but it still peaks at 3.5 GiB.
      3. Overall load time is similar, with most images getting loaded quickly and certain problematic frames taking 1-2 mins to load. Full playback in 1:30-3mins.

3) US Original Transcoding

  • OHIF: 3.8.0-beta.3
  • Metadata Patch: OHIF patched by interpreting YBR metadata images as RGB
    • Time to display the first frame is longer.
    • Similar resource usage patterns - ~1 core of CPU and ~600MiB RAM consumed. throughout viewing.
    • Never able to achieve full CINE playback due to problematic frames (amount of problematic frames is also increased compared to ingest transcoded) never loading.

4) US Ingest Transcoded

  • OHIF: 3.8.0-beta.3
  • Metadata Patch: OHIF patched by interpreting YBR metadata images as RGB

Exactly the same as without the patch (since ingest transcoded is already with RGB metadata).

Full usage/logs

What do you mean OHIF patched by?

In order to overcome the Orthanc convert YBR to RGB but does not change metadata issue, I’m forcing OHIF to interpret all YBR US studies as RGB by:

changing the source code in /platform/core/src/classes/MetadataProvider.ts
: case WADO_IMAGE_LOADER_TAGS.IMAGE_PIXEL_MODULE, to:
photometricInterpretation: instance.Modality === “US” && instance.PhotometricInterpretation.startsWith(“YBR”)
? “RGB”
: instance.PhotometricInterpretation

This is not an adequate fix though, and I’m hoping you and Alain can agree on how this is to be resolved long-term.

@alainmazy

Hi Alain,

I’ve tested modifying ohif.js and adding " requestTransferSyntaxUID: ‘1.2.840.10008.1.2.4.50’ " option to Orthanc data source. US study viewing performance in OHIF is close to that of Stone, but this also forces (and fails) transcoding for CT studies.

While I think the initial issue of having 3.6GiB memory consumption on a 750MiB uncompressed study should be investigated, practically, for optimal US storage/viewing/memory consumption when using Orthanc+OHIF there are two relatively simple solutions:

  1. Modifying Orthanc so that a user can configure responding to transferSyntax=* accept header with a syntax that the instance was stored in.

  2. Modifying OHIF so that it fetches /instances/instance/metadata > TransferSyntax and dynamically generates accept headers based on it.

Since I’m unfamiliar with C and Orthanc internals, I’ve started to work on solution 2. I think this is doable but I don’t think stored TransferSyntax metadata is usually requested from DicomWeb sources. This would involve writing a custom Orthanc DataSource and possibly modifying the core logic of how accept headers are generated.

Could you please give some guidance on how 1) can be done?

Kind Regards,

yomarbuzz

This is actually the current behavior as explained in this comment:

   * As a consequence, starting with release 1.5 of the DICOMweb
   * plugin, transcoding to "Little Endian Explicit" takes place by
   * default. If this transcoding is not desirable, the "Accept" HTTP
   * header can be set to
   * "multipart/related;type=application/dicom;transfer-syntax=*" (note
   * the asterisk "*") in order to prevent transcoding. The same
   * convention is used by the Google Cloud Platform:
   * https://cloud.google.com/healthcare/docs/dicom

I think this is even part of the DICOMWeb standard nowadays.

Best regards,

Alain

Thanks Alain!

My mistake was assuming OHIF was setting transferSyntax to * automatically.

It seems like when this is not explicitly set, Orthanc will respond with uncompressed data, which was causing my original issues.

It would be good if you could modify OHIF plugin configuration source code to specify the accept header like this.

I just tried setting this acceptHeader option and OHIF is still requesting explicit VR. Are you sure this option is supported in v3.6.0 ?

My config:

    "OHIF" : {
      "Enable": true,
      "DataSource" : "dicom-web",
      "UserConfiguration" : "/home/alain/o/build/configs/ohif.js"
    },

And the content of my ohif.js file:

/**
 */

window.config = {
    extensions: [
    ],
    modes: [],
    customizationService: {
      // Shows a custom route -access via http://localhost:3000/custom
      // helloPage: '@ohif/extension-default.customizationModule.helloPage',
    },
    showStudyList: true,
    // some windows systems have issues with more than 3 web workers
    maxNumberOfWebWorkers: 3,
    // below flag is for performance reasons, but it might not work for all servers
    omitQuotationForMultipartRequest: true,
    showWarningMessageForCrossOrigin: true,
    showCPUFallbackMessage: true,
    showLoadingIndicator: true,
    strictZSpacingForVolumeViewport: true,
    maxNumRequests: {
      interaction: 100,
      thumbnail: 75,
      // Prefetch number is dependent on the http protocol. For http 2 or
      // above, the number of requests can be go a lot higher.
      prefetch: 25,
    },
    // filterQueryParam: false,
    httpErrorHandler: error => {
      // This is 429 when rejected from the public idc sandbox too often.
      console.warn(error.status);
    },
    hotkeys: [
      {
        commandName: 'incrementActiveViewport',
        label: 'Next Viewport',
        keys: ['right'],
      },
      {
        commandName: 'decrementActiveViewport',
        label: 'Previous Viewport',
        keys: ['left'],
      },
      { commandName: 'rotateViewportCW', label: 'Rotate Right', keys: ['r'] },
      { commandName: 'rotateViewportCCW', label: 'Rotate Left', keys: ['l'] },
      { commandName: 'invertViewport', label: 'Invert', keys: ['i'] },
      {
        commandName: 'flipViewportHorizontal',
        label: 'Flip Horizontally',
        keys: ['h'],
      },
      {
        commandName: 'flipViewportVertical',
        label: 'Flip Vertically',
        keys: ['v'],
      },
      { commandName: 'scaleUpViewport', label: 'Zoom In', keys: ['+'] },
      { commandName: 'scaleDownViewport', label: 'Zoom Out', keys: ['-'] },
      { commandName: 'fitViewportToWindow', label: 'Zoom to Fit', keys: ['='] },
      { commandName: 'resetViewport', label: 'Reset', keys: ['space'] },
      { commandName: 'nextImage', label: 'Next Image', keys: ['down'] },
      { commandName: 'previousImage', label: 'Previous Image', keys: ['up'] },
      // {
      //   commandName: 'previousViewportDisplaySet',
      //   label: 'Previous Series',
      //   keys: ['pagedown'],
      // },
      // {
      //   commandName: 'nextViewportDisplaySet',
      //   label: 'Next Series',
      //   keys: ['pageup'],
      // },
      {
        commandName: 'setToolActive',
        commandOptions: { toolName: 'Zoom' },
        label: 'Zoom',
        keys: ['z'],
      },
      // ~ Window level presets
      {
        commandName: 'windowLevelPreset1',
        label: 'W/L Preset 1',
        keys: ['1'],
      },
      {
        commandName: 'windowLevelPreset2',
        label: 'W/L Preset 2',
        keys: ['2'],
      },
      {
        commandName: 'windowLevelPreset3',
        label: 'W/L Preset 3',
        keys: ['3'],
      },
      {
        commandName: 'windowLevelPreset4',
        label: 'W/L Preset 4',
        keys: ['4'],
      },
      {
        commandName: 'windowLevelPreset5',
        label: 'W/L Preset 5',
        keys: ['5'],
      },
      {
        commandName: 'windowLevelPreset6',
        label: 'W/L Preset 6',
        keys: ['6'],
      },
      {
        commandName: 'windowLevelPreset7',
        label: 'W/L Preset 7',
        keys: ['7'],
      },
      {
        commandName: 'windowLevelPreset8',
        label: 'W/L Preset 8',
        keys: ['8'],
      },
      {
        commandName: 'windowLevelPreset9',
        label: 'W/L Preset 9',
        keys: ['9'],
      },
    ],
  };
  
  /**
   * SPDX-FileCopyrightText: 2023 Sebastien Jodogne, UCLouvain, Belgium,
   * and 2018-2023 Open Health Imaging Foundation
   * SPDX-License-Identifier: MIT
   */
  
  window.config.routerBasename = '/ohif/';
  
  if (true) {
    window.config.dataSources = [
      {
        friendlyName: 'Orthanc DICOMweb',
        namespace: '@ohif/extension-default.dataSourcesModule.dicomweb',
        sourceName: 'dicomweb',
        configuration: {
          name: 'orthanc',
  
          wadoUriRoot: '../dicom-web',
          qidoRoot: '../dicom-web',
          wadoRoot: '../dicom-web',
          
          qidoSupportsIncludeField: false,
          supportsReject: false,
          imageRendering: 'wadors',
          thumbnailRendering: 'wadors',
          enableStudyLazyLoad: true,
          supportsFuzzyMatching: false,
          supportsWildcard: true,
          staticWado: true,
          singlepart: 'bulkdata',
          acceptHeader: [ 'multipart/related; type=application/octet-stream; transfer-syntax=*']
        }
      }
    ];
  
    window.config.defaultDataSourceName = 'dicomweb';
  
  } else {
    window.config.showStudyList = false;
    window.config.dataSources = [
      {
        friendlyName: 'Orthanc DICOM JSON',
        namespace: '@ohif/extension-default.dataSourcesModule.dicomjson',
        sourceName: 'dicomjson',
        configuration: {
          name: 'json',
        },
      }
    ];
  
    window.config.defaultDataSourceName = 'dicomjson';
  }

As a side note, I think I have just found one of the main reason why Multiframe instances are so slow with OHIF:
The whole DICOM file is transcoded for each frame being retrieved.

I’ll try to fix that ASAP.

I’m not sure about 3.6, I can confirm that changing the headers worked for the latest build of OHIF. I believe there were a lot of optimizations on OHIF side for multiframe DICOM as well, would be nice to bump the source version for the plugin too.

I’ve also found this post when researching the issue earlier. Could it be that the RAM spike and OOM killing for uncompressed images is caused by Orthanc holding multiple copies of the same file in memory? this would not be noticeable when OHIF requests a compressed syntax (full study was 48mb) but could easily overload memory with larger file sizes.

I’m working on packaging version OHIF 3.7.0 and I have set this setting as a default.

Hello Alain - wondering if this was addressed, i.e. whole DICOM file is transcoded for each frame being retrieved.

Thanks

Hi @olivert

This should have been fixed in latest versions since we are now packing OHIF 3.8.3: orthanc-ohif: 833abb2f82f4 NEWS

Alain