Transfers Plugin Reduced Memory Usage Patch

General
1

Hi Orthanc team,

Following the recent streaming updates to Orthanc Server, I spent some time to improve memory usage associated with the Accelerated Transfers Plugin.

We’ve experienced issues using this plugin with very large dicom files in memory limited environments. My hope is this patch will help reduce the memory usage associated with importing large files via the Transfer Plugin.

I’ve attached a patch for the plugin with 2 changes:

  1. Uses SystemToolbox::ComputeFileMD5 helper to calculate the uploaded file streamed from the file system rather than reading the entire file into memory. This allows the plugin to calculate the MD5 with significantly less memory
  2. Use boost::iostreams::mapped_file_source to upload the file rather than reading the entire file into memory to import it.

Full disclosure - I’m not a C++ expert. From the research I’ve done, mapped_file_source is supported cross platform (*nix and Windows) and allows the OS to read the file directly from the file without allocating additional memory. This should further reduce memory usage during importing the file, removing the previous mirrored in-memory c string of the entire file.

The patch builds and I have run end to end tests that confirm the plugin is still working.

I welcome your review and comments.

0001-stream-md5-mmap-commit.patch.txt (4.8 KB)

1 Like