Advanced Storage Plugin indexer fails on Orthanc-compressed DICOM files

General
1

I am using the Orthanc Advanced Storage Plugin to index an existing folder of DICOM files. The indexer works correctly with regular uncompressed files, but I encounter problems when the DICOM files were stored using Orthanc’s internal compression.

Here is the error message I get:

W: DcmItem: Length of element (9c78,bbec) is odd
E: DcmElement: Unknown Tag & Data (9c78,bbec) larger (3248247925) than remaining bytes (263494) in file, premature end of stream
E1026 18:47:03.890105          INDEXER OrthancException.cpp:62] Bad file format: Cannot parse an invalid DICOM file (size: 263510 bytes)

Is this expected when using Orthanc-compressed files?
If so, is there a recommended way to make the indexer handle these files, or should they be decompressed first before indexing?

2

Hi,

Yes, the indexer can not read compressed files. They must be decompressed first using the OrthancRecoverCompressedFile.exe.

3

thank you for your answer,

I am trying to migrate my Orthanc database from SQLite to PostgreSQL.
To transfer the data, I used the Orthanc Cloner tool with this command:

py -m orthanc_tools.orthanc_cloner --source_url=http://127.0.0.1:8070 --dest_url=http://127.0.0.1:8042 --dest_peer="peer" --mode="Transfer" --worker_threads_count=1 --persist_state_path="state" --error_folder_path="errors" --max_retries=2

However, I encountered this error:

--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\orthanc_tools\orthanc_cloner.py", line 116, in handle_stable_study
    raise Exception(str(transfer_job.info.content))
Exception: {'CompletedHttpQueries': 66, 'Compression': 'gzip', 'NetworkSpeedKBs': 10113, 'Peer': 'peer', 'Resources': [{'ID': '9123eabf-8ebc433d-dac3acd3-07e0f396-7e528366', 'Level': 'Study'}], 'TotalInstances': 450, 'TotalSizeMB': 291, 'UploadedSizeMB': 123}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\orthanc_tools\orthanc_monitor.py", line 222, in _process_changes
    self._handlers[change.change_type](change.sequence_id, change.resource_id, self._api_client)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\orthanc_tools\orthanc_cloner.py", line 140, in handle_stable_study
    raise Exception(f"Error while transferring study {study_id}: {str(ex)}")
Exception: Error while transferring study 9123eabf-8ebc433d-dac3acd3-07e0f396-7e528366: {'CompletedHttpQueries': 66, 'Compression': 'gzip', 'NetworkSpeedKBs': 10113, 'Peer': 'peer', 'Resources': [{'ID': '9123eabf-8ebc433d-dac3acd3-07e0f396-7e528366', 'Level': 'Study'}], 'TotalInstances': 450, 'TotalSizeMB': 291, 'UploadedSizeMB': 123}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\logging\__init__.py", line 1151, in emit
    msg = self.format(record)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\logging\__init__.py", line 999, in format
    return fmt.format(record)
           ~~~~~~~~~~^^^^^^^^
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\logging\__init__.py", line 712, in format
    record.message = record.getMessage()
                     ~~~~~~~~~~~~~~~~~^^
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\logging\__init__.py", line 400, in getMessage
    msg = msg % self.args
          ~~~~^~~~~~~~~~~
TypeError: not all arguments converted during string formatting
Call stack:
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\threading.py", line 1014, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\threading.py", line 1043, in _bootstrap_inner
    self.run()
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\threading.py", line 994, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\orthanc_tools\orthanc_monitor.py", line 226, in _process_changes
    logger.exception("Unhandled exception in event handler: ", ex)
Message: 'Unhandled exception in event handler: '
Arguments: (Exception("Error while transferring study 9123eabf-8ebc433d-dac3acd3-07e0f396-7e528366: {'CompletedHttpQueries': 66, 'Compression': 'gzip', 'NetworkSpeedKBs': 10113, 'Peer': 'peer', 'Resources': [{'ID': '9123eabf-8ebc433d-dac3acd3-07e0f396-7e528366', 'Level': 'Study'}], 'TotalInstances': 450, 'TotalSizeMB': 291, 'UploadedSizeMB': 123}"),)

Does the Orthanc Cloner support studies that were stored using Orthanc’s gzip compression?
If not, what is the recommended way to migrate compressed studies between two Orthanc servers?

4

I changed the mode to Peering, and now the transfer works correctly.

However, I’m not sure if Peering is the most efficient mode for transferring a large volume of data.
Is Peering the recommended mode for bulk migration, or would Transfer mode be faster?

5

According to the documentation, the fastest mode is the default one with 18 worker threads:

image
image880×621 28.6 KB

1 Like