Orthanc stops answering C-FIND after a few days since upgrade to Debian 13 / PostgreSQL 17 (W004 warnings)

General
1

Hello,

We run a fairly large Orthanc instance:

  • ~440,000 studies, 90+ TB of DICOM storage, ~440 GB PostgreSQL index
  • ~200 client workstations, mostly MicroDicom and RadiAnt, querying over DICOM C-FIND
  • "LimitFindResults": 350, so that queries without a date or PatientID can’t overload the server

We recently upgraded the OS from Debian 12 to Debian 13 and PostgreSQL from 15 to 17 using pg_upgradecluster. Current versions: Orthanc 1.12.7, PostgreSQL plugin 5.0, PostgreSQL 17.11.

Since the upgrade, Orthanc stops returning any results to C-FIND queries after a few days of uptime. Restarting Orthanc restores normal operation. While it is in this state:

  • the REST API still responds (e.g. listing modalities works)
  • W004 warnings keep appearing in the log, for example:
W0930 14:29:06.760337 DICOM-1 ResourceFinder.cpp:1193] W004: study has been stored with an old Orthanc version and does not have a MainDicomTagsSignature, you should POST to /studies/e1ed8e83-54c58f07-5894c2fd-76d6421f-4c08035c/reconstruct to update the list of tags saved in DB or run the Housekeeper plugin. Some MainDicomTags might be missing from this answer.

Most of our archive was stored with older Orthanc versions. We considered running the Housekeeper plugin, but estimate it would take about 6 months on this volume.

Our questions:

  1. What is the recommended way to debug this, and what should we collect when it happens?
  2. Would lowering LimitFindResults and increasing DicomScuTimeout reduce the load on the database?
  3. Would reconstructing all studies reduce the database load for C-FIND?

Relevant configuration: DicomThreadsCount 4, DicomScpTimeout 30, StorageAccessOnFind Always, PostgreSQL plugin IndexConnectionsCount 1.

Thank you.

2

Hi,

First, I would increase the IndexConnectionsCount. The default value in recent versions of the plugin is 40.

Then, I would of course update Orthanc and the PostgreSQL plugin (now version 11.0): PostgreSQL plugins — Orthanc Book documentation

Your StorageAccessOnFind configuration is not ideal but. Depending on the content of C-Find queries, this might trigger a lot of access to disks that slow down C-Find a lot. We always recommend to use “Never”

  // Performance setting to specify how Orthanc accesses the storage
  // area during find operations (C-FIND, "/tools/find", API route, and
  // QIDO-RS in the DICOMweb plugin). Three modes are available: (1) "Always"
  // allows Orthanc to read the storage area as soon as it needs an
  // information that is not present in its database (slowest mode),
  // (2) "Never" prevents Orthanc from accessing the storage area, and
  // makes it uses exclusively its database (fastest mode), and (3)
  // "Answers" allows Orthanc to read the storage area to generate its
  // answers, but not to filter the DICOM resources (balance between
  // the two modes). By default, the mode is "Always", which
  // corresponds to the behavior of Orthanc <= 1.5.0.
  "StorageAccessOnFind" : "Always",

If you did not see these W004 warnings before, it just means that nobody was querying for old studies.
You should worry more about W001 warnings (W001_TagsBeingReadFromStorage)

Hope this helps,

Alain.