Housekeeper 1.12.1 restarted processing when orthanc was restarted

General
1

Short summary of our setup:

  • running jodogne/orthanc-python:1.12.1 docker image
  • MainDicomTags configuration was changed 1-2 weeks ago and the Housekeeper plugin was deployed for the first time
  • on the 19th of September (2 days ago) we restarted Orthanc a few times while the Housekeeper was running for days already

After the restarts, the Housekeeper plugin started processing the changes from the beginning, even though 2 days ago it was already at ~47M/58M changes:

I0917 22:39:03.928263 PluginsManager.cpp:161] (plugins) Housekeeper: processed changes 46951479 / 58114506

Now our Housekeeper is back at 39M again:

I0921 06:03:52.448802 PluginsManager.cpp:161] (plugins) Housekeeper: processed changes 37009003 / 60567932
I0921 06:03:52.549105 PluginsManager.cpp:161] (plugins) Housekeeper: entering quiet period

I took a look into the DB and saw that in the serverproperties table there were 2 entries with the Housekeeper plugin’s ID (1025) but with different UUIDs in the “server” column:

server                                      |property|value                                    
--------------------------------------------+--------+----------------------------------------- ...
24ac28db-b11c281a-9ffcf627-e681c5e1-89058012|    1025|{¶   "CurrentlyProcessingConfiguration" : ...
ecab47b2-1418232f-267d18f8-f0eb651e-cfa0178c|    1025|{¶   "CurrentlyProcessingConfiguration" : ...

first entry’s value:

{
   "CurrentlyProcessingConfiguration" : 
   {
      "IngestTranscoding" : "1.2.840.10008.1.2.4.70",
      "MainDicomTagsSignature" : 
      {
         "Instance" : "0008,0005;0008,0008;0008,0012;0008,0013;0008,0016;0008,0018;0008,0023;0008,0033;0018,0050;0020,0012;0020,0013;0020,0032;0020,0037;0020,0052;0020,0100;0020,4000;0028,0004;0028,0008;0028,0010;0028,0011;0028,0030;0028,0100;0028,0101;0028,0103;0028,1050;0028,1051;0028,1052;0028,1053;0054,1330",
         "Patient" : "0010,0010;0010,0020;0010,0030;0010,0040;0010,1000",
         "Series" : "0008,0021;0008,0031;0008,0060;0008,0070;0008,0201;0008,1010;0008,103e;0008,1070;0018,0010;0018,0015;0018,0024;0018,1030;0018,1090;0018,1400;0020,000e;0020,0011;0020,0037;0020,0105;0020,1002;0040,0244;0040,0245;0040,0254;0040,0275;0054,0081;0054,0101;0054,1000",
         "Study" : "0008,0020;0008,0030;0008,0050;0008,0080;0008,0090;0008,0201;0008,1030;0020,000d;0020,0010;0032,1032;0032,1060"
      },
      "OrthancVersion" : "1.12.0",
      "StorageCompressionEnabled" : true
   },
   "LastChangeToProcess" : 58114506,
   "LastProcessedChange" : 47421879,
   "LastProcessedConfiguration" : 
   {
      "IngestTranscoding" : "",
      "MainDicomTagsSignature" : 
      {
         "Instance" : "0008,0012;0008,0013;0008,0018;0020,0012;0020,0013;0020,0032;0020,0037;0020,0100;0020,4000;0028,0008;0054,1330",
         "Patient" : "0010,0010;0010,0020;0010,0030;0010,0040;0010,1000",
         "Series" : "0008,0021;0008,0031;0008,0060;0008,0070;0008,1010;0008,103e;0008,1070;0018,0010;0018,0015;0018,0024;0018,1030;0018,1090;0018,1400;0020,000e;0020,0011;0020,0037;0020,0105;0020,1002;0040,0254;0054,0081;0054,0101;0054,1000",
         "Study" : "0008,0020;0008,0030;0008,0050;0008,0080;0008,0090;0008,1030;0020,000d;0020,0010;0032,1032;0032,1060"
      },
      "OrthancVersion" : "1.9.0",
      "StorageCompressionEnabled" : false
   },
   "LastTimeStarted" : "20230908T171222.831510",
   "Version" : 1
}

second entry’s value:

{
   "CurrentlyProcessingConfiguration" : 
   {
      "IngestTranscoding" : "1.2.840.10008.1.2.4.70",
      "MainDicomTagsSignature" : 
      {
         "Instance" : "0008,0005;0008,0008;0008,0012;0008,0013;0008,0016;0008,0018;0008,0023;0008,0033;0018,0050;0020,0012;0020,0013;0020,0032;0020,0037;0020,0052;0020,0100;0020,4000;0028,0004;0028,0008;0028,0010;0028,0011;0028,0030;0028,0100;0028,0101;0028,0103;0028,1050;0028,1051;0028,1052;0028,1053;0054,1330",
         "Patient" : "0010,0010;0010,0020;0010,0030;0010,0040;0010,1000",
         "Series" : "0008,0021;0008,0031;0008,0060;0008,0070;0008,0201;0008,1010;0008,103e;0008,1070;0018,0010;0018,0015;0018,0024;0018,1030;0018,1090;0018,1400;0020,000e;0020,0011;0020,0037;0020,0105;0020,1002;0040,0244;0040,0245;0040,0254;0040,0275;0054,0081;0054,0101;0054,1000",
         "Study" : "0008,0020;0008,0030;0008,0050;0008,0080;0008,0090;0008,0201;0008,1030;0020,000d;0020,0010;0032,1032;0032,1060"
      },
      "OrthancVersion" : "1.12.0",
      "StorageCompressionEnabled" : true
   },
   "LastChangeToProcess" : 60567932,
   "LastProcessedChange" : 29540550,
   "LastProcessedConfiguration" : 
   {
      "IngestTranscoding" : "",
      "MainDicomTagsSignature" : 
      {
         "Instance" : "0008,0012;0008,0013;0008,0018;0020,0012;0020,0013;0020,0032;0020,0037;0020,0100;0020,4000;0028,0008;0054,1330",
         "Patient" : "0010,0010;0010,0020;0010,0030;0010,0040;0010,1000",
         "Series" : "0008,0021;0008,0031;0008,0060;0008,0070;0008,1010;0008,103e;0008,1070;0018,0010;0018,0015;0018,0024;0018,1030;0018,1090;0018,1400;0020,000e;0020,0011;0020,0037;0020,0105;0020,1002;0040,0254;0054,0081;0054,0101;0054,1000",
         "Study" : "0008,0020;0008,0030;0008,0050;0008,0080;0008,0090;0008,1030;0020,000d;0020,0010;0032,1032;0032,1060"
      },
      "OrthancVersion" : "1.9.0",
      "StorageCompressionEnabled" : false
   },
   "LastTimeStarted" : "20230919T144339.706308",
   "Version" : 1
}

I tried to reproduce it by restarting the orthanc container. Also tried removing and then recreating it, neither created new entries in the serverproperties table.
Notice that that both CurrentlyProcessingConfiguration and LastProcessedConfiguration are unchanged between the two entries, only the timestamps and the change counts are different.

So my questions:

  • When is Orthanc supposed to create a new entry in the serverproperties table with a new UUID for the Housekeeper?
  • What could have happened that made our Housekeeper restart processing?
  • Where does the “server” column’s UUID come from?

I found this similar question: Housekepper plugin the DB configuration has changed since last run, will reprocess the whole DB ! but what Alain said there is not relevant here I think, because our first processing already took into account that old dicom-as-json files might be there.

2

Hi András,

Thanks for reporting this issue. I have just updated the Housekeeper plugin documentation with this additional text:

The housekeeper plugin saves its state in the Orthanc SQL database in the ServerProperties table. When running the Housekeeper plugin in a Docker environment, it is mandatory to set the DatabaseServerIdentifier configuration to make sure that an Orthanc container keeps the same identifier over multiple runs which should prevent the Housekeeper plugin to start again from scratch when Orthanc is restarted. If not set explicitly in the configuration, the DatabaseServerIdentifier might change from one run to the other.

Note that, even if you don’t use the Housekeeper plugin, it is recommended to set DatabaseServerIdentifier as soon as you use Docker since other settings like the jobs are using this id.

HTH,

Alain.

2 Likes