Database could not serialize access due to concurrent update

General
1

We are using Orthanc for about two years now as a larger installation with 10 TB of data on Alma Linux with all binaries, including plugins, being compiled from source. For over half a year now I had an Orthanc version running which I compiled back in Nov 2022 and recently I decided to do a system update and also recompile Orthanc and update the binaries and plugins to the latest version. Updating went smooth and everything is still working fine, however, I now sometimes have an error in the log files from the Database plugin:

E0711 14:30:04.268936 PluginsManager.cpp:153] Exception in database back-end: Database could not serialize access due to concurrent update, the transaction should be retried

We are using PostgreSQL as backend using the official plugin with multiple writers because the Orthanc setup consists of two Orthanc instances, one storage/receiver (only enabling C-STORE), and one client instance (C-MOVE, C-FIND). The REST-API of the client instance is queried on a regular basis by two Python scripts (auto-forward and auto-query) and also supplies a custom build web-interface (python/django) via the REST-API. So there is quite some traffic on the server, although hardware load is still minimal.

I do suspect that something is off with our database setup/config, although I do find it strange that this error did not occur before I updated Orthanc and the plugins to the latest versions and interestingly the error only occurs in the storage instance of Orthanc and never in the client instance.

Any ideas or help in debugging this further or tweaking the config would be highly appreciated :slight_smile:

The relevant configs:

storage.json

{
    "PostgreSQL" : {
      "EnableIndex" : true,
      "EnableStorage" : false,
      "Host" : "localhost",
      "Port" : 5432,
      "EnableSsl" : false,
      "Lock" : false,
      "MaximumConnectionRetries" : 10,
      "ConnectionRetryInterval" : 5,
      "IndexConnectionsCount" : 12
    },
    [...]
    "StorageCompression": false,
    "StorageAccessOnFind": "Never",
    "SaveJobs" : true,
    "JobsHistorySize": 100,
    "LimitFindResults" : 100,
    "LimitFindInstances" : 100,
    "TcpNoDelay" : true,
    "KeepAlive" : true,
    "DicomThreadsCount": 12,
    "MaximumStorageCacheSize": 512,
    "SyncStorageArea": false
  }

client.json

{
    "PostgreSQL" : {
      "EnableIndex" : true,
      "EnableStorage" : false,
      "Host" : "localhost",
      "Port" : 5432,
      "EnableSsl" : false,
      "Lock" : false,
      "MaximumConnectionRetries" : 10,
      "ConnectionRetryInterval" : 5,
      "IndexConnectionsCount" : 5
    },
    [...],
    "StorageCompression": false,
    "StorageAccessOnFind": "Never",
    "SaveJobs" : true,
    "JobsHistorySize": 100,
    "LimitFindResults" : 2000,
    "LimitFindInstances" : 2000,
    "TcpNoDelay" : true,
    "KeepAlive" : true,
    "DicomThreadsCount": 6,
    "MaximumStorageCacheSize": 512,
    "SyncStorageArea": false
  }
1 Like
2

Most recently also additional errors popped up:

E0712 07:25:37.692519 PluginsManager.cpp:153] PostgreSQL error: ERROR:  could not serialize access due to read/write dependencies among transactions
DETAIL:  Reason code: Canceled on identification as a pivot, during commit attempt.
HINT:  The transaction might succeed if retried.
WARNING:  there is no transaction in progress
E0712 07:25:37.693245 PluginsManager.cpp:153] Exception in database back-end: Error with the database engine
E0712 07:25:37.693322 PluginsManager.cpp:153] Exception in database back-end: Bad sequence of calls
E0712 07:25:37.693438 ServerContext.cpp:304] Cannot serialize the jobs engine: Error with the database engine

which does indeed suggest that there is a something wrong with the configuration :frowning:

3

Hi Martin,

This is actually not an error and this should be logged as a warning (see our TODO).

The transaction is actually retried and succeeds right after that.

HTH,

Alain

2 Likes
4

Hi Alain,

This basically means that the database was busy and that orthanc will retry the transaction according to MaximumConnectionRetries and ConnectionRetryInterval?

Thank you for the information. It just had me worried a bit :slight_smile:

best wishes
Martin

5

That’s correct !

Alain

2 Likes