Slow import

Google Groups archive
1

I was really amazed with plots in Orthanc twitter about postgresql-plugin performance. Mostly because it looks like import speed doesn’t depends on data-base size.
But after I’ve imported a part of our CT database (about 400k instances, 248GB) import speed have been decreased from 20 images/s to 4 images/s.
I’m not postresql expert so my postgres.conf may be not optimal, but it doesn’t seems then any changes in buffer and chunks sizes helps.
Also I’ve find some suspicious in stats log.

SELECT CAST(SUM(compressedSize) AS BIGINT) FROM AttachedFiles

This query occurs during each import operation and it takes about 0.2s to complete it.

Is there a way to speed up import procedure, without casting multiframe dicoms?

2

It looks like my problem with perfomance on large database is here

ServerIndex.cpp:


public:
`
Transaction(ServerIndex& index) :
index_(index),
isCommitted_(false)
{
transaction_.reset(index_.db_.StartTransaction());
transaction_->Begin();

assert(index_.currentStorageSize_ == index_.db_.GetTotalCompressedSize());

index_.listener_->StartTransaction();
}

`

Without this assert transfer speed increases to 30 images/s.
Do we really need to check storage size after each transaction? Is it safe to pass it?
I’m worry because I never do anything in c++, and two slashes I placed to comment assertion, are my first expirience of writing c++ code)

3

Hello,

Calls to “assert()” corresponds to assertion-driven development:
https://en.wikipedia.org/wiki/Assertion_(software_development)

These calls are there to ensure some invariants during development, and are automatically disabled in release builds (in which the “NDEBUG” macro is set).

Please make sure to set the option “-DCMAKE_BUILD_TYPE=Release” while invoking CMake, this will turn off all these safety checks and will bring full speed to Orthanc.

HTH,
Sébastien-