Very slow anonymization

Google Groups archive
1

Hi all,

I’m running Orthanc on Windows, using PostgreSQL as the database. I have about 1TB of CT images (~15,000 studies) already in Orthanc, which I would like to anonymize. I’m using a Python script to do this via the REST API.

I have found that the anonymization process is very, very slow. If I anonymize a CT study with 500 images, ~110MB, it takes just over a minute. It’s writing the anonymized studies at about 2MB/s. At this rate it will take almost a week to anonymize the entire database! This is on a dual-core Xeon workstation with fairly fast SATA disks.

Are there any tricks to making this go faster? Should I have used Linux instead?

Thanks,

Joe

2

Hello,

Are you using Postgres for both the index and storage, or just for the
index?

To troubleshoot this you'll need some monitoring data on the various
resources of the machine: CPU usage, memory usage, storage I/O, even
network I/O (it could just be waiting on network roundtrips
intermittently with dramatic effects on the overall performance). It
will be best to gather samples of these over time. Once you've isolated
the bottleneck, you can search for the culprit.

Just in case I would also run storage I/O benchmarks that do random
reads and writes at the same time; if it's a mechanical drive seeking
operations sometimes have a big effect on throughput).

Good luck,

3

Please also check the version of Orthanc you are using. There were some performance issues with 1.1 that have been fixed in 1.2

4

Turns out the problem was Windows Defender. I excluded the Orthanc data directory and throughput increased 5x.

Just to answer the rest of your questions, this was the latest Orthanc 1.2 for Windows. Postgres is used only for the index, not storage. Postgres is on SSD, but the data is on spinning disk. All connections (scripts, postgres, etc.) were pointed to 127.0.0.1 not localhost.

Thanks – Joe