Upload Speed Degradation Over Time

General
1

I’m encountering a significant drop in upload speed with Orthanc when storing DICOM data from scratch. I am using Orthanc with PostgreSQL, both at their latest versions, and transferring data via the dcmsend command.

Initially, the upload speed is around 110 files per second. However, after approximately 2 hours, the speed drops to about 80 files per second, and after 3 hours, it further decreases to 10 files per second. Restarting Orthanc (without restarting PostgreSQL) temporarily restores the speed to around 80 files per second.

Could you help identify the cause of this issue and suggest possible solutions?

Thank you!

Below is the relationship between the number of ‘changes’ and the time (in minutes).
2024-06-19 08:55 | 6680
2024-06-19 08:56 | 6939
2024-06-19 08:57 | 6992
2024-06-19 08:58 | 6972
2024-06-19 08:59 | 6838
2024-06-19 09:00 | 6797
2024-06-19 09:01 | 6912
2024-06-19 09:02 | 6760
2024-06-19 09:03 | 6897
2024-06-19 09:04 | 6843
2024-06-19 09:05 | 6758
2024-06-19 09:06 | 6680
2024-06-19 09:07 | 6840
2024-06-19 09:08 | 6731
2024-06-19 09:09 | 6690
2024-06-19 09:10 | 6904
2024-06-19 09:11 | 6837
2024-06-19 09:12 | 6864
2024-06-19 09:13 | 6903
2024-06-19 09:14 | 6692
2024-06-19 09:15 | 6688
2024-06-19 09:16 | 6920
2024-06-19 09:17 | 6823
2024-06-19 09:18 | 6765
2024-06-19 09:19 | 6770
2024-06-19 09:20 | 6576
2024-06-19 09:21 | 6605
2024-06-19 09:22 | 6715
2024-06-19 09:23 | 6530
2024-06-19 09:24 | 6531
2024-06-19 09:25 | 6451
2024-06-19 09:26 | 6463
2024-06-19 09:27 | 6333
2024-06-19 09:28 | 6301
2024-06-19 09:29 | 6174
2024-06-19 09:30 | 6271
2024-06-19 09:31 | 6406
2024-06-19 09:32 | 6332
2024-06-19 09:33 | 6511
2024-06-19 09:34 | 6579
2024-06-19 09:35 | 6429
2024-06-19 09:36 | 6367
2024-06-19 09:37 | 6447
2024-06-19 09:38 | 6439
2024-06-19 09:39 | 6315
2024-06-19 09:40 | 6440
2024-06-19 09:41 | 6292
2024-06-19 09:42 | 6221
2024-06-19 09:43 | 6179
2024-06-19 09:44 | 6040
2024-06-19 09:45 | 6080
2024-06-19 09:46 | 6211
2024-06-19 09:47 | 5955
2024-06-19 09:48 | 5914
2024-06-19 09:49 | 5894
2024-06-19 09:50 | 5973
2024-06-19 09:51 | 5905
2024-06-19 09:52 | 5806
2024-06-19 09:53 | 5867
2024-06-19 09:54 | 5846
2024-06-19 09:55 | 5763
2024-06-19 09:56 | 5753
2024-06-19 09:57 | 5755
2024-06-19 09:58 | 5713
2024-06-19 09:59 | 5681
2024-06-19 10:00 | 5661
2024-06-19 10:01 | 5620
2024-06-19 10:02 | 5569
2024-06-19 10:03 | 5568
2024-06-19 10:04 | 5579
2024-06-19 10:05 | 5485
2024-06-19 10:06 | 5487
2024-06-19 10:07 | 5581
2024-06-19 10:08 | 5456
2024-06-19 10:09 | 5504
2024-06-19 10:10 | 5517
2024-06-19 10:11 | 5447
2024-06-19 10:12 | 5417
2024-06-19 10:13 | 5331
2024-06-19 10:14 | 5328
2024-06-19 10:15 | 5251
2024-06-19 10:16 | 5280
2024-06-19 10:17 | 5180
2024-06-19 10:18 | 5152
2024-06-19 10:19 | 5095
2024-06-19 10:20 | 5084
2024-06-19 10:21 | 5084
2024-06-19 10:22 | 4945
2024-06-19 10:23 | 4742
2024-06-19 10:24 | 4773
2024-06-19 10:25 | 4886
2024-06-19 10:26 | 4826
2024-06-19 10:27 | 4749
2024-06-19 10:28 | 4391
2024-06-19 10:29 | 4276
2024-06-19 10:30 | 4494
2024-06-19 10:31 | 4194
2024-06-19 10:32 | 4280
2024-06-19 10:33 | 4392
2024-06-19 10:34 | 4241
2024-06-19 10:35 | 3851
2024-06-19 10:36 | 3751
2024-06-19 10:37 | 3383
2024-06-19 10:38 | 2756
2024-06-19 10:39 | 2644
2024-06-19 10:40 | 2588
2024-06-19 10:41 | 2310
2024-06-19 10:42 | 2116
2024-06-19 10:43 | 1829
2024-06-19 10:44 | 1713
2024-06-19 10:45 | 1591
2024-06-19 10:46 | 1616
2024-06-19 10:47 | 1537
2024-06-19 10:48 | 1441
2024-06-19 10:49 | 1542
2024-06-19 10:50 | 1477
2024-06-19 10:51 | 1340
2024-06-19 10:52 | 1277
2024-06-19 10:53 | 1096
2024-06-19 10:54 | 1048
2024-06-19 10:55 | 1090
2024-06-19 10:56 | 1049
2024-06-19 10:57 | 1008
2024-06-19 10:58 | 946
2024-06-19 10:59 | 891
2024-06-19 11:00 | 867
2024-06-19 11:01 | 793
2024-06-19 11:02 | 797
2024-06-19 11:03 | 810
2024-06-19 11:04 | 814
2024-06-19 11:05 | 814
2024-06-19 11:06 | 718
2024-06-19 11:07 | 697
2024-06-19 11:08 | 769
2024-06-19 11:09 | 735
2024-06-19 11:10 | 683
2024-06-19 11:11 | 693
2024-06-19 11:12 | 629
2024-06-19 11:13 | 667
2024-06-19 11:14 | 646
2024-06-19 11:15 | 731
2024-06-19 11:16 | 701
2024-06-19 11:17 | 724
2024-06-19 11:18 | 670
2024-06-19 11:19 | 669
2024-06-19 11:20 | 607
2024-06-19 11:21 | 583
2024-06-19 11:22 | 592
2024-06-19 11:23 | 442
2024-06-19 11:24 | 507
2024-06-19 11:25 | 345
2024-06-19 11:26 | 3362
2024-06-19 11:27 | 4445
2024-06-19 11:28 | 4639
2024-06-19 11:29 | 4619
2024-06-19 11:30 | 4593
2024-06-19 11:31 | 4558
2024-06-19 11:32 | 4307
2024-06-19 11:33 | 4538
2024-06-19 11:34 | 4548
2024-06-19 11:35 | 4635
2024-06-19 11:36 | 4524
2024-06-19 11:37 | 4455
2024-06-19 11:38 | 4448
2024-06-19 11:39 | 4431
2024-06-19 11:40 | 4400
2024-06-19 11:41 | 3962

2

The data comes from the storage folder of another Orthanc service, starting from the 00/00 folder, and totals 2TB. The receiving Orthanc service previously did not store any data.

3

Hi @Titan

I’m afraid I do not have an “off the shelf” explanation for that. I remember running tests for days and not noticing any significant difference.

May I first ask you to run a script on your setup that fills Orthanc with a test DB ?
You first need to install orthanc-tools · PyPI through pip install orthanc-tools==0.12.10 and then run

python -m orthanc_tools.orthanc_test_db_populator --url=http://localhost:8042 --user=user --password=pwd --studies=10000 --seed=42

The script generates test images and uploads them in Orthanc. All images have only 4 pixels and take a minimum amount of space on disk - they are invalid and you can not visualize them but it should be good enough to validate the performance.

Check if you observe the same behavior by monitoring the /changes like you did. If yes, please share your exact configuration such that I can hopefully reproduce it here.

You might also want to capture verbose logs at the beginning of the process and once it is slower to check the timing of each operation.

Best regards,

Alain.