Hello,
I have relatively large database (12TB, 52k studies, mysql db about 40GB). I’m using mysql plugin for indexing.
The problems appears when I delete some study with a large number of instances.
For example (output from my script):
Study e50e36af-28e7fb0d-cf05cf49-fd6addcd-c374d037 will be processed. Study size: 986 MB. Mode: move
Instance ID: c29dfc86-2ecffed8-8ffc70e4-6cce5a6b-a896e516 Number: 1446 of 1446
Study CountInstances and DicomDiskSize matches. Transfer is successful.
Study e50e36af-28e7fb0d-cf05cf49-fd6addcd-c374d037 is deleted on source server
Processing time 0:19:22.506793 minutes and seconds
Study with 1446 instances is deleted for more than 19 minutes. During that time none of the clients can retrieve any information from the server.
I delete older than XXX days studies and most of the people search the studies from the last day.
I suspect the mysql database for this issue. The disks can easily achieve 10K IOPS and deleting 1446 files would take less than a second. It seems some lock but I’m not sure why this happens when InnoDB should not lock the tables on delete.
Any ideas how to proceed?
Kind regards,
Nikolay
I have a typo in the subject but cannot edit it.
Hello. I faced this problem a few months ago: https://groups.google.com/g/orthanc-users/c/C9cjV9ro9cM/m/fEmLkoYdAQAJ
You can’t really solve the issue at this moment because Orthanc uses SERIALIZABLE isolation level. To mitigate the problem, you can implement a multi-writer architecture (e.g with Docker) where you switch writer instances that needs to delete exams.
A domingo, 12 de junho de 2022 à(s) 17:40:01 UTC+1, Nikolay Zaynelov escreveu:
Hello,
Please check out the FAQ about scalability: “Deleting large studies can take much time, because removing a large number of files from a filesystem can be an expensive operation (which might sound counter-intuitive). This is especially true with HDD drives, that can be much slower than SSD (an user has reported a 20 times speedup by switching from HDD to SSD). If switching from HDD to SDD is not applicable, it is possible to create an storage area plugin that delays the actual deletion from the filesystem. The plugin would maintain a queue (e.g. as a SQLite database) of files to be removed. The actual deletion from the filesystem would be done asynchronously in a separate thread. We are looking for funding from the industry to implement such a plugin.”
https://book.orthanc-server.com/faq/scalability.html#slow-deletions
Regards,
Sébastien-
Hi everyone,
This slow deletion issue should be solved by the Delayed Deletion plugin.
Great thanks to Nikolay and his company for funding this plugin !
Alain