Performance question

Google Groups archive
1

Hi there,

I have some technical questions about orthanc performances and optimisations.

Just to explain my goal, I'm building a system to share dicom through a web platform. For this I made a php app that deal with 2 orthanc servers (running in docker) one is exposed to the internet to receive dicom through orthanc peer and the second one is only accessible in localhost and act as dicom database system.

An institution is going to run this app(that will be also publicly available as a free and open source software, in few months).
This institution can dedicate a sufficient amount of money to buy the correct cloud infrastructure but I wanted to address some questions to choose the best performance / price ratio for the server (even if they can afford it I think it would be useless to overpay an overkill server).

Currently I'm making my test in Azure cloud with virtual instance of 2vGPu, 16 Gb of ram, 100Gb of SSD hard disk for Os, and 2*500 raid 0 UltraSSD hard disk for dicom storage(an a saas MySQL instance).

Docker configuration run 2 containers of orthanc and 1 MySQL container just for orthanc.
The volumes of the orthanc storage are mounted into the 1Tb raid 0 partition.

The questions are :

1) I read in orthanc documentation that the postgres/MySQL partition should be on a fast hard disk but the drive speed is less important for orthanc storage hard disk.
Why the hard disk is less impacting performances ?
My experience is that the multiplicity of small dicom files are very sensitive to the access time and SSD hard disk have a big advantage for that.
Is there some knowledge about the SSD vs HDD impact of storage (only storage folder, not database that remain on a SSD in all cases).
In short, is there some performance drop that I should see if I move to SSD to HDD ?
Can Osimis share it's experience when maintaining large hospital database, is HDD or SSD disk used ?
Eventually 2 or 4 HDD raid 0 could be a compromise compared to SSD performance ?
The HDD price is Azure is 4 times cheaper than SSD so that an important factor.

2) also in orthanc documentation I saw that compression settings is not recommended for performance.
I guess the compression is going to ask more CPU / RAM ressources but going to higher virtual instance with 8 or more vCPU is not really high price in Azure.
Is there some known hardware specification for which the compression setting will become non significant for performance ? Or is it going anyway slower with compression even with high powered hardware ?
I have read that the other side effect of compression is that migration will be harder to do needed time to make the uncompression before reimporting images. Is there other else limitations ?

3) a bit different question. Im facing a situation where I loose time a bit stupidly.
As my orthanc storage is localhost only it is managed only by my PHP app.
When a user ask to download dicoms the PHP server make a request to orthanc to make a zip archive, put the zip archive answer to a temp file and read it for the client browser download.
I saw that orthanc when building a zip is also making a temporary file before reading it in the rest API
So there is a bit strange sequence, orthanc is making a temp zip to read it for PHP, php store it in another temp file (as the zip may be bigger than the RAM) and read it again to the client.
Is there a way it could be more efficient without duplicating these temp file in the server ?
In particular is there a possibility one day to drive orthanc to generate a zip archive of dicom into a specific folder (specified in rest for example) and return only it's final location in Rest so php could consume it directly ?
It might be a Rest design violation, I have this just vague idea.
However the system works as it is now so that not a terrible issue (only optimization)

4) to finish by a simple question. To store the received dicom PHP ask for orthanc peer transfer from orthanc exposed session to the orthanc storage. The orthanc peer transfer is from localhost to localhost. With the peer accelerator I didn't notice any gain for this localhost transfer, but I didn't test for massive transfer.
I guess in a localhost to localhost the accelerator is useless as the ping is almost 0 and bandwidth unlimited, so the orthanc transert (which is extremely convenient for distant peer) can even slow down the localhost transfer due to zip generation /unziping before sending data. I'm I correct ?

Thanks for the time spent on reading this message,

Best regards,

Salim

2

I can try and help answer your first two questions:

  1. it is a combination, database is easier/cheaper to put on fast disk and is the first line of access, when you query, it is the database, only when you grab an image does the image disk get used, so general usage you will see a bigger bang for your buck with faster access on the database, especially with large queries and such. From my non-Orthanc experience, but other PACS systems, spinning disk is always used for images, usually slower disks, because when we get into 100’s of TB’s cost is a concern. Regarding RAID0, I would not recommend that at all unless you are OK with losing images.

  2. Can you reference where compression isn’t recommended? If they mean for example J2K Lossless, there should be no concern if images are already compressed when received and Orthanc is just processing the images for storage or transfer. I would see no reason why performance would be affected greatly when viewing either within Orthanc.

3

Dear Brian,

I'm safe with Raid 0 as the replication is made by the cloud service (all hard disk are virtualized instances)

In the documentation it is written

The Orthanc configuration file should have the following values for performance-related options (but make sure to understand their implications):
StorageCompression = false

https://book.orthanc-server.com/faq/scalability.html

So I was wondering if this recommendation is still valid for high powered hardware such as 8 vCPU with more than 32 gb of ram.

On the SSD /HDD speed my PHP app has already the series UID in memory. The API I use the most is the archive API to generate the ZIP containing the dicoms.
So basically the query in orthanc is limited but the dicom reading is intensive to generate the zip that usually contains around 1500 dicom files.

Best regards

Salim

4

Depending on the cloud environment, any RAID may not be useful as it is already RAID’ed behind the scenes, may want to check into that. May be adding more complications than required.

I would suggest checking into your DIOCM studies and identifying if they are compressed already with Lossless compression. If so other compression will just be adding work but not doing much.

The compression mentioned in that document is just storage compression, it does not help sending or receiving images, in my understanding (Orthanc team please let me know if I am wrong) if you use StorageCompression = true then the images will be compressed coming in, then compressed going out, so there would be more work involved and it would be providing no benefit, only an added performance loss as the images would be decompressed individually then zipped to be downloaded. My reference: https://book.orthanc-server.com/faq/features.html

5

The compression mentioned in that document is just storage compression, it does not help sending or receiving images, in my understanding (Orthanc team please let me know if I am wrong) if you use StorageCompression = true then the images will be compressed coming in, then compressed going out, so there would be more work involved and it would be providing no benefit, only an added performance loss as the images would be decompressed individually then zipped to be downloaded. My reference: https://book.orthanc-server.com/faq/features.html

=> I confirm this.

As of Orthanc 1.5.6, Orthanc never change the transfer syntax of the DICOM files (no transcoding).

The Orthanc configuration file should have the following values for performance-related options (but make sure to understand their implications):
StorageCompression = false

https://book.orthanc-server.com/faq/scalability.html

So I was wondering if this recommendation is still valid for high powered hardware such as 8 vCPU with more than 32 gb of ram.

=> Storage compression is quite cheap (it is similar to “gzip”). If you have such a powerful CPU, you should not see any impact on performance. This compression can reduce the storage by 25%-50%, which can have an extremely positive impact if using a cloud storage.

In either case, the answer to your initial questions is to make careful performance testing of your setup. Performance depends on a huge range of factors, including the GNU/Linux distribution and the filesystem format. Tip: Give a try to XFS:

Sébastien-

6

Back to the original post:

  1. […] In particular is there a possibility one day to drive orthanc to generate a zip archive of dicom into a specific folder (specified in rest for example) and return only it’s final location in Rest so php could consume it directly ? […]

=> This could easily be done by an Orthanc plugin:
https://book.orthanc-server.com/developers/creating-plugins.html

  1. […] I guess in a localhost to localhost the accelerator is useless as the ping is almost 0 and bandwidth unlimited, so the orthanc transert (which is extremely convenient for distant peer) can even slow down the localhost transfer due to zip generation /unziping before sending data. I’m I correct ?

Indeed. The transfers accelerator plugin only makes sense over remote TCP connections.

7

Thanks Sebastien for your answers,
You are correct it seems that the XFS partition is faster than Ext4 to generate Zip archive but I didn’t made a precise benchmark.
I’m now trying a Raid0 HDD on the server for storage, performances seems pretty good.

Thanks for you help (BTW i will send you the abstracts in the next few days)

Best regards,

Salim