Is there a practical size limit to SQLite when using the Indexer mode via the Advanced Storage plugin?

General
1

Hi,

This is more of a theoretical/practical question than a specific bug/issue, but I am working on building an internal PACS for our research group. We have about 10-20TB of multi-modal image data (depending on what data we choose to include) and upwards of 50 million DICOM files/instances.

Since we already have these data stored on disk (and they aren’t growing), we are planning to use the Advanced Storage plugin using the Indexer mode to simply index the existing content of our file system.

Is this size of data potentially too large for using the default SQLite-based indexing? I have read through various parts of the Orthanc book that seem to suggest that a more enterprise-ready approach is usually needed when scaling up to this size, but this always seems to be presented in the context of using the Orthanc Storage to upload/store the DICOM instances, so I’m not sure if this applies to the situation of using the Indexer mode.

Thanks in advance for any insights.

2

Hi @mattwrkntn

Actually Orthanc needs to store the same amount of data in the SQL database in indexer mode or in normal mode. The SQL DB will even be a little larger in indexer mode since Orthanc will store the path of each file in the DB which is not required in normal mode.

Therefore, for 50 millions instances, I strongly recommend using PostgreSQL.
In case you missed it, I recently added a page about “Which Database engine should I use ?” in the Orthanc book.

Best,

Alain

1 Like
3

Thanks for the response, @alainmazy. This is very helpful.

And thanks for sharing that link, I had not seen this yet.