Hi,
I use the object storage plugin to host old DICOM on Google Cloud Storage. All new files are stored on the filesystem and a task calls Orthanc to move old files to GCS.
Here’s my configuration :
ORTHANC__GOOGLE_CLOUD_STORAGE__BUCKET_NAME="bucket_name"
ORTHANC__GOOGLE_CLOUD_STORAGE__SERVICE_ACCOUNT_FILE="/etc/google-cloud-storage.json"
ORTHANC__GOOGLE_CLOUD_STORAGE__STORAGE_STRUCTURE="legacy"
ORTHANC__GOOGLE_CLOUD_STORAGE__HYBRID_MODE="WriteToFileSystem"
I got a strange error when uploading a new scanner (uploading to orthanc & local filesystem). It seems (sometimes?) Orthanc is trying to upload the files to the object storage (it shouldn’t) and after it wants to delete it (but can’t). Here’s a log of one file but it’s the same for all:
E0930 14:14:07.850133 HTTP-1 PluginsManager.cpp:153] Google Cloud Storage: error while deleting object 2c45f965-baca-483a-96bf-2d8a8f6b79a8: GoogleCloudStorage: error while deleting file 2c/45/2c45f965-baca-483a-96bf-2d8a8f6b79a8: Permanent error DeleteObject: No such object: bucket_name/2c/45/2c45f965-baca-483a-96bf-2d8a8f6b79a8
I0930 14:14:07.837059 HTTP-1 PluginsManager.cpp:161] (plugins) Google Cloud Storage (Secondary: object-storage): deleting attachment 2c45f965-baca-483a-96bf-2d8a8f6b79a8 of type 1
I0930 14:14:07.836697 HTTP-1 PluginsManager.cpp:161] (plugins) Google Cloud Storage (Primary: file-system): deleting attachment 2c45f965-baca-483a-96bf-2d8a8f6b79a8 of type 1
I0930 14:14:07.830476 HTTP-1 PluginsManager.cpp:161] (plugins) Google Cloud Storage (Primary: file-system): created attachment 2c45f965-baca-483a-96bf-2d8a8f6b79a8 (516.51KB in 25.27ms = 167.45Mbps)
I0930 14:14:07.805173 HTTP-1 PluginsManager.cpp:161] (plugins) Google Cloud Storage (Primary: file-system): creating attachment 2c45f965-baca-483a-96bf-2d8a8f6b79a8 of type 1
So, some uploads are working and some not.
Hi @MaximeG
The error message appears when trying to delete the file. Actually, the plugin does not know where the file is stored and it tries to delete it from both locations. The error should not appear → I have added a TODO for that.
BTW, it’s strange that the file is deleted directly after being added … any other information from the logs before or after ?
Alain
Hello,
This might be because this DICOM instance is already present in the Orthanc database. If OverwriteInstances is false (the default value), Orthanc will first store the DICOM instance onto the file storage (to be sure it is properly stored before applying any modification to the database), then notice that the DICOM instance is duplicated, and finally will remove the just-received DICOM instance.
Best,
Sébastien-
1 Like
Thx @alainmazy & @jodogne !
I’m a little confused, does it mean there is an issue / bug or not? Because for some scanners on the upload Orthanc raise an error (I’m looking into it…).
Or to you it’s just a logging error and it’s completely unrelated to our errors during the upload?
I tried to set the OverwriteInstances to true and you’re right, the scanner upload is now working.
I think something strange is happening here 
As explained above, this log is not an error, just an information message (the log line starts with I). Now, if you configure your cloud storage to prevent any deletion by Orthanc for security reasons, Orthanc will complain about not being able to delete files (which is obviously neither a bug, nor “something strange”, but your own configuration decision). Setting OverwriteInstances to true will bypass the need for deleting the files that are already stored.
Regards,
Sébastien-
Now, if you configure your cloud storage to prevent any deletion by Orthanc for security reasons, Orthanc will complain about not being able to delete files (which is obviously neither a bug, nor “something strange”, but your own configuration decision).
Oh, you’re right, but it’s a more mixed-up issue.
Since a few month Google Cloud Storage set the default Soft delete policy to true.
So, I think Orthanc (I didn’t checked the source code) enforce a real deletion, not a soft delete. That’s why we saw the log and our error. By setting OverwriteInstances to true we indeed bypass the issue.
Maybe the fix is to allow a soft delete on the object storage (at least on GCS)?
Source : https://cloud.google.com/resources/storage/soft-delete-announce
Hi @MaximeG
I think that whatever the configuration “OverwriteInstances”, Orthanc tries to delete some attachments when you re-upload the same instance:
- either to delete the previous instance in case
OverwriteInstances is true
- or to delete the newest instance that should not be kept in case
OverwriteInstances is false
As mentioned earlier, there are “false” error messages in this version of the plugin (not yet fixed - already in the TODO). So the best to clarify this topic is to launch a test Orthanc with a dedicated Google storage, upload a file and delete it and check if everything works fine (is the bucket empty after that).
Best regards,
Alain.
Hi @alainmazy
I trust your knowledge about Orthanc But as I mentioned before, I think the process on the Orthanc side is different when OverwriteInstances is true or false.
Why? Since I turned OverwriteInstances is true, I don’t see any “false” error messages, and the DICOM are correctly processed (it was not the case before, as mentioned).
So, as a conclusion I think the soft delete feature on GCS is an issue for Orthanc.
Ok, I believe you 
I have no opportunity to investigate that issue right now but I have referenced this conversation in our TODO. For the record, could you test with a new bucket, upload a file, delete it and check how many files you have in the bucket ? And, possibly reupload it and check again ?
Thanks
Alain.
