Issue with Orthanc hanging when script executes with jobs running

I mentioned this in previous posts, but at least now it seems that there is a way to recreate the problem, even though I haven’t seen a specific error message in the log, just a hang or a crash. I added an if statement to check if the study isStable or not as part of the debugging process and it returns the Busy message if the study is being modified and not stable, and otherwise works fine.

The crash/hang occurs when the method is called (it is a Python Plug-in script) while jobs that are modifying other series that are part of the study are running. It seems that the create-dicom call is the problematic ones, although possibly one of the earlier ones.

Just wondering why that is, and if it might be possible to call /tools/create-dicom Asynchronously so it runs as a job, or otherwise does not cause a crash.

def MakeKeyImage(output, uri, **request):

POST has {“StudyInstanceUID”:“StudyInstanceUID”",“data-url”:“data-url”,“ImageComments”:“Some Info”})

if request[‘method’] != ‘POST’:
output.SendMethodNotAllowed(‘POST’)
else:
response = dict();
query = json.loads(request[‘body’])
study = json.loads(orthanc.RestApiPost(‘/tools/lookup’, query[‘StudyInstanceUID’]))[0] # assume a unique StudyInstancUID on server and that it exists
study = json.loads(orthanc.RestApiGet(‘/studies/%s’ % study[‘ID’]))
if study[‘IsStable’] == True:
series = json.loads(orthanc.RestApiGet(‘/studies/’ + study[“ID”] + ‘/series’)) # get the series uuid’s, there has to be at least one
keyimagesID = False
instancenumber = 1
for seriesitem in series:

Check to see if the ‘KEY_IMAGES’ series already exists, sort of assumes that this will always be unqiue on this system

if ‘SeriesDescription’ in seriesitem[‘MainDicomTags’] and seriesitem[‘MainDicomTags’][‘SeriesDescription’] == ‘KEY_IMAGES’:
keyimagesID = seriesitem[‘ID’]
instancenumber = len(seriesitem[‘Instances’]) + 1

dicomdata = dict()

1.2.840.10008.5.1.4.1.1.7 is SOPClassUID for Secondary Capture Image Storage

Set things up to add to an existing series, or create one if it does not already exist

if keyimagesID:
parent = keyimagesID
dicomdata[‘Tags’] = {“ImageComments”:query[‘ImageComments’],“SOPClassUID”:“1.2.840.10008.5.1.4.1.1.7.4”,“InstanceNumber”: str(instancenumber)}
else:
parent = study[“ID”]
dicomdata[‘Tags’] = {“Modality”:“OT”,“SeriesDescription”: “KEY_IMAGES”,“SequenceName” : “KEY_IMAGES”,“ImageComments”:query[‘ImageComments’],“SeriesNumber”:“0”,“SOPClassUID”:“1.2.840.10008.5.1.4.1.1.7.4”,“InstanceNumber”: str(instancenumber)}
dicomdata[‘Parent’] = parent

image to save is html converted to a jpeg image using wkhtmltoimage.

https://pypi.org/project/imgkit/

options = {

‘format’: ‘jpeg’,

‘encoding’: “UTF-8”

}

False means to store it to a variable rather than to a file on disk

jpegimage = imgkit.from_string(query[‘html’], False, options=options)

Need to be a data-url in base64

jpegimage = base64.b64encode(jpegimage).decode()

jpegimage = ‘data:image/jpeg;base64,’+jpegimage

jpegimage = query[‘data_url’]

logging.info(jpegimage)

dicomdata[‘Content’] = jpegimage
dicom = json.loads(orthanc.RestApiPost(‘/tools/create-dicom’, json.dumps(dicomdata)))
response[‘keyimagesID’] = keyimagesID
response[‘status’] = study
response[‘create-dicom’] = dicom
else:
response[‘status’] = “Unstable”
output.AnswerBuffer(json.dumps(response), ‘application/json’)

orthanc.RegisterRestCallback(‘/make_key_image’, MakeKeyImage)

Stephen D. Scotti

Hi Stephen,

Since it seems that other jobs are running at the same time, it’s virtually impossible to analyze what’s going on without a full reproducible simplified setup (docker setup + sample image + curl commands to trigger the jobs).

Best regards,

Alain

I might end up going to the Orthanc Con 2022 in Las Palmas. Now that I can reproduce it (on my laptop), I might just try running it by someone there, or otherwise keep fiddling with it on my system.

Adding the " if study[‘IsStable’] == True:" conditional actually does prevent the crash/hang from happening and not a big issue from the end-user standpoint because it was a rare, intermittent event that depends upon those jobs running at the same time apparently, or at least the Study being Unstable.

That is why it took awhile to track down the conditions that were causing it, although not a complete fix at this point.

/sds

Note that the OrthancCon is not a debugging/support session :wink: I, personally, won’t be in Las Palmas and I’m sure Sébastien will be more than busy handling the conference.

Just a follow up about this.

It seems that the culprit in the Python Plug-in scirpt is the one statement below, since if I comment that out nothing crashes, but that is the whole point of the script. I might be able to setup one of my GitHub repos with instructions on how to reproduce the problem. It seems to be reproducible.

dicom = json.loads(orthanc.RestApiPost(‘/tools/create-dicom’, json.dumps(dicomdata)))

  1. If I push instances via DICOM (e.g. using HOROS) that are part of the study that the script is adding an instance to using tools/create-dicom via the Python Plug-in script AND at the same time execute that script via the API (called from the Stone Viewer) using the /make_key_image path.
    OR
  2. One of my Jobs that sometimes modifies a Study (renaming a sequence) is running at the same time that the script is executed.

Checking for the study being stable seems to circumvent the problem.

However, I think that if I execute the same code in def MakeKeyImage(output, uri, **request): from an API (i.e. shell script, curl, or my own API) without using the Python script it works without hanging the system.

The strange thing is that I don’t see any error messages or codes anywhere. The system just ‘hangs’ and I have to restart the Orthanc container to resume operations.

Thanks.