OrthancPlugins RestApiPost is consuming much RAM than RestApiGet

Google Groups archive
1

Hi Authors,

I notices that using RestApiPost is consuming very much RAM. This is the sample code in my plugin

Case 1- Using RestApiPost
int size_t num = 1000000;
std::string instanceId = 1.2.3.4.5.6;
for (size_it i = 0; i < num; ++i)
{
std::string dicom;
Json::Value body = Json::objectValue;
body[“Keep”].append(“StudyInstanceUID”);
body[“Keep”].append(“SeriesInstanceUID”);
body[“Keep”].append(“SeriesDescription”); // VinDr relies on this field
body[“Force”] = true;
OrthancPlugins::MemoryBuffer answer;
answer.RestApiPost(“/instances/” + instanceId + “/anonymize”, body, false);
answer.ToString(dicom);
}

Case 2 - Using RestApiGet
int size_t num = 1000000;
std::string instanceId = 1.2.3.4.5.6;
for (size_it i = 0; i < num; ++i)
{
std::string dicom;

OrthancPlugins::RestApiGetString(dicom, “/instances/” + instanceId + “/file”, false)

}

Case 1 : RAM increases from 30Mb to 800Mb
Case 2: RAM almost does not increase

I can use the MALLOC_ARENA_MAX to control the memory in case 1. But I wonder why calling the internal SDK (like calling function) causes high memory consuming ?

Thanks,
Chris

2

Hi Christopher,

Could you share more info about your test setup such that we can fully reproduce it here ? (https://book.orthanc-server.com/users/support.html#discussing-a-minimal-working-example)

On my side, I’ve compiled a plugin with your code and, after 8 minutes of execution, I don’t see any significant increase in memory usage with htop:

image.png

image.png

I have also run it with valgrind and no memory leaks were detected.

Best regards,

Alain.

3

Hi Alain,
This is my plugin. I mimic the StowClientJob class in DicomWebClient.cpp (https://book.orthanc-server.com/plugins/dicomweb.html).

This is my configuration (in orthanc.json) for running the plugin

“MyConfig”: {
“Enable”: true,
“Anonymized”: false,
“Url”: “http://localhost:8080/test”,
“Timeout”: 240
},

The config Anonymized is controlling the following snippet of code. If I set it true, it means Plugin uses RestApiPost and memory consumption is very large. If I set it false, it means Plugin uses RestApiGetString, and memory does not increase

if (ANONYMIZED) {
Json::Value body = Json::objectValue;
body[“Keep”].append(“StudyInstanceUID”);
body[“Keep”].append(“SeriesInstanceUID”);
body[“Force”] = true;
OrthancPlugins::MemoryBuffer answer;
if (answer.RestApiPost(“/instances/” + instanceId + “/anonymize”, body, false))
{
answer.ToString(dicom);
networkSize_ += dicom.size();
context.SetContent(“NetworkSizeMB”, boost::lexical_caststd::string
(networkSize_ / static_cast<uint64_t>(1024 * 1024)));

return true;
}
}

else
{
if (OrthancPlugins::RestApiGetString(dicom, “/instances/” + instanceId + “/file”, false))
{
networkSize_ += dicom.size();
context.SetContent(“NetworkSizeMB”, boost::lexical_caststd::string
(networkSize_ / static_cast<uint64_t>(1024 * 1024)));

return true;
}
}

This is the command line to send to the Orthanc to test the plugin
for i in {1…20}; do curl -i localhost:8042/mytest/b16a0924-a19d7cc3-1199297d-885ca0c4-bad22280;done

Plugin.cpp (15.8 KB)

4

This capture when using RestApiGetString
1.png

This capture when using RestApiPost

2.png

5

This is the valgrind I ran if you need more log

==136897== 428,064 bytes in 98 blocks are possibly lost in loss record 3,288 of 3,301
==136897== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x725DA6: sqlite3MemMalloc (sqlite3.c:22838)
==136897== by 0x726767: mallocWithAlarm (sqlite3.c:26670)
==136897== by 0x726814: sqlite3Malloc (sqlite3.c:26700)
==136897== by 0x7356F3: pcache1Alloc (sqlite3.c:48882)
==136897== by 0x7358E7: pcache1AllocPage (sqlite3.c:48978)
==136897== by 0x73636D: pcache1FetchStage2 (sqlite3.c:49447)
==136897== by 0x7364D8: pcache1FetchNoMutex (sqlite3.c:49551)
==136897== by 0x73650B: pcache1Fetch (sqlite3.c:49593)
==136897== by 0x7347CD: sqlite3PcacheFetch (sqlite3.c:48031)
==136897== by 0x73C121: getPageNormal (sqlite3.c:55939)
==136897== by 0x73C5C9: sqlite3PagerGet (sqlite3.c:56118)
==136897==
==136897== 431,728 bytes in 4,906 blocks are still reachable in loss record 3,289 of 3,301
==136897== at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x8973B4: DcmDataDictionary::loadDictionary(char const*, bool) (dcdict.cc:556)
==136897== by 0x814A68: Orthanc::LoadEmbeddedDictionary(DcmDataDictionary&, Orthanc::FrameworkResources::FileResourceId) (FromDcmtkBridge.cpp:146)
==136897== by 0x814D09: Orthanc::FromDcmtkBridge::InitializeDictionary(bool) (FromDcmtkBridge.cpp:307)
==136897== by 0x4203A9: Orthanc::InitializeFramework(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool) (OrthancFramework.cpp:82)
==136897== by 0x284EF4: Orthanc::OrthancInitialize(char const*) (OrthancInitialization.cpp:287)
==136897== by 0x205655: main (main.cpp:1982)
==136897==
==136897== 524,288 bytes in 1 blocks are still reachable in loss record 3,290 of 3,301
==136897== at 0x483C813: operator new[](unsigned long, std::nothrow_t const&) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x8A341C: DcmElement::DcmElement(DcmElement const&) (dcelem.cc:106)
==136897== by 0x90E81E: DcmOtherByteOtherWord::DcmOtherByteOtherWord(DcmOtherByteOtherWord const&) (dcvrobow.cc:62)
==136897== by 0x917FC2: DcmPolymorphOBOW::DcmPolymorphOBOW(DcmPolymorphOBOW const&) (dcvrpobw.cc:42)
==136897== by 0x8DE1CC: DcmPixelData::DcmPixelData(DcmPixelData const&) (dcpixel.cc:109)
==136897== by 0x8E4972: DcmPixelData::clone() const (dcpixel.h:277)
==136897== by 0x8BB5A4: DcmItem::DcmItem(DcmItem const&) (dcitem.cc:122)
==136897== by 0x886390: DcmDataset::DcmDataset(DcmDataset const&) (dcdatset.cc:76)
==136897== by 0x88B3A2: DcmDataset::clone() const (dcdatset.h:70)
==136897== by 0x8EF978: DcmSequenceOfItems::DcmSequenceOfItems(DcmSequenceOfItems const&) (dcsequen.cc:88)
==136897== by 0x8AF7DA: DcmFileFormat::DcmFileFormat(DcmFileFormat const&) (dcfilefo.cc:101)
==136897== by 0x8B74B6: DcmFileFormat::clone() const (dcfilefo.h:83)

==136897== 1,169,100 bytes in 25,980 blocks are still reachable in loss record 3,299 of 3,301
==136897== at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x729C38C: void std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) (in /home/phtran/orthanc/libOrthancVinDr.so)
==136897== by 0x72A1C82: (anonymous namespace)::SendToVinDr(_OrthancPluginRestOutput_t*, char const*, OrthancPluginHttpRequest const*) [clone .isra.0] (in /home/phtran/orthanc/libOrthancVinDr.so)
==136897== by 0x72A2565: OrthancPluginErrorCode OrthancPlugins::Internals::Protect<&(anonymous namespace)::SendToVinDr>(_OrthancPluginRestOutput_t*, char const*, OrthancPluginHttpRequest const*) (in /home/phtran/orthanc/libOrthancVinDr.so)
==136897== by 0x3ADC35: Orthanc::OrthancPlugins::PImpl::RestCallback::InvokeInternal(Orthanc::OrthancPlugins::PImpl::PluginHttpOutput&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, OrthancPluginHttpRequest const&) (OrthancPlugins.cpp:960)
==136897== by 0x3ADD2F: Orthanc::OrthancPlugins::PImpl::RestCallback::Invoke(boost::recursive_mutex&, Orthanc::OrthancPlugins::PImpl::PluginHttpOutput&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, OrthancPluginHttpRequest const&) (OrthancPlugins.cpp:992)
==136897== by 0x39CC52: Orthanc::OrthancPlugins::Handle(Orthanc::HttpOutput&, Orthanc::RequestOrigin, char const*, char const*, Orthanc::HttpMethod, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::less<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, void const*, unsigned long) (OrthancPlugins.cpp:2110)
==136897== by 0x283031: Orthanc::OrthancHttpHandler::Handle(Orthanc::HttpOutput&, Orthanc::RequestOrigin, char const*, char const*, Orthanc::HttpMethod, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::less<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, void const*, unsigned long) (OrthancHttpHandler.cpp:88)
==136897== by 0x492ECB: Orthanc::InternalCallback(Orthanc::HttpOutput&, Orthanc::HttpMethod&, Orthanc::HttpServer&, mg_connection*, mg_request_info const*) (HttpServer.cpp:1398)
==136897== by 0x4934DB: Orthanc::ProtectedCallback(mg_connection*, mg_request_info const*) (HttpServer.cpp:1448)
==136897== by 0x493CDD: Orthanc::Callback(mg_connection*) (HttpServer.cpp:1536)
==136897== by 0x56DA79: handle_request (civetweb.c:14007)
==136897==
==136897== 2,105,930 bytes in 2 blocks are still reachable in loss record 3,300 of 3,301
==136897== at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x49F935D: std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==136897== by 0x49F9E8F: std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_replace_aux(unsigned long, unsigned long, unsigned long, char) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==136897== by 0x9E3D99: Orthanc::ChunkedBuffer::Flatten(std::__cxx11::basic_string<char, std::char_traits, std::allocator >&) (ChunkedBuffer.cpp:173)
==136897== by 0x4881B2: Orthanc::HttpClient::CurlRequestBody::CallbackInternal(char*, unsigned long) (HttpClient.cpp:242)
==136897== by 0x488427: Orthanc::HttpClient::Curl==136897== 1,169,100 bytes in 25,980 blocks are still reachable in loss record 3,299 of 3,301
==136897== at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x729C38C: void std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) (in /home/phtran/orthanc/libOrthancVinDr.so)
==136897== by 0x72A1C82: (anonymous namespace)::SendToVinDr(_OrthancPluginRestOutput_t*, char const*, OrthancPluginHttpRequest const*) [clone .isra.0] (in /home/phtran/orthanc/libOrthancVinDr.so)
==136897== by 0x72A2565: OrthancPluginErrorCode OrthancPlugins::Internals::Protect<&(anonymous namespace)::SendToVinDr>(_OrthancPluginRestOutput_t*, char const*, OrthancPluginHttpRequest const*) (in /home/phtran/orthanc/libOrthancVinDr.so)
==136897== by 0x3ADC35: Orthanc::OrthancPlugins::PImpl::RestCallback::InvokeInternal(Orthanc::OrthancPlugins::PImpl::PluginHttpOutput&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, OrthancPluginHttpRequest const&) (OrthancPlugins.cpp:960)
==136897== by 0x3ADD2F: Orthanc::OrthancPlugins::PImpl::RestCallback::Invoke(boost::recursive_mutex&, Orthanc::OrthancPlugins::PImpl::PluginHttpOutput&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, OrthancPluginHttpRequest const&) (OrthancPlugins.cpp:992)
==136897== by 0x39CC52: Orthanc::OrthancPlugins::Handle(Orthanc::HttpOutput&, Orthanc::RequestOrigin, char const*, char const*, Orthanc::HttpMethod, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::less<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, void const*, unsigned long) (OrthancPlugins.cpp:2110)
==136897== by 0x283031: Orthanc::OrthancHttpHandler::Handle(Orthanc::HttpOutput&, Orthanc::RequestOrigin, char const*, char const*, Orthanc::HttpMethod, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::less<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, void const*, unsigned long) (OrthancHttpHandler.cpp:88)
==136897== by 0x492ECB: Orthanc::InternalCallback(Orthanc::HttpOutput&, Orthanc::HttpMethod&, Orthanc::HttpServer&, mg_connection*, mg_request_info const*) (HttpServer.cpp:1398)
==136897== by 0x4934DB: Orthanc::ProtectedCallback(mg_connection*, mg_request_info const*) (HttpServer.cpp:1448)
==136897== by 0x493CDD: Orthanc::Callback(mg_connection*) (HttpServer.cpp:1536)
==136897== by 0x56DA79: handle_request (civetweb.c:14007)
==136897==
==136897== 2,105,930 bytes in 2 blocks are still reachable in loss record 3,300 of 3,301
==136897== at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x49F935D: std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==136897== by 0x49F9E8F: std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_replace_aux(unsigned long, unsigned long, unsigned long, char) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28)
==136897== by 0x9E3D99: Orthanc::ChunkedBuffer::Flatten(std::__cxx11::basic_string<char, std::char_traits, std::allocator >&) (ChunkedBuffer.cpp:173)
==136897== by 0x4881B2: Orthanc::HttpClient::CurlRequestBody::CallbackInternal(char*, unsigned long) (HttpClient.cpp:242)
==136897== by 0x488427: Orthanc::HttpClient::CurlRequestBody::Callback(char*, unsigned long, unsigned long, void*) (HttpClient.cpp:286)
==136897== by 0x59B125: Curl_fillreadbuffer (transfer.c:247)
==136897== by 0x59C6C0: readwrite_upload (transfer.c:1007)
==136897== by 0x59CD2B: Curl_readwrite (transfer.c:1231)
==136897== by 0x5857EB: multi_runsingle (multi.c:2257)
==136897== by 0x585E6C: curl_multi_perform (multi.c:2520)
==136897== by 0x57788D: easy_transfer (easy.c:606)
==136897==
==136897== 65,011,712 bytes in 124 blocks are still reachable in loss record 3,301 of 3,301
==136897== at 0x483C813: operator new[](unsigned long, std::nothrow_t const&) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x8A59BD: DcmElement::newValueField() (dcelem.cc:784)
==136897== by 0x8A52C3: DcmElement::loadValue(DcmInputStream*) (dcelem.cc:686)
==136897== by 0x8A7674: DcmElement::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcelem.cc:1241)
==136897== by 0x918C4B: DcmPolymorphOBOW::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcvrpobw.cc:289)
==136897== by 0x8E1CF5: DcmPixelData::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcpixel.cc:890)
==136897== by 0x8C1ABF: DcmItem::readSubElement(DcmInputStream&, DcmTag&, unsigned int, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcitem.cc:1272)
==136897== by 0x8C33CD: DcmItem::readUntilTag(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int, DcmTagKey const&) (dcitem.cc:1449)
==136897== by 0x88849C: DcmDataset::readUntilTag(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int, DcmTagKey const&) (dcdatset.cc:440)
==136897== by 0x8B55D4: DcmFileFormat::readUntilTag(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int, DcmTagKey const&) (dcfilefo.cc:763)
==136897== by 0x8B5102: DcmFileFormat::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcfilefo.cc:700)
==136897== by 0x81CE1A: Orthanc::FromDcmtkBridge::LoadFromMemoryBuffer(void const*, unsigned long) (FromDcmtkBridge.cpp:2203)
==136897==
==136897== LEAK SUMMARY:
==136897== definitely lost: 96 bytes in 1 blocks
==136897== indirectly lost: 0 bytes in 0 blocks
==136897== possibly lost: 529,197 bytes in 1,054 blocks
==136897== still reachable: 81,562,200 bytes in 203,483 blocks
==136897== of which reachable via heuristic:
==136897== stdstring : 738 bytes in 18 blocks
==136897== length64 : 243,736 bytes in 336 blocks
==136897== newarray : 280 bytes in 2 blocks
==136897== suppressed: 0 bytes in 0 blocks
==136897==

==136897== ERROR SUMMARY: 118 errors from 118 contexts (suppressed: 0 from 0)RequestBody::Callback(char*, unsigned long, unsigned long, void*) (HttpClient.cpp:286)
==136897== by 0x59B125: Curl_fillreadbuffer (transfer.c:247)
==136897== by 0x59C6C0: readwrite_upload (transfer.c:1007)
==136897== by 0x59CD2B: Curl_readwrite (transfer.c:1231)
==136897== by 0x5857EB: multi_runsingle (multi.c:2257)
==136897== by 0x585E6C: curl_multi_perform (multi.c:2520)
==136897== by 0x57788D: easy_transfer (easy.c:606)
==136897==
==136897== 65,011,712 bytes in 124 blocks are still reachable in loss record 3,301 of 3,301
==136897== at 0x483C813: operator new[](unsigned long, std::nothrow_t const&) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==136897== by 0x8A59BD: DcmElement::newValueField() (dcelem.cc:784)
==136897== by 0x8A52C3: DcmElement::loadValue(DcmInputStream*) (dcelem.cc:686)
==136897== by 0x8A7674: DcmElement::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcelem.cc:1241)
==136897== by 0x918C4B: DcmPolymorphOBOW::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcvrpobw.cc:289)
==136897== by 0x8E1CF5: DcmPixelData::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcpixel.cc:890)
==136897== by 0x8C1ABF: DcmItem::readSubElement(DcmInputStream&, DcmTag&, unsigned int, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcitem.cc:1272)
==136897== by 0x8C33CD: DcmItem::readUntilTag(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int, DcmTagKey const&) (dcitem.cc:1449)
==136897== by 0x88849C: DcmDataset::readUntilTag(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int, DcmTagKey const&) (dcdatset.cc:440)
==136897== by 0x8B55D4: DcmFileFormat::readUntilTag(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int, DcmTagKey const&) (dcfilefo.cc:763)
==136897== by 0x8B5102: DcmFileFormat::read(DcmInputStream&, E_TransferSyntax, E_GrpLenEncoding, unsigned int) (dcfilefo.cc:700)
==136897== by 0x81CE1A: Orthanc::FromDcmtkBridge::LoadFromMemoryBuffer(void const*, unsigned long) (FromDcmtkBridge.cpp:2203)
==136897==
==136897== LEAK SUMMARY:
==136897== definitely lost: 96 bytes in 1 blocks
==136897== indirectly lost: 0 bytes in 0 blocks
==136897== possibly lost: 529,197 bytes in 1,054 blocks
==136897== still reachable: 81,562,200 bytes in 203,483 blocks
==136897== of which reachable via heuristic:
==136897== stdstring : 738 bytes in 18 blocks
==136897== length64 : 243,736 bytes in 336 blocks
==136897== newarray : 280 bytes in 2 blocks
==136897== suppressed: 0 bytes in 0 blocks
==136897==
==136897== ERROR SUMMARY: 118 errors from 118 contexts (suppressed: 0 from 0)

6

Hi Christopher,

Sorry but, still no memory increase and no memory leaks here but well, I don’t have a server to mockup your webservice and I don’t have the same DICOM files as you have.

At this point, I can not investigate further by trial and error. It takes time to understand your full setup, complete the missing CMakeLists, modify include path, fix undefined constants, understand that the id in the curl command line is a study id - not an instance id, check with 2 configurations, run tests and run valgrind. This is getting out of the scope of the community support.

Please provide a full Docker or docker-compose setup in which you build your plugin and run it, a sample dicom file and exact instructions to reproduce.

Best regards,

Alain.

7

Hi Alain,

One way to reproduce my issue is using the DicomWebPlugin, then send studies to another Peer Orthanc. The steps are

1/ Download source code from https://hg.orthanc-server.com/orthanc-dicomweb/
2/ In file DicomWebClient.cpp, replace the following code (line from 605 to 612) (attached file the one I did replace)

if (OrthancPlugins::RestApiGetString(dicom, “/instances/” + instances_[i] + “/file”, false))
{
networkSize_ += dicom.size();
context.SetContent(“NetworkSizeMB”, boost::lexical_caststd::string
(networkSize_ / static_cast<uint64_t>(1024 * 1024)));

return true;
}

By

Json::Value body = Json::objectValue;
body[“Keep”].append(“StudyInstanceUID”);
body[“Keep”].append(“SeriesInstanceUID”);
body[“Force”] = true;
OrthancPlugins::MemoryBuffer answer;
if (answer.RestApiPost(“/instances/” + instances_[i] + “/anonymize”, body, false))
{
answer.ToString(dicom);
networkSize_ += dicom.size();
context.SetContent(“NetworkSizeMB”, boost::lexical_caststd::string
(networkSize_ / static_cast<uint64_t>(1024 * 1024)));
return true;
}

3/ Compile code and put the libOrthancDicomWeb.so to orthanc’s plugin folder.
4/ Run Orthanc and send one study to another Peer Orthanc via Web UI

3.png

This is the RES memory when sending 2 studies

4.png

Thanks you in advance

DicomWebClient.cpp (42.6 KB)

8

Hi Alain,
The attachment is my compiled DicomWebPlugin (my above changes). You can mount it to orthanc docker or run orthanc with command line. And compare with the original version of DicomWebPlugin (https://lsb.orthanc-server.com/plugin-dicom-web/) to see the increasing RAM.

Thanks,
Chris

libOrthancDicomWeb.zip (3.17 MB)

9

Hi Christopher,

Do you agree that there’s no memory leak with this test plugin ? I don’t see any.

Concerning memory consumption, Indeed, there’s an increase in memory consumption when you call the RestApiPost(‘/anonymize’) compared to the code with RestApiGet() but, that’s just how glibc handles allocations/deallocations. And, there are probably more allocations when calling /anonymize compared to a call to /file.

Since there are no leaks, there’s hardly anything we can improve there.

Note that there are still a few env var that you can tune to tune the way glibc releases memory: https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunables.html. You can certainly play with some of these settings

Setting MallocArenaMax to 1 is the best way to reduce memory usage but might reduce performance.

There is also a bunch of useful info in this SO response: https://stackoverflow.com/questions/39753265/malloc-is-using-10x-the-amount-of-memory-necessary. Basically, it says that free() does not always release memory to the OS especially if the OS memory is not under stress.

You may also add malloc_stats() call to show the memory usage and you’ll see that the “used bytes” are quite stable while the “system bytes” may vary.

HTH,

Alain.

4.png

3.png

10

Hi Alain,
Thank you very much for your dedicated time to answer all of my questions. That would be very helpful. I will try to find another approach to solve the anonymized issue. Maybe write code to anynomize in Plugin instead of calling RestApi. I would look forward to a new SDK that provides anonymize feature from your team in the future.

Thanks again,
Chris