ZFP+ADIOS2 Leaks memory
Created by: NAThompson
Describe the bug
It appears that all memory sent into ADIOS and passed through ZFP is leaked.
To Reproduce
- Clone the latest version of ZFP (08adb27dbc7ed91373ba76312aa7e48504a02190), build with
-fsanitize=address -fsanitize=undefined
to demonstrate that the leaks are not in ZFP. - Build ADIOS2 on top of the latest commit (currently 1e9c1248) with the options:
ADIOS2/build$ cmake -DCMAKE_CXX_COMPILER="g++-9" -DCMAKE_C_COMPILER="gcc-9" -DCMAKE_CXX_FLAGS="-fsanitize=address -fsanitize=undefined" -DADIOS2_USE_MPI=OFF -DCMAKE_BUILD_TYPE=Debug ../
ADIOS2/build$ make -j`nproc`
ADIOS2/build$ ctest -V
278: Test command: /home/4nt/ADIOS2/build/bin/Test.Engine.BP.WriteReadZfpConfig "--gtest_filter=*/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/*" "BP4"
278: Test timeout computed to be: 1500
278: Note: Google Test filter = */BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/*
278: [==========] Running 9 tests from 1 test case.
278: [----------] Global test environment set-up.
278: [----------] 9 tests from ZfpConfigFile/BPWriteReadZfpConfig
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/0
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/0 (11 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/1
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/1 (3 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/2
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/2 (4 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/3
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/3 (3 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/4
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/4 (3 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/5
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/5 (4 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/6
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/6 (6 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/7
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/7 (5 ms)
278: [ RUN ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/8
278: [ OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/8 (5 ms)
278: [----------] 9 tests from ZfpConfigFile/BPWriteReadZfpConfig (44 ms total)
278:
278: [----------] Global test environment tear-down
278: [==========] 9 tests from 1 test case ran. (44 ms total)
278: [ PASSED ] 9 tests.
278:
278: =================================================================
278: ==21184==ERROR: LeakSanitizer: detected memory leaks
278:
278: Direct leak of 360 byte(s) in 9 object(s) allocated from:
278: #0 0x7fa2d0960f78 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10bf78)
278: #1 0x7fa2c0e10be9 in stream_open /home/4nt/zfp/src/inline/bitstream.c:428
278: #2 0x7fa2c9a8c1d8 in adios2::core::compress::CompressZFP::Compress(void const*, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&) const /home/4nt/ADIOS2/source/adios2/operator/compress/CompressZFP.cpp:49
278: #3 0x7fa2c97d1331 in void adios2::format::BPOperation::SetDataDefault<double>(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&, adios2::core::Variable<double>::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/BPOperation.tcc:34
278: #4 0x7fa2c97e048c in adios2::format::BPZFP::SetData(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&, adios2::core::VariableBase::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/compress/BPZFP.cpp:53
278: #5 0x7fa2c8d46384 in void adios2::format::BPSerializer::PutOperationPayloadInBuffer<double>(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/BPSerializer.tcc:375
278: #6 0x7fa2c93d464d in void adios2::format::BP4Serializer::PutVariablePayload<double>(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&, bool, adios2::core::Variable<double>::Span*) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bp4/BP4Serializer.tcc:120
278: #7 0x7fa2c8b61cfb in void adios2::core::engine::BP4Writer::PutSyncCommon<double>(adios2::core::Variable<double>&, adios2::core::Variable<double>::Info const&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:99
278: #8 0x7fa2c8b1d0fc in void adios2::core::engine::BP4Writer::PerformPutCommon<double>(adios2::core::Variable<double>&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:137
278: #9 0x7fa2c8af4099 in adios2::core::engine::BP4Writer::PerformPuts() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:83
278: #10 0x7fa2c8af578b in adios2::core::engine::BP4Writer::EndStep() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:94
278: #11 0x7fa2c9c4be82 in adios2::Engine::EndStep() /home/4nt/ADIOS2/bindings/CXX11/adios2/cxx11/Engine.cpp:122
278: #12 0x56554f102202 in ZfpRate2DSmallSel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:795
278: #13 0x56554f1090a6 in BPWriteReadZfpConfig_ADIOS2BPWriteReadZfp2DSmallSel_Test::TestBody() /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:883
278: #14 0x56554f21b1ed in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278: #15 0x56554f1fee52 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278: #16 0x56554f1672ad in testing::Test::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2485
278: #17 0x56554f16a857 in testing::TestInfo::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2661
278: #18 0x56554f16d880 in testing::TestCase::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2779
278: #19 0x56554f1a1db9 in testing::internal::UnitTestImpl::RunAllTests() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:5036
278: #20 0x56554f221395 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278: #21 0x56554f204008 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278: #22 0x56554f196193 in testing::UnitTest::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:4652
278: #23 0x56554f10cc81 in RUN_ALL_TESTS() (/home/4nt/ADIOS2/build/bin/Test.Engine.BP.WriteReadZfpConfig+0x23fc81)
278: #24 0x56554f10951b in main /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:902
278: #25 0x7fa2c1a29b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
278:
278: Direct leak of 360 byte(s) in 9 object(s) allocated from:
278: #0 0x7fa2d0960f78 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10bf78)
278: #1 0x7fa2c0e10be9 in stream_open /home/4nt/zfp/src/inline/bitstream.c:428
278: #2 0x7fa2c9a8c1d8 in adios2::core::compress::CompressZFP::Compress(void const*, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&) const /home/4nt/ADIOS2/source/adios2/operator/compress/CompressZFP.cpp:49
278: #3 0x7fa2c97cefa5 in void adios2::format::BPOperation::SetDataDefault<float>(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&, adios2::core::Variable<float>::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/BPOperation.tcc:34
278: #4 0x7fa2c97e0004 in adios2::format::BPZFP::SetData(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&, adios2::core::VariableBase::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/compress/BPZFP.cpp:53
278: #5 0x7fa2c8d439a6 in void adios2::format::BPSerializer::PutOperationPayloadInBuffer<float>(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/BPSerializer.tcc:375
278: #6 0x7fa2c93ce9b9 in void adios2::format::BP4Serializer::PutVariablePayload<float>(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&, bool, adios2::core::Variable<float>::Span*) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bp4/BP4Serializer.tcc:120
278: #7 0x7fa2c8b5e4c7 in void adios2::core::engine::BP4Writer::PutSyncCommon<float>(adios2::core::Variable<float>&, adios2::core::Variable<float>::Info const&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:99
278: #8 0x7fa2c8b1c68c in void adios2::core::engine::BP4Writer::PerformPutCommon<float>(adios2::core::Variable<float>&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:137
278: #9 0x7fa2c8af3c9e in adios2::core::engine::BP4Writer::PerformPuts() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:83
278: #10 0x7fa2c8af578b in adios2::core::engine::BP4Writer::EndStep() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:94
278: #11 0x7fa2c9c4be82 in adios2::Engine::EndStep() /home/4nt/ADIOS2/bindings/CXX11/adios2/cxx11/Engine.cpp:122
278: #12 0x56554f102202 in ZfpRate2DSmallSel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:795
278: #13 0x56554f1090a6 in BPWriteReadZfpConfig_ADIOS2BPWriteReadZfp2DSmallSel_Test::TestBody() /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:883
278: #14 0x56554f21b1ed in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278: #15 0x56554f1fee52 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278: #16 0x56554f1672ad in testing::Test::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2485
278: #17 0x56554f16a857 in testing::TestInfo::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2661
278: #18 0x56554f16d880 in testing::TestCase::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2779
278: #19 0x56554f1a1db9 in testing::internal::UnitTestImpl::RunAllTests() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:5036
278: #20 0x56554f221395 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278: #21 0x56554f204008 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278: #22 0x56554f196193 in testing::UnitTest::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:4652
278: #23 0x56554f10cc81 in RUN_ALL_TESTS() (/home/4nt/ADIOS2/build/bin/Test.Engine.BP.WriteReadZfpConfig+0x23fc81)
278: #24 0x56554f10951b in main /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:902
278: #25 0x7fa2c1a29b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
278:
278: SUMMARY: AddressSanitizer: 720 byte(s) leaked in 18 allocation(s).
42/83 Test #278: Engine.BP.*/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/*.BP4 .........***Failed 0.28 sec
- OS/Platform: Ubuntu 18.04
- Build: gcc 9