crosstyan/cvmmap-streamer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: avoid zstd segfault in vendored mcap writer

Browse Source 
Replace the vendored MCAP Zstd chunk writer's ZSTD_compress2-based path with ZSTD_compressCCtx and keep the selected compression level on the wrapper state.

The previous implementation crashed in Debug and real zed_svo_to_mcap runs on this machine when the first compressed chunk was flushed with --mcap-compression zstd. The same exports succeeded with none and lz4, which narrowed the failure to the shared vendored MCAP Zstd path rather than exporter sync logic.

Also extend mcap_multi_record_tester to accept a compression argument so the Zstd path can be exercised directly during regression testing.

Verified with:
- ./build-debug/bin/mcap_multi_record_tester /tmp/mcap_multi_zstd_test.mcap zstd
- ./build-debug/bin/zed_svo_to_mcap --input jump/experiment/2/2026-03-18T11-27-15/2026-03-18T11-27-15_zed1.svo2 --output /tmp/zed1_single_zstd_fixed.mcap --mcap-compression zstd --end-frame 1
- ./build-debug/bin/zed_svo_to_mcap --segment-dir jump/experiment/2/2026-03-18T11-27-15 --output /tmp/multi_zstd_fixed.mcap --mcap-compression zstd --end-frame 1
This commit is contained in:
crosstyan
2026-03-20 10:14:54 +00:00
parent 1ef318bde4
commit 039379f5fe
3 changed files with 37 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
third_party/mcap/include/mcap/writer.hpp
Vendored
+1
View File
@@ -312,6 +312,7 @@ public:
private:
std::vector<std::byte> uncompressedBuffer_;
std::vector<std::byte> compressedBuffer_;
CompressionLevel compressionLevel_ = CompressionLevel::Default;
ZSTD_CCtx_s* zstdContext_ = nullptr;
};
#endif
third_party/mcap/include/mcap/writer.inl
Vendored
+11 -7
View File
@@ -241,10 +241,13 @@ int ZStdCompressionLevel(CompressionLevel level) {
// ZStdWriter //////////////////////////////////////////////////////////////////
ZStdWriter::ZStdWriter(CompressionLevel compressionLevel, uint64_t chunkSize) {
ZStdWriter::ZStdWriter(CompressionLevel compressionLevel, uint64_t chunkSize)
: compressionLevel_(compressionLevel) {
zstdContext_ = ZSTD_createCCtx();
ZSTD_CCtx_setParameter(zstdContext_, ZSTD_c_compressionLevel,
internal::ZStdCompressionLevel(compressionLevel));
if (zstdContext_ == nullptr) {
std::cerr << "ZSTD_createCCtx failed\n";
std::abort();
}
uncompressedBuffer_.reserve(chunkSize);
}
@@ -259,15 +262,16 @@ void ZStdWriter::handleWrite(const std::byte* data, uint64_t size) {
void ZStdWriter::end() {
const auto dstCapacity = ZSTD_compressBound(uncompressedBuffer_.size());
compressedBuffer_.resize(dstCapacity);
const size_t dstSize = ZSTD_compress2(zstdContext_, compressedBuffer_.data(), dstCapacity,
uncompressedBuffer_.data(), uncompressedBuffer_.size());
const size_t dstSize =
ZSTD_compressCCtx(zstdContext_, compressedBuffer_.data(), dstCapacity,
uncompressedBuffer_.data(), uncompressedBuffer_.size(),
internal::ZStdCompressionLevel(compressionLevel_));
if (ZSTD_isError(dstSize)) {
const auto errCode = ZSTD_getErrorCode(dstSize);
std::cerr << "ZSTD_compress2 failed: " << ZSTD_getErrorName(dstSize) << " ("
std::cerr << "ZSTD_compressCCtx failed: " << ZSTD_getErrorName(dstSize) << " ("
<< ZSTD_getErrorString(errCode) << ")\n";
std::abort();
}
ZSTD_CCtx_reset(zstdContext_, ZSTD_reset_session_only);
compressedBuffer_.resize(dstSize);
}