[VL] Some filter conditions in MiscOperatorSuite failed

[marin-ma]

Backend

VL (Velox)

Bug description

Reverted PR facebookincubator/velox#15565 in #11266
in_or in_not and not in in MiscOperatorSuite failed with this pr. The failure is under investigation.

Gluten version

main branch

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

Error:

libc++abi: terminating due to uncaught exception of type facebook::velox::VeloxUserError: Exception: VeloxUserError
Error Source: USER
Error Code: UNSUPPORTED
Reason: Filter(MultiRange, deterministic, no nulls): testInt64Range() is not supported.
Retriable: False
Context: Split [Hive: file:///Users/rong/workspace/github/apache/gluten/backends-velox/target/scala-2.13/test-classes/tpch-data-parquet/lineitem/part-00000-6c374e0a-7d76-401b-8458-a8e31f8ab704-c000.snappy.parquet 0 - 1863237] Task Gluten_Stage_0_TID_0_VTID_0
Function: testInt64Range
File: /Users/rong/workspace/github/apache/gluten/ep/build-velox/build/velox_ep/./velox/type/Filter.h
Line: 241


call stack from microbenchmark:


(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
  * frame #0: 0x0000000128d47eb8 libvelox.dylib`facebook::velox::common::Filter::testInt64Range(this=0x0000600003a06cd0, (null)=1, (null)=2000, (null)=false) const at Filter.h:241:5
    frame #1: 0x00000001298eadf8 libvelox.dylib`facebook::velox::common::(anonymous namespace)::testIntFilter(filter=0x0000600003a06cd0, intStats=0x00006000006453b0, mayHaveNull=false) at ScanSpec.cpp:231:20
    frame #2: 0x00000001298eaad4 libvelox.dylib`facebook::velox::common::testFilter(filter=0x0000600003a06cd0, stats=0x00006000006453b0, totalRows=60175, type=std::__1::shared_ptr<const facebook::velox::Type>::element_type @ 0x0000000134021a90 strong=1 weak=1) at ScanSpec.cpp:382:14
    frame #3: 0x000000012a780170 libvelox.dylib`facebook::velox::parquet::ParquetData::rowGroupMatches(this=0x0000600001a46d80, rowGroupId=0, filter=0x0000600003a06cd0) at ParquetData.cpp:89:12
    frame #4: 0x000000012a77fe74 libvelox.dylib`facebook::velox::parquet::ParquetData::filterRowGroups(this=0x0000600001a46d80, scanSpec=0x000000011df32620, (null)=0, writerContext=0x000000011df36e48, result=0x000000016fdf9f90) at ParquetData.cpp:56:33
    frame #5: 0x0000000129907bf0 libvelox.dylib`facebook::velox::dwio::common::SelectiveColumnReader::filterRowGroups(this=0x000000011df375e0, rowGroupSize=0, context=0x000000011df36e48, result=0x000000016fdf9f90) const at SelectiveColumnReader.cpp:66:16
    frame #6: 0x000000012a78a070 libvelox.dylib`facebook::velox::parquet::StructColumnReader::filterRowGroups(this=0x000000011df36ec0, rowGroupSize=0, context=0x000000011df36e48, result=0x000000016fdf9f90) const at StructColumnReader.cpp:254:12
    frame #7: 0x000000012a388c70 libvelox.dylib`facebook::velox::parquet::ParquetRowReader::Impl::filterRowGroups(this=0x000000011df36bc0) at ParquetReader.cpp:1251:20
    frame #8: 0x000000012a3b63e0 libvelox.dylib`facebook::velox::parquet::ParquetRowReader::Impl::Impl(this=0x000000011df36bc0, readerBase=std::__1::shared_ptr<facebook::velox::parquet::ReaderBase>::element_type @ 0x000000011df32fb8 strong=2 weak=1, options=0x000000011df32bf0) at ParquetReader.cpp:1237:5
    frame #9: 0x000000012a3b6000 libvelox.dylib`facebook::velox::parquet::ParquetRowReader::Impl::Impl(this=0x000000011df36bc0, readerBase=std::__1::shared_ptr<facebook::velox::parquet::ReaderBase>::element_type @ 0x000000011df32fb8 strong=2 weak=1, options=0x000000011df32bf0) at ParquetReader.cpp:1200:31
    frame #10: 0x000000012a388a98 libvelox.dylib`std::__1::__unique_if<facebook::velox::parquet::ParquetRowReader::Impl>::__unique_single std::__1::make_unique[abi:ne180100]<facebook::velox::parquet::ParquetRowReader::Impl, std::__1::shared_ptr<facebook::velox::parquet::ReaderBase> const&, facebook::velox::dwio::common::RowReaderOptions const&>(__args=std::__1::shared_ptr<facebook::velox::parquet::ReaderBase>::element_type @ 0x000000011df32fb8 strong=2 weak=1, __args=0x000000011df32bf0) at unique_ptr.h:597:30
    frame #11: 0x000000012a3889dc libvelox.dylib`facebook::velox::parquet::ParquetRowReader::ParquetRowReader(this=0x0000600003649010, readerBase=std::__1::shared_ptr<facebook::velox::parquet::ReaderBase>::element_type @ 0x000000011df32fb8 strong=2 weak=1, options=0x000000011df32bf0) at ParquetReader.cpp:1412:11
    frame #12: 0x000000012a388b80 libvelox.dylib`facebook::velox::parquet::ParquetRowReader::ParquetRowReader(this=0x0000600003649010, readerBase=std::__1::shared_ptr<facebook::velox::parquet::ReaderBase>::element_type @ 0x000000011df32fb8 strong=2 weak=1, options=0x000000011df32bf0) at ParquetReader.cpp:1411:52
    frame #13: 0x000000012a389af0 libvelox.dylib`std::__1::__unique_if<facebook::velox::parquet::ParquetRowReader>::__unique_single std::__1::make_unique[abi:ne180100]<facebook::velox::parquet::ParquetRowReader, std::__1::shared_ptr<facebook::velox::parquet::ReaderBase> const&, facebook::velox::dwio::common::RowReaderOptions const&>(__args=std::__1::shared_ptr<facebook::velox::parquet::ReaderBase>::element_type @ 0x000000011df32fb8 strong=2 weak=1, __args=0x000000011df32bf0) at unique_ptr.h:597:30
    frame #14: 0x000000012a389a9c libvelox.dylib`facebook::velox::parquet::ParquetReader::createRowReader(this=0x0000600003459b80, options=0x000000011df32bf0) const at ParquetReader.cpp:1471:10
    frame #15: 0x0000000131a6c6d8 libvelox.dylib`facebook::velox::connector::hive::SplitReader::createRowReader(this=0x000000011df32a30, metadataFilter=nullptr, rowType=std::__1::shared_ptr<const facebook::velox::RowType>::element_type @ 0x0000600001a46c18 strong=5 weak=1, rowSizeTrackingEnabled= Has Value=false ) at SplitReader.cpp:408:33
    frame #16: 0x0000000131a6bca4 libvelox.dylib`facebook::velox::connector::hive::SplitReader::prepareSplit(this=0x000000011df32a30, metadataFilter=nullptr, runtimeStats=0x000000011e8d4ff0, fileReadOps=0x000000016fdfa588) at SplitReader.cpp:188:3
    frame #17: 0x0000000131a48ddc libvelox.dylib`facebook::velox::connector::hive::HiveDataSource::addSplit(this=0x000000011e8d4e00, split=std::__1::shared_ptr<facebook::velox::connector::ConnectorSplit>::element_type @ 0x000000011df32168 strong=5 weak=1) at HiveDataSource.cpp:363:17
    frame #18: 0x0000000131f356f4 libvelox.dylib`facebook::velox::exec::TableScan::getSplit(this=0x000000011e82d400) at TableScan.cpp:390:20
    frame #19: 0x0000000131f33b30 libvelox.dylib`facebook::velox::exec::TableScan::getOutput(this=0x000000011e82d400) at TableScan.cpp:175:32
    frame #20: 0x0000000131c28a88 libvelox.dylib`facebook::velox::exec::(anonymous namespace)::getOutput(op=0x000000011e82d400, result=nullptr) at Driver.cpp:450:16
    frame #21: 0x0000000131c286d8 libvelox.dylib`facebook::velox::exec::Driver::runInternal(std::__1::shared_ptr<facebook::velox::exec::Driver>&, std::__1::shared_ptr<facebook::velox::exec::BlockingState>&, std::__1::shared_ptr<facebook::velox::RowVector>&)::$_3::operator()(this=0x000000016fdfb588) const at Driver.cpp:578:15
    frame #22: 0x0000000131c11520 libvelox.dylib`void facebook::velox::exec::Driver::withDeltaCpuWallTimer<facebook::velox::exec::Driver::runInternal(std::__1::shared_ptr<facebook::velox::exec::Driver>&, std::__1::shared_ptr<facebook::velox::exec::BlockingState>&, std::__1::shared_ptr<facebook::velox::RowVector>&)::$_3>(this=0x0000600000d44ee0, op=0x000000011e82d400, opTimingMember=0x00000000000000e8, opFunction=0x000000016fdfb588) at Driver.cpp:1197:3
    frame #23: 0x0000000131c0f778 libvelox.dylib`facebook::velox::exec::Driver::runInternal(this=0x0000600000d44ee0, self=std::__1::shared_ptr<facebook::velox::exec::Driver>::element_type @ 0x0000600000d44ee0 strong=4 weak=4, blockingState=nullptr, result=nullptr) at Driver.cpp:575:13
    frame #24: 0x0000000131c0e9d8 libvelox.dylib`facebook::velox::exec::Driver::next(this=0x0000600000d44ee0, future=0x000000016fdfbad0, blockingOp=0x000000016fdfbac8, blockingReason=0x000000016fdfbac4) at Driver.cpp:284:21
    frame #25: 0x0000000131f5575c libvelox.dylib`facebook::velox::exec::Task::next(this=0x000000011e939a00, future=0x000000016fdfbdd8) at Task.cpp:884:29
    frame #26: 0x00000001283a0eac libvelox.dylib`gluten::WholeStageResultIterator::next(this=0x000000011df11200) at WholeStageResultIterator.cc:237:23
    frame #27: 0x0000000100036da8 generic_benchmark`gluten::ResultIterator::getNext(this=0x0000600002162b18) at ResultIterator.h:83:22
    frame #28: 0x00000001000267b4 generic_benchmark`gluten::ResultIterator::hasNext(this=0x0000600002162b18) at ResultIterator.h:45:5
    frame #29: 0x0000000100024a04 generic_benchmark`$_1::operator()(this=0x000000011f08deb8, state=0x000000016fdfc988, planFile="/tmp/micro/plan_9_0_0.json", splitFiles=size=1, dataFiles=size=0, localDirs=size=1, runtimeFactory= Lambda in File GenericBenchmark.cc at Line 709, readerType=kNone) const at GenericBenchmark.cc:462:28
    frame #30: 0x0000000100023fe0 generic_benchmark`benchmark::internal::Benchmark* benchmark::RegisterBenchmark<$_1&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::function<gluten::VeloxRuntime* (gluten::MemoryManager*)>&, gluten::FileReaderType>(this=0x000000011f08deb8, st=0x000000016fdfc988)>&, gluten::FileReaderType&&)::'lambda'(benchmark::State&)::operator()(benchmark::State&) const at benchmark.h:1115:41
    frame #31: 0x0000000100023b38 generic_benchmark`benchmark::internal::LambdaBenchmark<benchmark::internal::Benchmark* benchmark::RegisterBenchmark<$_1&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::function<gluten::VeloxRuntime* (gluten::MemoryManager*)>&, gluten::FileReaderType>(char const*, $_1&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::function<gluten::VeloxRuntime* (gluten::MemoryManager*)>&, gluten::FileReaderType&&)::'lambda'(benchmark::State&)>::Run(this=0x000000011f08de00, st=0x000000016fdfc988) at benchmark.h:1074:52
    frame #32: 0x0000000100064d0c generic_benchmark`benchmark::internal::BenchmarkInstance::Run(unsigned long long, int, benchmark::internal::ThreadTimer*, benchmark::internal::ThreadManager*, benchmark::internal::PerfCountersMeasurement*) const + 72
    frame #33: 0x0000000100077dd0 generic_benchmark`benchmark::internal::(anonymous namespace)::RunInThread(benchmark::internal::BenchmarkInstance const*, unsigned long long, int, benchmark::internal::ThreadManager*, benchmark::internal::PerfCountersMeasurement*) + 76
    frame #34: 0x00000001000779fc generic_benchmark`benchmark::internal::BenchmarkRunner::DoNIterations() + 680
    frame #35: 0x0000000100078498 generic_benchmark`benchmark::internal::BenchmarkRunner::DoOneRepetition() + 140
    frame #36: 0x000000010005f1a4 generic_benchmark`benchmark::RunSpecifiedBenchmarks(benchmark::BenchmarkReporter*, benchmark::BenchmarkReporter*) + 2704
    frame #37: 0x000000010000b9fc generic_benchmark`main(argc=1, argv=0x000000016fdff1d0) at GenericBenchmark.cc:758:3
    frame #38: 0x000000018f1beb98 dyld`start + 6076