Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OSS build broken as of May, 2024 -> PRs are all blocked #310

Open
therealgymmy opened this issue May 3, 2024 · 8 comments
Open

OSS build broken as of May, 2024 -> PRs are all blocked #310

therealgymmy opened this issue May 3, 2024 · 8 comments
Assignees
@therealgymmy
Labels
build

Comments

@therealgymmy
Copy link
Contributor

https://github.com/facebook/CacheLib/actions/runs/8937959842/job/24551185009

Thrift dependency fails to build. We cannot import PRs when our OSS builds are broken. We're actively looking into this.

/__w/CacheLib/CacheLib/cachelib/external/fbthrift/thrift/lib/cpp2/server/Cpp2Connection.cpp
In file included from /__w/CacheLib/CacheLib/cachelib/external/fbthrift/./thrift/lib/cpp2/server/Cpp2Worker.h:38,
                 from /__w/CacheLib/CacheLib/cachelib/external/fbthrift/./thrift/lib/cpp2/server/Cpp2Connection.h:32,
                 from /__w/CacheLib/CacheLib/cachelib/external/fbthrift/thrift/lib/cpp2/server/Cpp2Connection.cpp:17:
/__w/CacheLib/CacheLib/cachelib/external/fbthrift/./thrift/lib/cpp2/server/ThriftServer.h:99:10: fatal error: wangle/ssl/TLSInMemoryTicketProcessor.h: No such file or directory
 #include <wangle/ssl/TLSInMemoryTicketProcessor.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@therealgymmy
Copy link
Contributor Author

Previous issue is now resolved however we ran into the following issue this past weekend. :/

usr/bin/cmake -E cmake_link_script CMakeFiles/FizzTool.dir/link.txt --verbose=YES
/usr/bin/c++ -O2 -g -DNDEBUG -rdynamic CMakeFiles/FizzTool.dir/tool/Main.cpp.o CMakeFiles/FizzTool.dir/tool/FizzClientCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzClientLoadGenCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzCommandCommon.cpp.o CMakeFiles/FizzTool.dir/tool/FizzGenerateDelegatedCredentialCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzServerBenchmarkCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzServerCommand.cpp.o -o bin/fizz  -Wl,-rpath,/__w/CacheLib/CacheLib/build-fizz/lib:/__w/CacheLib/CacheLib/opt/cachelib/lib:/__w/CacheLib/CacheLib/opt/cachelib/lib64: lib/libfizz.so.1.0.0 /usr/lib64/libsodium.so /__w/CacheLib/CacheLib/opt/cachelib/lib/libfolly.so.0.58.0-dev /__w/CacheLib/CacheLib/opt/cachelib/lib64/libfmt.so.10.2.1 -lboost_context -lboost_filesystem -lboost_program_options -lboost_regex -lboost_system -lboost_thread -lboost_chrono -lboost_date_time -lboost_atomic -ldouble-conversion /__w/CacheLib/CacheLib/opt/cachelib/lib64/libglog.so -levent -lz -lbz2 -llzma -llz4 -lsnappy -ldwarf -Wl,-Bstatic -liberty -Wl,-Bdynamic -laio -lsodium -ldl -lunwind -lssl -lcrypto /usr/lib64/libz.so /__w/CacheLib/CacheLib/opt/cachelib/lib64/libzstd.so /__w/CacheLib/CacheLib/opt/cachelib/lib/libgflags.so.2.2.2 
/usr/bin/ld: CMakeFiles/FizzTool.dir/tool/FizzGenerateDelegatedCredentialCommand.cpp.o: in function `fizz::tool::fizzGenerateDelegatedCredentialCommand(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)':
/__w/CacheLib/CacheLib/cachelib/external/fizz/fizz/tool/FizzGenerateDelegatedCredentialCommand.cpp:186: undefined reference to `fizz::extensions::generateDelegatedCredentialPEM(fizz::extensions::DelegatedCredential, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)'
collect2: error: ld returned 1 exit status
make[2]: Leaving directory '/__w/CacheLib/CacheLib/build-fizz'

@therealgymmy
Copy link
Contributor Author

mvfst broke as well. Also fixed internally. Pending the next submodule sync.

@therealgymmy therealgymmy mentioned this issue May 13, 2024
Open
@therealgymmy
Copy link
Contributor Author

therealgymmy commented May 13, 2024 •
edited
Loading

welp. folly is now broken. This is kinda ludicrous :-) one thing breaks after another.

https://github.com/facebook/CacheLib/actions/runs/9065222623/job/24905320734

/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1733:15: note: suggested alternative: 'BOOST_PP_IIF'
     iovec vec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
               ^~~~~~~~~~~
               BOOST_PP_IIF
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1736:30: error: 'vec' was not declared in this scope
     writeChainImpl(callback, vec, count, std::move(buf), flags);
                              ^~~
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp: In member function 'virtual folly::AsyncSocket::WriteResult folly::AsyncSocket::sendSocketMessage(const iovec*, size_t, folly::WriteFlags, folly::AsyncSocket::WriteRequestTag)':
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3774:18: error: 'BOOST_PP_IF' was not declared in this scope
     iovec tmpVec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
                  ^~~~~~~~~~~
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3774:18: note: suggested alternative: 'BOOST_PP_IIF'
     iovec tmpVec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
                  ^~~~~~~~~~~
                  BOOST_PP_IIF
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3783:9: error: 'tmpVec' was not declared in this scope
         tmpVec,
         ^~~~~~
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3783:9: note: suggested alternative: 'tmpnam'
         tmpVec,
         ^~~~~~
         tmpnam
[ 55%] Building CXX object CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o
/usr/bin/c++ -DFMT_SHARED -DFOLLY_XLOG_STRIP_PREFIXES=\"/__w/CacheLib/CacheLib/cachelib/external/folly:/__w/CacheLib/CacheLib/build-folly\" -DGFLAGS_IS_A_DLL=0 -D_GNU_SOURCE -D_REENTRANT -I/__w/CacheLib/CacheLib/cachelib/external/folly -I/__w/CacheLib/CacheLib/build-folly -I/__w/CacheLib/CacheLib/opt/cachelib/include -I/usr/include/libdwarf -O2 -g -DNDEBUG -std=gnu++17 -fPIC -g -finput-charset=UTF-8 -fsigned-char -Wall -Wno-deprecated -Wno-deprecated-declarations -Wno-sign-compare -Wno-unused -Wuninitialized -Wunused-label -Wunused-result -Wshadow-compatible-local -Wno-noexcept-type -faligned-new -fopenmp -MD -MT CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o -MF CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o.d -o CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o -c /__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncTimeout.cpp
make[2]: *** [CMakeFiles/folly_base.dir/build.make:2624: CMakeFiles/folly_base.dir/folly/io/async/AsyncSocket.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/__w/CacheLib/CacheLib/build-folly'
make[1]: *** [CMakeFiles/Makefile2:145: CMakeFiles/folly_base.dir/all] Error 2
make[1]: Leaving directory '/__w/CacheLib/CacheLib/build-folly'
make: *** [Makefile:136: all] Error 2
build-package.sh: error: make failed

@therealgymmy
Copy link
Contributor Author

therealgymmy commented May 20, 2024 •
edited
Loading

Our build is partially back. However, centos 8.1, 8.5, rocky 8.6 and ubuntu 20 are broken on folly.

In member function ‘virtual void folly::AsyncSocket::writeChain(folly::AsyncWriter::WriteCallback*, std::unique_ptr<folly::IOBuf>&&, folly::WriteFlags)’:
/home/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1733:15: error: ‘BOOST_PP_IF’ was not declared in this scope
     iovec vec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
               ^~~~~~~~~~~
/home/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1733:15: note: suggested alternative: ‘BOOST_PP_IIF’
     iovec vec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
               ^~~~~~~~~~~
               BOOST_PP_IIF
/home/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1736:30: error: ‘vec’ was not declared in this scope
     writeChainImpl(callback, vec, count, std::move(buf), flags);

@therealgymmy therealgymmy mentioned this issue May 20, 2024
Closed
@therealgymmy therealgymmy self-assigned this May 20, 2024
@therealgymmy
Copy link
Contributor Author

mvfst is broken. All builds are broken once again.

/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp: In member function 'virtual void quic::InplaceQuicPacketBuilder::appendPaddingFrame()':
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:676:29: note: '<anonymous>' declared here
  676 |     packet_.frames.push_back(PaddingFrame());
      |     ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
In file included from /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.h:13,
                 from /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:8:
In constructor 'quic::KnobFrame::KnobFrame(quic::KnobFrame&&)',
    inlined from 'quic::QuicSimpleFrame::QuicSimpleFrame(quic::QuicSimpleFrame&&)' at /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/Types.h:859:1,
    inlined from 'quic::QuicWriteFrame::QuicWriteFrame(quic::QuicWriteFrame&&)' at /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/Types.h:901:1,
    inlined from 'void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = quic::QuicWriteFrame; _Args = {quic::QuicWriteFrame}; _Tp = quic::QuicWriteFrame]' at /usr/include/c++/11/ext/new_allocator.h:162:4,
    inlined from 'static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = quic::QuicWriteFrame; _Args = {quic::QuicWriteFrame}; _Tp = quic::QuicWriteFrame]' at /usr/include/c++/11/bits/alloc_traits.h:516:17,
    inlined from 'std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::emplace_back(_Args&& ...) [with _Args = {quic::QuicWriteFrame}; _Tp = quic::QuicWriteFrame; _Alloc = std::allocator<quic::QuicWriteFrame>]' at /usr/include/c++/11/bits/vector.tcc:115:30,
    inlined from 'void std::vector<_Tp, _Alloc>::push_back(std::vector<_Tp, _Alloc>::value_type&&) [with _Tp = quic::QuicWriteFrame; _Alloc = std::allocator<quic::QuicWriteFrame>]' at /usr/include/c++/11/bits/stl_vector.h:1204:21,
    inlined from 'virtual void quic::InplaceQuicPacketBuilder::appendPaddingFrame()' at /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:676:29:
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/Types.h:109:19: warning: '*(quic::KnobFrame*)((char*)&<unnamed> + offsetof(quic::value_type, quic::QuicWriteFrame::<unnamed>)).quic::KnobFrame::len' may be used uninitialized [-Wmaybe-uninitialized]
  109 |         len(other.len),
      |             ~~~~~~^~~
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp: In member function 'virtual void quic::InplaceQuicPacketBuilder::appendPaddingFrame()':
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:676:29: note: '<anonymous>' declared here
  676 |     packet_.frames.push_back(PaddingFrame());
      |     ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~

byrnedj added a commit to byrnedj/CacheLib that referenced this issue May 28, 2024
@byrnedj
fixes fizz commit for now (should drop once facebook#310 is closed)
256c267
@byrnedj byrnedj mentioned this issue May 28, 2024
Open
byrnedj added a commit to byrnedj/CacheLib that referenced this issue May 29, 2024
@byrnedj
fixes commit for now (should drop once facebook#310 is closed)
52618b5
@therealgymmy
Copy link
Contributor Author

Now it is fbthrift once again :-) what a month. We'll do what we can to continue chase down the fixes one at a time. Long-term, we're discussing to get rid of our thrift dependency (and its own dependencies) as well.

@haowu14
Copy link
Contributor

haowu14 commented Jun 20, 2024

Dependencies are fixed.

On Ubuntu 22 there's a cachelib compilation failure. Attempting a fix.

In file included from /home/runner/work/CacheLib/CacheLib/cachelib/allocator/tests/ChainedHashTest.cpp:18:
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h: In member function ‘void facebook::cachelib::tests::AccessTypeTest<AccessType>::testIteratorMayContainNull()’:
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h:596:49: error: need ‘typename’ before ‘facebook::cachelib::tests::AccessTypeTest<AccessType>::Node::Handle’ because ‘facebook::cachelib::tests::AccessTypeTest<AccessType>::Node’ is a dependent scope
  596 |               [&keyToReturnNull](Node* node) -> Node::Handle {
      |                                                 ^~~~
      |                                                 typename 
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h: In instantiation of ‘void facebook::cachelib::tests::AccessTypeTest<AccessType>::testIteratorMayContainNull() [with AccessType = facebook::cachelib::ChainedHashTable]’:
/home/runner/work/CacheLib/CacheLib/cachelib/allocator/tests/ChainedHashTest.cpp:57:29:   required from here
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h:595:13: error: no matching function for call to ‘facebook::cach

@triplewy
Copy link

I'm interested in using cachelib but am a bit confused on its networking dependencies. My understanding was that cachelib is simply a storage engine without any networking capabilities. Is there any progress on removal of Thrift from its dependencies?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@therealgymmy therealgymmy

Labels
build
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@therealgymmy @triplewy @haowu14