rocky系统编译ceph
不同系统编译ceph 遇到的问题大多情况是 缺包；缺包的话有两种方式
做不同系统编译ceph 遇到的问题大多情况是 缺包；缺包的话有两种方式
- 直接从yum 源里面安装
- 从官方 https://rockylinux.pkgs.org/ 下载 rpm包，手动安装
这两个方法可以解决大多数的缺包问题，当然也有些包没有适配操作系统的可能需要自己拉源码编译，这种比较繁琐，这个方法放在最后，实在不行才自己编；
还有一个问题则是编译时遇到的问题，由于gcc 版本不同（跨度很大），有些语法不是完全兼容的，这个时候需要分析 报错的内容，（优先谷歌下，绝大多数问题，已有前人踩坑）
环境配置
由于没有物理机，直接在用容器编
[root@9ebdf34fb3b7 ceph]# cat /etc/redhat-release Rocky Linux release 8.6 (Green Obsidian) [root@9ebdf34fb3b7 ceph]#
[root@8dd40184b527 /]# cat /etc/rocky-release Rocky Linux release 9.3 (Blue Onyx)
[root@9ebdf34fb3b7 ceph]# ldd --version ldd (GNU libc) 2.28 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.
[root@8dd40184b527 build]# uname -a Linux 8dd40184b527 3.10.0-1160.21.1.el7.x86_64 #1 SMP Tue Mar 16 18:28:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
[root@8dd40184b527 build]#
安装必要的依赖
执行 install-deps.sh 脚本 ，由于是rocky，脚本里面没有涉及改系统的选项， 手动添加下，（rokcy也是 centos的分支）
[root@8dd40184b527 ceph_copy]# ./install-deps.sh rocky is unknown, dependencies will have to be installed manually.
[root@8dd40184b527 ceph_copy]#
[root@8dd40184b527 ceph_copy]# sed -i 's/virtuozzo/rocky/g' ./install-deps.sh
[root@8dd40184b527 ceph_copy]# ./install-deps.sh
Using dnf to install dependencies Last metadata expiration check: 0:10:34 ago on Sun Feb 18 07:28:20 2024. Dependencies resolved. =====================
...
上面的脚本安装的可能不全，下面是遇到编译问题，直接从yum源安装的
[root@8dd40184b527 ceph_copy]# echo "cmake gcc python3-devel.x86_64 make wget ncurses-devel.x86_64 python snappy.x86_64 libaio-devel.x86_64 fuse3-devel.x86_64 openldap-devel.x86_64 python3-sphinx.noarch libblkid-devel.x86_64 systemd-devel keyutils-libs-devel.x86_64 lz4-devel.x86_64 libcurl-devel.x86_64 nss-devel.x86_64 openssl-devel.x86_64 expat-devel.x86_64 liboath-devel.x86_64 python lttng-ust-devel.x86_64 libbabeltrace-devel.x86_64 libcap-ng-devel.x86_64 librabbitmq-devel.x86_64 gperf libnl3-devel.x86_64 librdkafka-devel.x86_64 libcap-ng-devel.x86_64 keyutils-libs-devel.x86_64" > /tmp/install
[root@8dd40184b527 ceph_copy]# for i in $(cat /tmp/install);do yum install -y $i;done
执行 do_cmake 执行 do\_cmake
do\_cmake 会先检测一些必要的依赖是否已安装
CMake Error at CMakeLists.txt:63 (message): Can't find sphinx-build.
-- Configuring incomplete, errors occurred!
yum install pip && pip install Sphinx -i https://pypi.tuna.tsinghua.edu.cn/simple
# 这个库可以不需要的，直接在 do_cmkae 注释掉也行
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find verbs (missing: VERBS_LIBRARIES VERBS_INCLUDE_DIR) Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/Findverbs.cmake:13 (find_package_handle_standard_args) CMakeLists.txt:204 (find_package)
# RDMA 需要的用的库，目前我们产品没有用用到，不用加入编译，直接注释 后者 do_cmake 加入 -DWITH_RDMA=OFF
-- Found Backtrace: /usr/include CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find udev (missing: UDEV_LIBRARIES UDEV_INCLUDE_DIR) Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/Findudev.cmake:30 (find_package_handle_standard_args) CMakeLists.txt:217 (find_package)
yum install systemd-devel
-- Found udev: /usr/lib64/libudev.so CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find blkid (missing: BLKID_LIBRARIES BLKID_INCLUDE_DIR) Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/Findblkid.cmake:31 (find_package_handle_standard_args) CMakeLists.txt:219 (find_package)
yum install libblkid-devel.x86_64
-- Found udev: /usr/lib64/libudev.so CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find blkid (missing: BLKID_LIBRARIES BLKID_INCLUDE_DIR) Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/Findblkid.cmake:31 (find_package_handle_standard_args) CMakeLists.txt:219 (find_package)
yum install libblkid-devel.x86_64
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find OpenLdap (missing: OPENLDAP_INCLUDE_DIR LDAP_LIBRARY LBER_LIBRARY) Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/FindOpenLdap.cmake:16 (find_package_handle_standard_args) CMakeLists.txt:244 (find_package)
yum install -y openldap-devel.x86_64
缺少level 包 Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/Findleveldb.cmake:18
yum 源码里没有，手动下载并安装
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/l/leveldb-devel-1.23-4.el9.x86_64.rpm
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/l/leveldb-1.23-4.el9.x86_64.rpm
rpm -ivh leveldb-1.23-4.el9.x86_64.rpm
rpm -ivh leveldb-devel-1.23-4.el9.x86_64.rpm
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find snappy (missing: SNAPPY_LIBRARIES SNAPPY_INCLUDE_DIR) Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/Findsnappy.cmake:25
(find_package_handle_standard_args) CMakeLists.txt:349 (find_package)
yum 源码里没有，手动下载并安装
wget https://dl.rockylinux.org/pub/rocky/9/devel/x86_64/os/Packages/s/snappy-devel-1.1.8-8.el9.x86_64.rpm
rpm -ivh snappy-devel-1.1.8-8.el9.x86_64.rpm
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find OATH (missing: OATH_LIBRARY OATH_INCLUDE_DIR) Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/FindOATH.cmake:17
(find_package_handle_standard_args) CMakeLists.txt:461 (find_package) yum 源码里没有，手动下载并安装
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/l/liboath-2.6.7-2.el9.x86_64.rpm
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/l/liboath-devel-2.6.7-2.el9.x86_64.rpm
rpm -ivh liboath-2.6.7-2.el9.x86_64.rpm
rpm -ivh liboath-devel-2.6.7-2.el9.x86_64.rpm
-- crypto soname: libcrypto.so.3 -- Found PythonInterp: /usr/bin/python (found suitable version "3.9.18", minimum required is "2.7")
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
(Required is at least version "2.7")
yum install python3-devel.x86_64
-- Found PythonLibs: /usr/lib64/libpython3.9.so (found suitable version "3.9.18", minimum required is "2.7") CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR)
yum install zlib-devel.x86_64
Make Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find LTTngUST (missing: LTTNGUST_LIBRARIES LTTNGUST_INCLUDE_DIRS) Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/FindLTTngUST.cmake:107 (find_package_handle_standard_args) CMakeLists.txt:570 (find_package)
LTTN 用不到，编译选项关闭 -DWITH_LTTNG=OFF -DWITH_BABELTRACE=OFF
如果真的需要 安装 这两个包
lttng-ust-devel.x86_64 libbabeltrace-devel.x86_64
CMake Error at cmake/modules/FindCython.cmake:13 (message): Could not find cython: /usr/bin/python: No module named cython Call Stack (most recent call first): src/pybind/CMakeLists.txt:23 (find_package)
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find Cython (missing: CYTHON_VERSION)
Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/FindCython.cmake:16 (find_package_handle_standard_args) src/pybind/CMakeLists.txt:23 (find_package)
pip install cython -i https://pypi.tuna.tsinghua.edu.cn/simple
-- Found Cython: 0.29 CMake Error at src/pybind/CMakeLists.txt:68 (message): mgr plugins require python3 binding ## force ceph to build against python3:
#指定pyton版本 添加编译选项 -DWITH_PYTHON3=ON
-- Could NOT find nl-genl-3 (missing: GENL_LIBRARIES GENL_INCLUDE_DIR) CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find keyutils (missing: KEYUTILS_LIBRARIES KEYUTILS_INCLUDE_DIR) Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/Findkeyutils.cmake:16 (find_package_handle_standard_args) CMakeLists.txt:225 (find_package)
yum install keyutils-libs-devel.x86_64
Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) cmake/modules/FindRabbitMQ.cmake:9 (find_package_handle_standard_args) src/rgw/CMakeLists.txt:134 (find_package)
rpm 官方下载
wget https://dl.rockylinux.org/pub/rocky/9/devel/x86_64/os/Packages/l/librabbitmq-devel-0.11.0-7.el9.x86_64.rpm rpm -ivh librabbitmq-devel-0.11.0-7.el9.x86_64.rpm
/boost/thread/pthread/thread_data.hpp:60:5: error: missing binary operator before token "(" 60 | #if PTHREAD_STACK_MIN > 0 | ^~~~~~~~~~~~~~~~~ ...skipped <pbin.v2/libs/coroutine/build/gcc-10.3.1/release/link-static/threadapi-pthread/threading-multi/visibility-hidden>libboost_coroutine.a(clean) for lack of <pbin.v2/libs/coroutine/build/gcc-10.3.1/release/link-static/threadapi-pthread/threading-multi/visibility-hidden>posix/stack_traits.o... ...skipped <pbin.v2/libs/coroutine/build/gcc-10.3.1/release/link-static/threadapi-pthread/threading-multi/visibility-hidde z 修改 boost 源文件的 /root/ceph/build/boost/src/Boost/boost/thread/pthread/thread_data.hpp #ifdef PTHREAD_STACK_MIN /data/rphuang/ceph_copy/build/boost/src/Boost/boost/thread/pthread
高版本的gcc 不支持这个语法，
需要将 #if PTHREAD\_STACK\_MIN > 0 改为 #ifdef PTHREAD\_STACK\_MIN
缺少 librdkafka 这个再编译过程中才检测到的
wget https://dl.rockylinux.org/pub/rocky/9/devel/x86_64/os/Packages/l/librdkafka-1.6.1-102.el9.x86_64.rpm
wget https://dl.rockylinux.org/pub/rocky/9/devel/x86_64/os/Packages/l/librdkafka-devel-1.6.1-102.el9.x86_64.rpm
wget https://dl.rockylinux.org/pub/rocky/9/devel/x86_64/os/Packages/l/libzstd-devel-1.5.1-2.el9.x86_64.rpm
rpm -ivh libzstd-devel-1.5.1-2.el9.x86_64.rpm
rpm -ivh librdkafka-1.6.1-102.el9.x86_64.rpm
rpm -ivh librdkafka-devel-1.6.1-102.el9.x86_64.rpm
缺少 gperf
ir<std::__cxx11::basic_string<char>, bool>’} to make the copy explicit or ‘const std::pair<const std::__cxx11::basic_string<char>, bool>&’ to prevent copying /data/rphuang/ceph_rocky/ceph/src/rgw/rgw_rados.cc:15:10: fatal error: gperftools/heap-checker.h: No such file or directory 15 | #include "gperftools/heap-checker.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/g/gperftools-devel-2.9.1-2.el9.x86_64.rpm
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/l/libunwind-1.6.2-1.el9.x86_64.rpm
wget https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/g/gperftools-2.9.1-2.el9.x86_64.rpm
rpm -ivh libunwind-1.6.2-1.el9.x86_64.rpm
rpm -ivh libperf-devel-5.14.0-362.18.1.el9_3.0.1.x86_64.rpm
rpm -ivh gperftools-devel-2.9.1-2.el9.x86_64.rpm rpm -ivh gperftools-libs-2.9.1-2.el9.x86_64.rpm
`使用vstaart 脚本搭建集群 Populating config ... Traceback (most recent call last):
File "/data/rphuang/ceph_copy/build/bin/ceph", line 151, in <module>
from ceph_daemon import admin_socket, DaemonWatcher, Termsize File "/data/rphuang/ceph_copy/src/pybind/ceph_daemon.py", line 24,
in <module> from prettytable import PrettyTable, HEADER ModuleNotFoundError: No module named 'prettytable'
# 用vstart 启动集群的时候 提示没有 prettytable pip install PrettyTable ````