How to write CMakeList using a library built in cmake

Asked 2 years ago, Updated 2 years ago, 379 views

I want to generate ORC files that I can read in Athena without using queries

org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat

in the format
https://github.com/apache/orc
I thought it would be good to use this

 git clone https://github.com/apache/orc.git
cmake-Sorc-Borc/build-DBUILD_JAVA=OFF
cdorc/build
make package
make test-out


after running ~/homebrew/lib
~/homebrew/include
I've got a lot of things in

https://orc.apache.org/docs/core-cpp.html#writing-orc-files
Next, I tried to compile this sample by writing only the following declaration parts.

main.cpp 

#include<orc/OrcFile.hh>

int main() {
  ORC_UNIQUE_PTR<orc::OutputStream>outStream=orc::writeLocalFile("my-file.orc");
  ORC_UNIQUE_PTR<orc::Type>schema(
    orc::Type::buildTypeFromString("struct<x:int,y:int>"));
  orc::WriterOptions options;
  ORC_UNIQUE_PTR<orc::Writer>writer=
  createWriter(*schema, outStream.get(), options);
  return 0;
}

CMakeLists.txt 

cmake_minimum_required (VERSION 2.8)
project(orc-test)
set(CMAKE_CXX_STANDARD11)

add_executable(orc-test main.cpp)

I wrote cmake. But when I made it,

 fatal error: 'orc/OrcFile.hh' file not found

It's called

This is my first time using cmake, so I thought it would automatically include and link, but isn't that what it is?

cmake_minimum_required (VERSION 2.8)
project(orc-test CXX)
set(CMAKE_CXX_STANDARD11)

include_directories(~/homebrew/include)
add_executable(orc-test main.cpp)

When I added this, it seemed that the main.cpp compilation went through, so I went to the linker.

[50%] Building CXX object CMakeFiles/orc-test.dir/main.cpp.o
[100%]Linking CXX executable orc-test
Undefined symbols for architecture x86_64:
  "orc::createWriter(orc::Type const&, orc::OutputStream*, orc::WriterOptions const&), referred from:
      _main in main.cpp.o
  "orc::WriterOptions::WriterOptions()" from:
      _main in main.cpp.o
  "orc::WriterOptions::~WriterOptions()" from:
      _main in main.cpp.o
  "orc::writeLocalFile(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator>char>>const&)", referenced from:
      _main in main.cpp.o
  "orc::type::buildTypeFromString(std::__1::basic_string<char,std::__1::char_trats<char>,std::__1::allocator<char>>const&), referred from:
      _main in main.cpp.o
ld —symbol(s) not found for architecture x86_64
clang:error:linker command failed with exit code 1 (use-v to see invocation)
make[2]:***[orc-test] Error 1
make[1]:*** [CMakeFiles/orc-test.dir/all] Error 2
make: ***[all] Error 2

I am not sure about this error

When installing orc
share/doc/orc/LICENSE
share/doc/orc/NOTICE
Only *.a, include/ have *.hh installed on and lib/
It doesn't look like a cmake configuration file is installed, but
How do I call CMakeLists.txt to use orc built with this cmake?

As I remember in the past, when you put aws-sdk-c++ in cmake,
~/homebrew/lib/cmake/
There is a file called .cmake in
I remember that the cmake file in the project was automatically included and linked without much effort...

Additional information

When I wrote the CMakeList as follows,

cmake_minimum_required (VERSION 2.8)
project(orc-test CXX)
set(CMAKE_CXX_STANDARD11)

include_directories(~/homebrew/include)
link_directories(~/homebrew/lib)
target_link_libraries (~/homebrew/lib/liborc.a)
add_executable(orc-test main.cpp)

A warning similar to the following appears:
After that, the result of make remains the same.

CMakeWarning(dev) at CMakeLists.txt:6(target_link_libraries):
  Cannot specify link libraries for target "~/homebrew/lib/liborc.a" which is
  not built by this project.

  CMake does not support this but used to work incidentally and is being
  allowed for compatibility.

  Policy CMP0016 is not set: target_link_libraries() reports error ifits
  only argument is not a target.Run "cmake --help-policy CMP0016" for
  policy details.Use the cmake_policy command to set the policy and
  suppress this warning.
This warning is for project developers. Use-Wno-dev to suppress it.

Additional information

cmake_minimum_required (VERSION 2.8)
project(orc-test CXX)
set(CMAKE_CXX_STANDARD11)

include_directories(~/homebrew/include)
link_directories(~/homebrew/lib)
add_executable(orc-test main.cpp)
target_link_libraries (orc-test liborc)

When I wrote CMake as above, the error was as follows.

 [50%] Linking CXX executable orc-test
ld —library not found for-liborc
clang:error:linker command failed with exit code 1 (use-v to see invocation)
make[2]:***[orc-test] Error 1
make[1]:*** [CMakeFiles/orc-test.dir/all] Error 2
make: ***[all] Error 2

Also, I'm curious about the installed
~/home/brew/lib/liborc.a
did not have permission to execute
The error did not change when I added execution permission in chmod

ls-l to /homebrew/lib/liborc.a
-rw-r--r--1xxxxxxxxxxxxx219139122 18 23:07/Users/xxxxx/homebrew/lib/liborc.a

The file definitely exists in

Note 3

cmake_minimum_required (VERSION 2.8)
project(orc-test CXX)
set(CMAKE_CXX_STANDARD11)

find_package (Threads REQUIRED)
include_directories(~/homebrew/include)
link_directories(~/homebrew/lib)
add_executable(orc-test main.cpp)
target_link_libraries(orc-test)
    orc
    snappy
    protobuf
    protocol
    lz4
    zstd
    z
    Threads::Threads
)

I tried to change it like this, but it still fails

Undefined symbols for architecture x86_64:
  "hdfs::FileSystem::New(hdfs::IoService*&, std::__1::basic_string<char, std::__1::char_trats<char>std:::__1::allocator<char>>>const&hdfs::::::::conf) "
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::ConfigParser::LoadDefaultResources()" from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::ConfigParser::ConfigParser()" from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::ConfigParser::~ConfigParser()" from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::URI::parse_from_string(std::__1::basic_string<char, std::__1::char_trats<char>, std:::__1::allocator<char>>const&), referred from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::URI::URI()" from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::Options::Options()" from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::StatInfo::StatInfo()" from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::IoService::New()" from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::ConfigParser::get_options(hdfs::Options &)const", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::ConfigParser::ValidateResources()const", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::URI::str(bool)const", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::URI::get_host(bool)const", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::URI::get_path(bool)const", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::URI::get_port()const", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::URI::has_port()const", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
  "hdfs::Status::ToString()const", referenced from:
      orc::HdfsFileInputStream::read(void*, unsigned long long, unsigned long long) in liborc.a(OrcHdfsFile.cc.o)
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_trats<char>, std::__1::allocator<char>>>) inliborc.a(OrcHdfsFile.cc)o)
ld —symbol(s) not found for architecture x86_64

The error content is the same, but where the error occurs
Replace with hdfs::FileSystem instead of orc::createWriter

How do I know that libraries like this are used in the first place?

Could you tell me the compilation command when it works?

/Library/Developer/CommandLineTools/usr/bin/c++-isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk-Wl, -search_paths_first-Wl, -headerpad_max_install_names-L$HreomboomwE2/Lib/HreomwE$Lib/Live/HOME $HOME$HOME/L/L/LOME $HOP/HOP/L/L/L/HOME$HOME/readline/lib-L$HOME/homebrew/opt/opensl/lib-L$HOME/homebrew/opt/libffi/lib-L$HOME/homebrew/opt/gettext/lib CMakeFiles/orc-test.dir/main.cpp.o-o orc-test-L$HOME/homebrew/lib-lib-librom-lib-lib-librom-lib-lib-librom-lib-librom-librom-librom-librom-librom-librom-librom-librom-librom-librom-librom-librom

In my environment, I try to compile with these commands.
Maybe it's because I'm writing unnecessary things on .zshrc

c++ build cmake

2022-09-30 21:55

2 Answers

Is it because the library path is not set?
Try adding this 

link_directories(~/homebrew/lib)

Additional
target_link_libraries must be followed by add_executable.
Try it here.

cmake_minimum_required (VERSION 2.8)
project(orc-test CXX)
set(CMAKE_CXX_STANDARD11)

include_directories(~/homebrew/include)
link_directories(~/homebrew/lib)
add_executable(orc-test main.cpp)
target_link_libraries(orc-testorc)

Note 2
I even checked the build on hand.
The specified library was missing and pthread was required.
Adding libraries depends on the order listed (probably orc at the top) so be careful.

cmake_minimum_required (VERSION 2.8)
project(orc-test CXX)
set(CMAKE_CXX_STANDARD11)

find_package (Threads REQUIRED)
include_directories(~/homebrew/include)
link_directories(~/homebrew/lib)
add_executable(orc-test main.cpp)
target_link_libraries(orc-test)
    orc
    snappy
    protobuf
    protocol
    lz4
    zstd
    z
    Threads::Threads
)

Note 3
You see an architectural error, but are you able to browse the library you built?
This is a CentOS 7 build confirmation, but I think it's the same on a Mac.
If you use the packaged libraries after build, there should be no errors.

Build ORC 

$cdorc
$ mkdir build
$ cd build
$ cmake..-DBUILD_JAVA=OFF
$ make package
$ make test-out
$ mvORC-1.7.0-SNAPSHOT-Linux.tar.gz to /.
$ cd~/
$ tar xfz ORC-1.7.0-SNAPSHOT-Linux.tar.gz
$ cd ORC-1.7.0-SNAPSHOT-Linux/

You should be in the package you have built so far.
Place main.cpp and CMakeLists.txt here.

$mkdir build
$ cd build
$ cmake..
$ make
$ ./orc-test
$ cat my-file.orc
ORC

orc-test has been created, so running it should generate my-file.orc.

Also leave the CMakeLists.txt.The questioner does not have -lpthread in his compilation command. Are you adding find_package and Threads::Threads?

cmake_minimum_required (VERSION 2.8)
project(orc-test CXX)
set(CMAKE_CXX_STANDARD11)
find_package (Threads REQUIRED)
include_directories (~/ORC-1.7.0-SNAPSHOT-Linux/include)
link_directories (~/ORC-1.7.0-SNAPSHOT-Linux/lib)
add_executable(orc-test main.cpp)
target_link_libraries(orc-test)
    orc
    snappy
    protobuf
    protocol
    lz4
    zstd
    z
    Threads::Threads
)

I will also include the compilation commands.

/usr/bin/c++-rdynamic CMakeFiles/orc-test.dir/main.cpp.o-oorc-test-L/home/user/ORC-1.7.0-SNAPSHOT-Linux/lib-Wl, -rpath, /home/user/ORC-1.7.0-SNAPSHOT-Lubrot-lup

2022-09-30 21:55

If you're having trouble with library dependencies on a project using CMake, you can leave it to vcpkg.

Example was created in GitHub.
You can clone it as follows (it will take some time to download and build all the libraries you depend on):

 git clone -- recursive https://github.com/tueda/test-apache-orc.git
cd test-apache-orc
cmake
make

The point is

  • vcpkg is imported as a submodule of Git.
  • vcpkg.jsonSpecifies the library that relies directly on (only orc in this case).
{
  "name": "test-apache-orc",
  "version-string": "0.0.1",
  "dependencies": [
    "orc"
  ]
}
  • CMAKE_TOOLCHAIN_FILE is configured at the beginning of
  • CMakeLists.txt (before project(...)) to use the vcpkg deployed as a submodule.
set(CMAKE_TOOLCHAIN_FILE)
    ${CMAKE_CURRENT_SOURCE_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake
    CACHE STRING "Vcpkg toolchain file")
  • Unfortunately, orc (1.6.4) does not have a target for CMake, so CMakeLists.txt must write a variety of dependent libraries in target_link_libraries(...).However, vcpkg will tell you what to write (or Predient wisdom).
$./vcpkg/vcpkg install orc

...

The package orc does not provide CMake targets:

    find_package(protobuf CONFIG REQUIRED)
    find_package (ZLIB REQUIRED)
    find_package (Snappy CONFIG REQUIRED)

    find_library(LZ4_LIBRARY NAMES lz4dlz4)
    find_library(ZSTD_LIBRARY NAMES zstd)
    find_library(ORC_LIBRARY NAMES orc)

    target_link_libraries(main PRIVATE${ORC_LIBRARY}protobuf::libprotocprotobuf::libprotobufSnappy::snappy${LZ4_LIBRARY}${ZSTD_LIBRARY}ZLIB::::ZLIB)

2022-09-30 21:55

If you have any answers or tips

Popular Tags
python x 4647
android x 1593
java x 1494
javascript x 1427
c x 927
c++ x 878
ruby-on-rails x 696
php x 692
python3 x 685
html x 656
Popular Questions

542 Unable to install versioned in Google Colab

557 Scrap text information after the "View More" button when searching in the Yahoo! News search window

549 PHP ssh2_scp_send fails to send files as intended

708 When building Fast API+Uvicorn environment with PyInstaller, console=False results in an error

546 Understanding How to Configure Google API Key

© 2024 OneMinuteCode. All rights reserved.