[Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with offline compiler
Guo, Yejun
yejun.guo at intel.com
Tue Jan 6 20:09:08 PST 2015
Yes, I expected to compile with -t option. Will also refine the gbe_bin_generator usage information.
-----Original Message-----
From: Zhigang Gong [mailto:zhigang.gong at linux.intel.com]
Sent: Wednesday, January 07, 2015 10:40 AM
To: Guo, Yejun
Cc: beignet at lists.freedesktop.org
Subject: Re: [Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with offline compiler
On Wed, Jan 07, 2015 at 03:25:14AM +0000, Guo, Yejun wrote:
> Yes, I'm talking about test_load_program_from_bin_file. During build time, the source kernel is built into binary "compiler_ceil.bin" with the standalone compiler. During run time, the binary file is loaded and executed, no compiler is needed at this time.
By default, compiler_ceil.bin will be a llvm binary rather than a gen binary. Thus it still requires compiler when use it on the target system. Please also refine the gbe_bin_generator usage information, current information is incomplete:
if (argc < 2) {
cout << "Usage: kernel_path [-pbuild_parameter]\n[-obin_path]" << endl;
return 0;
}
We need to add a -t option which is used to specify the gen target pci id.
And if it is not provided, the gbe_bin_generator will generate llvm bitcode rather than gen binary.
If we want to make this test workable on target system which doesn't have gbe compiler, we must set the -t pci_id options when invoke the gbe_bin_generator to create the compiler_ceil.bin.
>
>
> -----Original Message-----
> From: Zhigang Gong [mailto:zhigang.gong at linux.intel.com]
> Sent: Wednesday, January 07, 2015 11:11 AM
> To: Guo, Yejun
> Cc: beignet at lists.freedesktop.org
> Subject: RE: [Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with
> offline compiler
>
> > -----Original Message-----
> > From: Beignet [mailto:beignet-bounces at lists.freedesktop.org] On
> > Behalf Of Guo, Yejun
> > Sent: Wednesday, January 7, 2015 10:54 AM
> > To: Zhigang Gong
> > Cc: beignet at lists.freedesktop.org
> > Subject: Re: [Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with
> > offline compiler
> >
> > Thanks for your nice comments, it would be much clear and user friendly.
> >
> > Could you review the third patch of this patch set? It is not
> > relative to the stand alone compiler, it is about to enable rtti
> > compile option, it is needed for some old stl version.
>
> I will review it latter.
>
> >
> > For the profile discussion, agree that the
> > CL_DEVICE_COMPILER_AVAILABLE could be true if we provide the stand
> > alone compiler. Just to clarify that the current implementation is
> > to report false if unable to load libgbe.so, we can add the stand
> > alone compiler into consideration, anyway, it can be discussed in a separate thread.
>
> Sure, that's next step. We could discuss it after this series laned.
>
> >
> > For the test case demo, there is a utest case shows how to use the
> > binary kernel, which is prebuilt by the standalone compiler, I'll
> > modify them to make it clearer.
>
> I just checked both binary load test cases:
> test_load_program_from_bin_file
> test_load_program_from_gen_bin
>
> None of them could work on target system without compiler. Is there any other cases I missed?
>
>
> >
> > For the CMake option name USE_STANDALONE_GBE_COMPILER,
> > STANDALONE_GBE_COMPILER_DIR, BUILD_STANDALONE_GBE_COMPILER, and
> > utest disable, and the document, do not object your comments, and
> > will modify accordingly.
>
> OK, thanks.
>
> >
> > -----Original Message-----
> > From: Beignet [mailto:beignet-bounces at lists.freedesktop.org] On
> > Behalf Of Zhigang Gong
> > Sent: Wednesday, January 07, 2015 8:31 AM
> > To: Guo, Yejun
> > Cc: beignet at lists.freedesktop.org
> > Subject: Re: [Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with
> > offline compiler
> >
> > Thanks for the patchset.
> >
> > I have the following comments as below:
> >
> > 1. It's better to use USE_STANDALONE_GBE_COMPILER rather than define a
> > STATIC_GBE_COMPILER_PATH to indicate we will use that standalone
> > gbe compiler. So we use USE_STANDALONE_GBE_COMPILER to control
> > whether
> > we will use a standalone compiler, then at configuration stage, we
> > need to write a new CMake script to find the
> > STANDALONE_GBE_COMPILER_DIR.
> > It may search from some default path and prefer to find it from the
> > environment variable GBE_BIN_GENERATER_PATH if possible.
> >
> > 2. If we set BUILD_STANDALONE_GBE_COMPILER, We'd better to write a
> > new cmake
> > target to generate such a standalone tarball includes:
> > beignet/beignet.bc
> > beignet/beignet.pch
> > beignet/include/ocl_xxx.h
> > beignet/gbe_bin_generator
> > Then user just need to untar this file to a standard path, such
> > as /usr/local/lib
> > or /usr/lib. Then on target system, user just need to set
> > USE_STANDALONE_GBE_COMPILER
> > and without set any other environment variables as the cmake script will
> > search that directory. If user want to put it in a customized place, then
> > user need to specify GBE_BIN_GENERATER_PATH.
> >
> > 3. As to the CL_EMBEDDED_PROFILE profile, I know you are talking about the
> > CL_DEVICE_COMPILER_AVAILABLE which must be true for full profile and
> > could be false for embedded profile. But other than that, we are comply
> > with full profile even on target system. And if the standalone compiler
> > could work, we even can make the compiler avialable, right? We need to
> > document the detail difference rather than say we support
> > CL_EMBEDDED_PROFILE,
> > and I also don't see you change the:
> > DECL_INFO_STRING(profile, "FULL_PROFILE")
> > I agree not to change that.
> >
> > 4. We need to disable all the unit test cases which require compiler.
> >
> > 5. We need to add some test cases to demostrate how to build gen
> > kernel binary
> > and how to load and run the gen kernel binary on target system.
> >
> > 6. We need a new document to specify the detail instructions of how to use this
> > new function.
> >
> > Thanks,
> > Zhigang Gong.
> >
> > On Wed, Dec 31, 2014 at 03:35:19PM +0800, Guo Yejun wrote:
> > > At some platforms with old c/c++ environment, only
> > > CL_EMBEDDED_PROFILE is supported, but the offline compiler
> > > (gbe_bin_generater and some other files) is still needed during
> > > the build. So, add a CMAKE option STATIC_GBE_COMPILER_PATH as the
> > > path to the already built offline
> > compiler.
> > > Once this option is given, all the compiler relative code will not
> > > be built any more, only libcl.so and libgebinterp.so will be built.
> > >
> > > The process looks like:
> > > 1. build beignet with option BUILD_STATIC_GBE_COMPILER=true inside
> > > mainstream c/c++ environment. The built files under the build
> > > directory that we cared here are:
> > > backend/src/libocl//usr/local/lib/beignet//beignet.bc
> > > backend/src/libocl//usr/local/lib/beignet//beignet.pch
> > > backend/src/libocl//usr/local/lib/beignet//include/*
> > > backend/src/gbe_bin_generater
> > >
> > > 2. copy these files to /your_path_to_compiler inside old
> > > environment The files under /your_path_to_compiler looks like:
> > > beignet.bc beignet.pch gbe_bin_generater include
> > >
> > > 3. Within old environment, build with
> > > STATIC_GBE_COMPILER_PATH=/your_path_to_compiler
> > > libcl.so and libgbeinterp.so will be built here, libgbe.so and
> > > gbe_bin_generater will not be built here.
> > >
> > > Signed-off-by: Guo Yejun <yejun.guo at intel.com>
> > > ---
> > > CMakeLists.txt | 3 +++
> > > backend/CMakeLists.txt | 20 +++++++++++++++++---
> > > backend/src/CMakeLists.txt | 24 +++++++++++++++++-------
> > > src/CMakeLists.txt | 5 +++--
> > > utests/CMakeLists.txt | 5 +++--
> > > 5 files changed, 43 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/CMakeLists.txt b/CMakeLists.txt index
> > > 4da3561..5cb31c2
> > > 100644
> > > --- a/CMakeLists.txt
> > > +++ b/CMakeLists.txt
> > > @@ -76,9 +76,12 @@ set (CMAKE_C_FLAGS_RELWITHDEBINFO "-O2 -g
> > -DGBE_DEBUG=1")
> > > set (CMAKE_C_FLAGS_MINSIZEREL "-Os -DNDEBUG
> > -DGBE_DEBUG=0")
> > > set (CMAKE_C_FLAGS_RELEASE "-O2 -DNDEBUG
> > -DGBE_DEBUG=0")
> > >
> > > +# Do not involve llvm if the offline compiler is given IF(NOT
> > > +STATIC_GBE_COMPILER_PATH)
> > > # Front end stuff we need
> > > #INCLUDE(CMake/FindLLVM.cmake)
> > > Find_Package(LLVM 3.3)
> > > +ENDIF(NOT STATIC_GBE_COMPILER_PATH)
> > >
> > > set (CMAKE_SHARED_LINKER_FLAGS "-Wl,--no-undefined
> > ${LLVM_LDFLAGS}")
> > >
> > > diff --git a/backend/CMakeLists.txt b/backend/CMakeLists.txt index
> > > bf96baf..36b6beb 100644
> > > --- a/backend/CMakeLists.txt
> > > +++ b/backend/CMakeLists.txt
> > > @@ -36,14 +36,28 @@ include_directories
> > (${CMAKE_CURRENT_BINARY_DIR})
> > > # Project source code
> > > ##############################################################
> > > add_subdirectory (src)
> > > +
> > > +if (STATIC_GBE_COMPILER_PATH)
> > > +set(LOCAL_OCL_BITCODE_BIN
> > ${STATIC_GBE_COMPILER_PATH}/beignet.bc)
> > > +set(LOCAL_OCL_HEADER_DIR ${STATIC_GBE_COMPILER_PATH}/include)
> > > +set(LOCAL_OCL_PCH_OBJECT
> > ${STATIC_GBE_COMPILER_PATH}/beignet.pch)
> > > +endif (STATIC_GBE_COMPILER_PATH)
> > > +
> > > set(LOCAL_OCL_BITCODE_BIN "${LOCAL_OCL_BITCODE_BIN}"
> > PARENT_SCOPE)
> > > set(LOCAL_OCL_HEADER_DIR "${LOCAL_OCL_HEADER_DIR}"
> > PARENT_SCOPE)
> > > set(LOCAL_OCL_PCH_OBJECT "${LOCAL_OCL_PCH_OBJECT}"
> > PARENT_SCOPE)
> > > -
> > > set(LOCAL_GBE_OBJECT_DIR ${LOCAL_GBE_OBJECT_DIR}
> > PARENT_SCOPE)
> > > set(LOCAL_INTERP_OBJECT_DIR ${LOCAL_INTERP_OBJECT_DIR}
> > PARENT_SCOPE)
> > >
> > > set (GBE_BIN_GENERATER
> > > - env OCL_BITCODE_LIB_PATH=${LOCAL_OCL_BITCODE_BIN}
> > OCL_HEADER_FILE_DIR=${LOCAL_OCL_HEADER_DIR}
> > OCL_PCH_PATH=${LOCAL_OCL_PCH_OBJECT}
> > LD_LIBRARY_PATH=${CMAKE_CURRENT_BINARY_DIR}/src
> > ${CMAKE_CURRENT_BINARY_DIR}/src/gbe_bin_generater
> > > - PARENT_SCOPE)
> > > + env OCL_BITCODE_LIB_PATH=${LOCAL_OCL_BITCODE_BIN}
> > > + OCL_HEADER_FILE_DIR=${LOCAL_OCL_HEADER_DIR}
> > > + OCL_PCH_PATH=${LOCAL_OCL_PCH_OBJECT})
> > >
> > > +if (STATIC_GBE_COMPILER_PATH)
> > > +set (GBE_BIN_GENERATER
> > > + ${GBE_BIN_GENERATER}
> > ${STATIC_GBE_COMPILER_PATH}/gbe_bin_generater
> > > + PARENT_SCOPE)
> > > +else (STATIC_GBE_COMPILER_PATH)
> > > +set (GBE_BIN_GENERATER
> > > + ${GBE_BIN_GENERATER}
> > LD_LIBRARY_PATH=${CMAKE_CURRENT_BINARY_DIR}/src
> > ${CMAKE_CURRENT_BINARY_DIR}/src/gbe_bin_generater
> > > + PARENT_SCOPE)
> > > +endif (STATIC_GBE_COMPILER_PATH)
> > > diff --git a/backend/src/CMakeLists.txt
> > > b/backend/src/CMakeLists.txt index 118a745..81cf7b8 100644
> > > --- a/backend/src/CMakeLists.txt
> > > +++ b/backend/src/CMakeLists.txt
> > > @@ -9,15 +9,19 @@ configure_file (
> > > "GBEConfig.h"
> > > )
> > >
> > > +#do not involve libocl if the offline compiler is given, #since
> > > +we will not build the online compiler if (NOT
> > > +STATIC_GBE_COMPILER_PATH)
> > > add_subdirectory(libocl)
> > > +add_dependencies(beignet_bitcode libocl) endif ()
> > > +
> > > set (LOCAL_GBE_OBJECT_DIR
> > "${CMAKE_CURRENT_BINARY_DIR}/libgbe.so"
> > > PARENT_SCOPE) set (LOCAL_INTERP_OBJECT_DIR
> > > "${CMAKE_CURRENT_BINARY_DIR}/libgbeinterp.so" PARENT_SCOPE) set
> > > (LOCAL_OCL_BITCODE_BIN "${OCL_OBJECT_DIR}/beignet.bc"
> > PARENT_SCOPE)
> > > set (LOCAL_OCL_HEADER_DIR "${OCL_OBJECT_DIR}/include/"
> > PARENT_SCOPE)
> > > set (LOCAL_OCL_PCH_OBJECT "${OCL_OBJECT_DIR}/beignet.local.pch"
> > > PARENT_SCOPE)
> > >
> > > -add_dependencies(beignet_bitcode libocl)
> > > -
> > > set (GBE_SRC
> > > ${ocl_blob_file}
> > > sys/vector.hpp
> > > @@ -130,14 +134,16 @@ set (GBE_LINK_LIBRARIES include_directories
> > > (.) link_directories (${LLVM_LIBRARY_DIRS} ${DRM_LIBDIR})
> > > include_directories(${LLVM_INCLUDE_DIRS})
> > > -add_library (gbe SHARED ${GBE_SRC})
> > >
> > > +#don't build libgbe (the online compiler) if the offline compiler
> > > +is given if (NOT STATIC_GBE_COMPILER_PATH) add_library (gbe
> > > +SHARED
> > > +${GBE_SRC})
> > > target_link_libraries(gbe ${GBE_LINK_LIBRARIES})
> > > +add_dependencies(gbe beignet_bitcode) endif (NOT
> > > +STATIC_GBE_COMPILER_PATH)
> > >
> > > add_library(gbeinterp SHARED gbe_bin_interpreter.cpp)
> > >
> > > -add_dependencies(gbe beignet_bitcode)
> > > -
> > > if (LLVM_VERSION_NODOT VERSION_EQUAL 34)
> > > find_library(TERMINFO NAMES tinfo ncurses)
> > > if (${TERMINFO} STREQUAL TERMINFO-NOTFOUND) @@ -150,6 +156,9
> > @@
> > > endif(LLVM_VERSION_NODOT VERSION_EQUAL 34)
> > >
> > > link_directories (${LLVM_LIBRARY_DIR} ${DRM_LIBDIR})
> > >
> > > +
> > > +#if the offline compiler is given, do not build/install it
> > > +anymore if (NOT STATIC_GBE_COMPILER_PATH)
> > > if (BUILD_STATIC_GBE_COMPILER STREQUAL "true")
> > macro(remove_cxx_flag
> > > flag)
> > > string(REPLACE "${flag}" "" CMAKE_CXX_FLAGS
> > "${CMAKE_CXX_FLAGS}")
> > > @@ -164,9 +173,10 @@ ADD_EXECUTABLE(gbe_bin_generater
> > > gbe_bin_generater.cpp) TARGET_LINK_LIBRARIES(gbe_bin_generater
> > > gbe) endif (BUILD_STATIC_GBE_COMPILER STREQUAL "true")
> > >
> > > -
> > > install (TARGETS gbe LIBRARY DESTINATION ${BEIGNET_INSTALL_DIR})
> > > -install (TARGETS gbeinterp LIBRARY DESTINATION
> > > ${BEIGNET_INSTALL_DIR}) install (FILES
> > > ${OCL_OBJECT_DIR}/beignet.bc DESTINATION ${BEIGNET_INSTALL_DIR})
> > > install (FILES ${OCL_OBJECT_DIR}/beignet.pch DESTINATION
> > > ${BEIGNET_INSTALL_DIR}) install (FILES ${OCL_HEADER_FILES}
> > > DESTINATION
> > > ${BEIGNET_INSTALL_DIR}/include)
> > > +endif (NOT STATIC_GBE_COMPILER_PATH)
> > > +
> > > +install (TARGETS gbeinterp LIBRARY DESTINATION
> > > +${BEIGNET_INSTALL_DIR})
> > > diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index
> > > 7182bad..a55f84d 100644
> > > --- a/src/CMakeLists.txt
> > > +++ b/src/CMakeLists.txt
> > > @@ -10,18 +10,19 @@ foreach (KF ${KERNEL_FILES})
> > > set (input_file ${KERNEL_PATH}/${KF}.cl)
> > > set (output_file ${KERNEL_PATH}/${KF}_str.c)
> > > list (APPEND KERNEL_STR_FILES ${output_file})
> > > + list (GET GBE_BIN_GENERATER -1 GBE_BIN_FILE)
> > > if(GEN_PCI_ID)
> > > add_custom_command(
> > > OUTPUT ${output_file}
> > > COMMAND rm -rf ${output_file}
> > > COMMAND ${GBE_BIN_GENERATER} -s ${input_file}
> > > -o${output_file}
> > -t${GEN_PCI_ID}
> > > - DEPENDS ${input_file}
> > ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater)
> > > + DEPENDS ${input_file} ${GBE_BIN_FILE})
> > > else(GEN_PCI_ID)
> > > add_custom_command(
> > > OUTPUT ${output_file}
> > > COMMAND rm -rf ${output_file}
> > > COMMAND ${GBE_BIN_GENERATER} -s ${input_file}
> > -o${output_file}
> > > - DEPENDS ${input_file}
> > ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater)
> > > + DEPENDS ${input_file} ${GBE_BIN_FILE})
> > > endif(GEN_PCI_ID)
> > > endforeach (KF)
> > > endmacro (MakeKernelBinStr)
> > > diff --git a/utests/CMakeLists.txt b/utests/CMakeLists.txt index
> > > 5b29c0b..9623dc1 100644
> > > --- a/utests/CMakeLists.txt
> > > +++ b/utests/CMakeLists.txt
> > > @@ -219,16 +219,17 @@ endif (X11_FOUND)
> > >
> > > SET (kernel_bin
> > ${CMAKE_CURRENT_SOURCE_DIR}/../kernels/compiler_ceil)
> > >
> > > +list (GET GBE_BIN_GENERATER -1 GBE_BIN_FILE)
> > > if(GEN_PCI_ID)
> > > ADD_CUSTOM_COMMAND(
> > > OUTPUT ${kernel_bin}.bin
> > > COMMAND ${GBE_BIN_GENERATER} ${kernel_bin}.cl
> > > -o${kernel_bin}.bin -t${GEN_PCI_ID}
> > > - DEPENDS
> > > ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater
> > > ${kernel_bin}.cl)
> > > + DEPENDS ${GBE_BIN_FILE} ${kernel_bin}.cl)
> > > else(GEN_PCI_ID)
> > > ADD_CUSTOM_COMMAND(
> > > OUTPUT ${kernel_bin}.bin
> > > COMMAND ${GBE_BIN_GENERATER} ${kernel_bin}.cl
> > > -o${kernel_bin}.bin
> > > - DEPENDS
> > > ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater
> > > ${kernel_bin}.cl)
> > > + DEPENDS ${GBE_BIN_FILE} ${kernel_bin}.cl)
> > > endif(GEN_PCI_ID)
> > >
> > > ADD_CUSTOM_TARGET(kernel_bin.bin
> > > --
> > > 1.9.1
> > >
> > > _______________________________________________
> > > Beignet mailing list
> > > Beignet at lists.freedesktop.org
> > > http://lists.freedesktop.org/mailman/listinfo/beignet
> > _______________________________________________
> > Beignet mailing list
> > Beignet at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/beignet
> > _______________________________________________
> > Beignet mailing list
> > Beignet at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/beignet
>
> _______________________________________________
> Beignet mailing list
> Beignet at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/beignet
More information about the Beignet
mailing list