[Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with offline compiler
Guo, Yejun
yejun.guo at intel.com
Tue Jan 6 19:25:14 PST 2015
Yes, I'm talking about test_load_program_from_bin_file. During build time, the source kernel is built into binary "compiler_ceil.bin" with the standalone compiler. During run time, the binary file is loaded and executed, no compiler is needed at this time.
-----Original Message-----
From: Zhigang Gong [mailto:zhigang.gong at linux.intel.com]
Sent: Wednesday, January 07, 2015 11:11 AM
To: Guo, Yejun
Cc: beignet at lists.freedesktop.org
Subject: RE: [Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with offline compiler
> -----Original Message-----
> From: Beignet [mailto:beignet-bounces at lists.freedesktop.org] On Behalf
> Of Guo, Yejun
> Sent: Wednesday, January 7, 2015 10:54 AM
> To: Zhigang Gong
> Cc: beignet at lists.freedesktop.org
> Subject: Re: [Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with
> offline compiler
>
> Thanks for your nice comments, it would be much clear and user friendly.
>
> Could you review the third patch of this patch set? It is not relative
> to the stand alone compiler, it is about to enable rtti compile
> option, it is needed for some old stl version.
I will review it latter.
>
> For the profile discussion, agree that the
> CL_DEVICE_COMPILER_AVAILABLE could be true if we provide the stand
> alone compiler. Just to clarify that the current implementation is to
> report false if unable to load libgbe.so, we can add the stand alone
> compiler into consideration, anyway, it can be discussed in a separate thread.
Sure, that's next step. We could discuss it after this series laned.
>
> For the test case demo, there is a utest case shows how to use the
> binary kernel, which is prebuilt by the standalone compiler, I'll
> modify them to make it clearer.
I just checked both binary load test cases:
test_load_program_from_bin_file
test_load_program_from_gen_bin
None of them could work on target system without compiler. Is there any other cases I missed?
>
> For the CMake option name USE_STANDALONE_GBE_COMPILER,
> STANDALONE_GBE_COMPILER_DIR, BUILD_STANDALONE_GBE_COMPILER, and utest
> disable, and the document, do not object your comments, and will
> modify accordingly.
OK, thanks.
>
> -----Original Message-----
> From: Beignet [mailto:beignet-bounces at lists.freedesktop.org] On Behalf
> Of Zhigang Gong
> Sent: Wednesday, January 07, 2015 8:31 AM
> To: Guo, Yejun
> Cc: beignet at lists.freedesktop.org
> Subject: Re: [Beignet] [PATCH 2/3] support CL_EMBEDDED_PROFILE with
> offline compiler
>
> Thanks for the patchset.
>
> I have the following comments as below:
>
> 1. It's better to use USE_STANDALONE_GBE_COMPILER rather than define a
> STATIC_GBE_COMPILER_PATH to indicate we will use that standalone
> gbe compiler. So we use USE_STANDALONE_GBE_COMPILER to control
> whether
> we will use a standalone compiler, then at configuration stage, we
> need to write a new CMake script to find the
> STANDALONE_GBE_COMPILER_DIR.
> It may search from some default path and prefer to find it from the
> environment variable GBE_BIN_GENERATER_PATH if possible.
>
> 2. If we set BUILD_STANDALONE_GBE_COMPILER, We'd better to write a new
> cmake
> target to generate such a standalone tarball includes:
> beignet/beignet.bc
> beignet/beignet.pch
> beignet/include/ocl_xxx.h
> beignet/gbe_bin_generator
> Then user just need to untar this file to a standard path, such as
> /usr/local/lib
> or /usr/lib. Then on target system, user just need to set
> USE_STANDALONE_GBE_COMPILER
> and without set any other environment variables as the cmake script will
> search that directory. If user want to put it in a customized place, then
> user need to specify GBE_BIN_GENERATER_PATH.
>
> 3. As to the CL_EMBEDDED_PROFILE profile, I know you are talking about the
> CL_DEVICE_COMPILER_AVAILABLE which must be true for full profile and
> could be false for embedded profile. But other than that, we are comply
> with full profile even on target system. And if the standalone compiler
> could work, we even can make the compiler avialable, right? We need to
> document the detail difference rather than say we support
> CL_EMBEDDED_PROFILE,
> and I also don't see you change the:
> DECL_INFO_STRING(profile, "FULL_PROFILE")
> I agree not to change that.
>
> 4. We need to disable all the unit test cases which require compiler.
>
> 5. We need to add some test cases to demostrate how to build gen
> kernel binary
> and how to load and run the gen kernel binary on target system.
>
> 6. We need a new document to specify the detail instructions of how to use this
> new function.
>
> Thanks,
> Zhigang Gong.
>
> On Wed, Dec 31, 2014 at 03:35:19PM +0800, Guo Yejun wrote:
> > At some platforms with old c/c++ environment, only
> > CL_EMBEDDED_PROFILE is supported, but the offline compiler
> > (gbe_bin_generater and some other files) is still needed during the
> > build. So, add a CMAKE option STATIC_GBE_COMPILER_PATH as the path
> > to the already built offline
> compiler.
> > Once this option is given, all the compiler relative code will not
> > be built any more, only libcl.so and libgebinterp.so will be built.
> >
> > The process looks like:
> > 1. build beignet with option BUILD_STATIC_GBE_COMPILER=true inside
> > mainstream c/c++ environment. The built files under the build
> > directory that we cared here are:
> > backend/src/libocl//usr/local/lib/beignet//beignet.bc
> > backend/src/libocl//usr/local/lib/beignet//beignet.pch
> > backend/src/libocl//usr/local/lib/beignet//include/*
> > backend/src/gbe_bin_generater
> >
> > 2. copy these files to /your_path_to_compiler inside old environment
> > The files under /your_path_to_compiler looks like:
> > beignet.bc beignet.pch gbe_bin_generater include
> >
> > 3. Within old environment, build with
> > STATIC_GBE_COMPILER_PATH=/your_path_to_compiler
> > libcl.so and libgbeinterp.so will be built here, libgbe.so and
> > gbe_bin_generater will not be built here.
> >
> > Signed-off-by: Guo Yejun <yejun.guo at intel.com>
> > ---
> > CMakeLists.txt | 3 +++
> > backend/CMakeLists.txt | 20 +++++++++++++++++---
> > backend/src/CMakeLists.txt | 24 +++++++++++++++++-------
> > src/CMakeLists.txt | 5 +++--
> > utests/CMakeLists.txt | 5 +++--
> > 5 files changed, 43 insertions(+), 14 deletions(-)
> >
> > diff --git a/CMakeLists.txt b/CMakeLists.txt index 4da3561..5cb31c2
> > 100644
> > --- a/CMakeLists.txt
> > +++ b/CMakeLists.txt
> > @@ -76,9 +76,12 @@ set (CMAKE_C_FLAGS_RELWITHDEBINFO "-O2 -g
> -DGBE_DEBUG=1")
> > set (CMAKE_C_FLAGS_MINSIZEREL "-Os -DNDEBUG
> -DGBE_DEBUG=0")
> > set (CMAKE_C_FLAGS_RELEASE "-O2 -DNDEBUG
> -DGBE_DEBUG=0")
> >
> > +# Do not involve llvm if the offline compiler is given IF(NOT
> > +STATIC_GBE_COMPILER_PATH)
> > # Front end stuff we need
> > #INCLUDE(CMake/FindLLVM.cmake)
> > Find_Package(LLVM 3.3)
> > +ENDIF(NOT STATIC_GBE_COMPILER_PATH)
> >
> > set (CMAKE_SHARED_LINKER_FLAGS "-Wl,--no-undefined
> ${LLVM_LDFLAGS}")
> >
> > diff --git a/backend/CMakeLists.txt b/backend/CMakeLists.txt index
> > bf96baf..36b6beb 100644
> > --- a/backend/CMakeLists.txt
> > +++ b/backend/CMakeLists.txt
> > @@ -36,14 +36,28 @@ include_directories
> (${CMAKE_CURRENT_BINARY_DIR})
> > # Project source code
> > ##############################################################
> > add_subdirectory (src)
> > +
> > +if (STATIC_GBE_COMPILER_PATH)
> > +set(LOCAL_OCL_BITCODE_BIN
> ${STATIC_GBE_COMPILER_PATH}/beignet.bc)
> > +set(LOCAL_OCL_HEADER_DIR ${STATIC_GBE_COMPILER_PATH}/include)
> > +set(LOCAL_OCL_PCH_OBJECT
> ${STATIC_GBE_COMPILER_PATH}/beignet.pch)
> > +endif (STATIC_GBE_COMPILER_PATH)
> > +
> > set(LOCAL_OCL_BITCODE_BIN "${LOCAL_OCL_BITCODE_BIN}"
> PARENT_SCOPE)
> > set(LOCAL_OCL_HEADER_DIR "${LOCAL_OCL_HEADER_DIR}"
> PARENT_SCOPE)
> > set(LOCAL_OCL_PCH_OBJECT "${LOCAL_OCL_PCH_OBJECT}"
> PARENT_SCOPE)
> > -
> > set(LOCAL_GBE_OBJECT_DIR ${LOCAL_GBE_OBJECT_DIR}
> PARENT_SCOPE)
> > set(LOCAL_INTERP_OBJECT_DIR ${LOCAL_INTERP_OBJECT_DIR}
> PARENT_SCOPE)
> >
> > set (GBE_BIN_GENERATER
> > - env OCL_BITCODE_LIB_PATH=${LOCAL_OCL_BITCODE_BIN}
> OCL_HEADER_FILE_DIR=${LOCAL_OCL_HEADER_DIR}
> OCL_PCH_PATH=${LOCAL_OCL_PCH_OBJECT}
> LD_LIBRARY_PATH=${CMAKE_CURRENT_BINARY_DIR}/src
> ${CMAKE_CURRENT_BINARY_DIR}/src/gbe_bin_generater
> > - PARENT_SCOPE)
> > + env OCL_BITCODE_LIB_PATH=${LOCAL_OCL_BITCODE_BIN}
> > + OCL_HEADER_FILE_DIR=${LOCAL_OCL_HEADER_DIR}
> > + OCL_PCH_PATH=${LOCAL_OCL_PCH_OBJECT})
> >
> > +if (STATIC_GBE_COMPILER_PATH)
> > +set (GBE_BIN_GENERATER
> > + ${GBE_BIN_GENERATER}
> ${STATIC_GBE_COMPILER_PATH}/gbe_bin_generater
> > + PARENT_SCOPE)
> > +else (STATIC_GBE_COMPILER_PATH)
> > +set (GBE_BIN_GENERATER
> > + ${GBE_BIN_GENERATER}
> LD_LIBRARY_PATH=${CMAKE_CURRENT_BINARY_DIR}/src
> ${CMAKE_CURRENT_BINARY_DIR}/src/gbe_bin_generater
> > + PARENT_SCOPE)
> > +endif (STATIC_GBE_COMPILER_PATH)
> > diff --git a/backend/src/CMakeLists.txt b/backend/src/CMakeLists.txt
> > index 118a745..81cf7b8 100644
> > --- a/backend/src/CMakeLists.txt
> > +++ b/backend/src/CMakeLists.txt
> > @@ -9,15 +9,19 @@ configure_file (
> > "GBEConfig.h"
> > )
> >
> > +#do not involve libocl if the offline compiler is given, #since we
> > +will not build the online compiler if (NOT
> > +STATIC_GBE_COMPILER_PATH)
> > add_subdirectory(libocl)
> > +add_dependencies(beignet_bitcode libocl) endif ()
> > +
> > set (LOCAL_GBE_OBJECT_DIR
> "${CMAKE_CURRENT_BINARY_DIR}/libgbe.so"
> > PARENT_SCOPE) set (LOCAL_INTERP_OBJECT_DIR
> > "${CMAKE_CURRENT_BINARY_DIR}/libgbeinterp.so" PARENT_SCOPE) set
> > (LOCAL_OCL_BITCODE_BIN "${OCL_OBJECT_DIR}/beignet.bc"
> PARENT_SCOPE)
> > set (LOCAL_OCL_HEADER_DIR "${OCL_OBJECT_DIR}/include/"
> PARENT_SCOPE)
> > set (LOCAL_OCL_PCH_OBJECT "${OCL_OBJECT_DIR}/beignet.local.pch"
> > PARENT_SCOPE)
> >
> > -add_dependencies(beignet_bitcode libocl)
> > -
> > set (GBE_SRC
> > ${ocl_blob_file}
> > sys/vector.hpp
> > @@ -130,14 +134,16 @@ set (GBE_LINK_LIBRARIES include_directories
> > (.) link_directories (${LLVM_LIBRARY_DIRS} ${DRM_LIBDIR})
> > include_directories(${LLVM_INCLUDE_DIRS})
> > -add_library (gbe SHARED ${GBE_SRC})
> >
> > +#don't build libgbe (the online compiler) if the offline compiler
> > +is given if (NOT STATIC_GBE_COMPILER_PATH) add_library (gbe SHARED
> > +${GBE_SRC})
> > target_link_libraries(gbe ${GBE_LINK_LIBRARIES})
> > +add_dependencies(gbe beignet_bitcode) endif (NOT
> > +STATIC_GBE_COMPILER_PATH)
> >
> > add_library(gbeinterp SHARED gbe_bin_interpreter.cpp)
> >
> > -add_dependencies(gbe beignet_bitcode)
> > -
> > if (LLVM_VERSION_NODOT VERSION_EQUAL 34)
> > find_library(TERMINFO NAMES tinfo ncurses)
> > if (${TERMINFO} STREQUAL TERMINFO-NOTFOUND) @@ -150,6 +156,9
> @@
> > endif(LLVM_VERSION_NODOT VERSION_EQUAL 34)
> >
> > link_directories (${LLVM_LIBRARY_DIR} ${DRM_LIBDIR})
> >
> > +
> > +#if the offline compiler is given, do not build/install it anymore
> > +if (NOT STATIC_GBE_COMPILER_PATH)
> > if (BUILD_STATIC_GBE_COMPILER STREQUAL "true")
> macro(remove_cxx_flag
> > flag)
> > string(REPLACE "${flag}" "" CMAKE_CXX_FLAGS
> "${CMAKE_CXX_FLAGS}")
> > @@ -164,9 +173,10 @@ ADD_EXECUTABLE(gbe_bin_generater
> > gbe_bin_generater.cpp) TARGET_LINK_LIBRARIES(gbe_bin_generater gbe)
> > endif (BUILD_STATIC_GBE_COMPILER STREQUAL "true")
> >
> > -
> > install (TARGETS gbe LIBRARY DESTINATION ${BEIGNET_INSTALL_DIR})
> > -install (TARGETS gbeinterp LIBRARY DESTINATION
> > ${BEIGNET_INSTALL_DIR}) install (FILES ${OCL_OBJECT_DIR}/beignet.bc
> > DESTINATION ${BEIGNET_INSTALL_DIR}) install (FILES
> > ${OCL_OBJECT_DIR}/beignet.pch DESTINATION ${BEIGNET_INSTALL_DIR})
> > install (FILES ${OCL_HEADER_FILES} DESTINATION
> > ${BEIGNET_INSTALL_DIR}/include)
> > +endif (NOT STATIC_GBE_COMPILER_PATH)
> > +
> > +install (TARGETS gbeinterp LIBRARY DESTINATION
> > +${BEIGNET_INSTALL_DIR})
> > diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index
> > 7182bad..a55f84d 100644
> > --- a/src/CMakeLists.txt
> > +++ b/src/CMakeLists.txt
> > @@ -10,18 +10,19 @@ foreach (KF ${KERNEL_FILES})
> > set (input_file ${KERNEL_PATH}/${KF}.cl)
> > set (output_file ${KERNEL_PATH}/${KF}_str.c)
> > list (APPEND KERNEL_STR_FILES ${output_file})
> > + list (GET GBE_BIN_GENERATER -1 GBE_BIN_FILE)
> > if(GEN_PCI_ID)
> > add_custom_command(
> > OUTPUT ${output_file}
> > COMMAND rm -rf ${output_file}
> > COMMAND ${GBE_BIN_GENERATER} -s ${input_file}
> > -o${output_file}
> -t${GEN_PCI_ID}
> > - DEPENDS ${input_file}
> ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater)
> > + DEPENDS ${input_file} ${GBE_BIN_FILE})
> > else(GEN_PCI_ID)
> > add_custom_command(
> > OUTPUT ${output_file}
> > COMMAND rm -rf ${output_file}
> > COMMAND ${GBE_BIN_GENERATER} -s ${input_file}
> -o${output_file}
> > - DEPENDS ${input_file}
> ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater)
> > + DEPENDS ${input_file} ${GBE_BIN_FILE})
> > endif(GEN_PCI_ID)
> > endforeach (KF)
> > endmacro (MakeKernelBinStr)
> > diff --git a/utests/CMakeLists.txt b/utests/CMakeLists.txt index
> > 5b29c0b..9623dc1 100644
> > --- a/utests/CMakeLists.txt
> > +++ b/utests/CMakeLists.txt
> > @@ -219,16 +219,17 @@ endif (X11_FOUND)
> >
> > SET (kernel_bin
> ${CMAKE_CURRENT_SOURCE_DIR}/../kernels/compiler_ceil)
> >
> > +list (GET GBE_BIN_GENERATER -1 GBE_BIN_FILE)
> > if(GEN_PCI_ID)
> > ADD_CUSTOM_COMMAND(
> > OUTPUT ${kernel_bin}.bin
> > COMMAND ${GBE_BIN_GENERATER} ${kernel_bin}.cl -o${kernel_bin}.bin
> > -t${GEN_PCI_ID}
> > - DEPENDS
> > ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater
> > ${kernel_bin}.cl)
> > + DEPENDS ${GBE_BIN_FILE} ${kernel_bin}.cl)
> > else(GEN_PCI_ID)
> > ADD_CUSTOM_COMMAND(
> > OUTPUT ${kernel_bin}.bin
> > COMMAND ${GBE_BIN_GENERATER} ${kernel_bin}.cl -o${kernel_bin}.bin
> > - DEPENDS
> > ${CMAKE_CURRENT_BINARY_DIR}/../backend/src/gbe_bin_generater
> > ${kernel_bin}.cl)
> > + DEPENDS ${GBE_BIN_FILE} ${kernel_bin}.cl)
> > endif(GEN_PCI_ID)
> >
> > ADD_CUSTOM_TARGET(kernel_bin.bin
> > --
> > 1.9.1
> >
> > _______________________________________________
> > Beignet mailing list
> > Beignet at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/beignet
> _______________________________________________
> Beignet mailing list
> Beignet at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/beignet
> _______________________________________________
> Beignet mailing list
> Beignet at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/beignet
More information about the Beignet
mailing list