[Beignet] [PATCH] Disable the PCH valid check to save a lot of compiling time.

[He Junyan]

I think we need to consider the policy now.
The clang will do the validation check for PCH as an outside AST source,
so this including:
Language feature, target feature, Preprocessing option and clang version
Target feature seems never to change.
According to CL 1.1's spec, we also just have one or to option which
will change the Language feature.
For Preprocessing option, we can just extract the -DXXX option from the
building options and add back after the check. Clang has such functions.

The clang version seems hard to handle, because we can not assume which
version the user is using. It is also impossible for us the generate one
PCH for each clang version. So I think the dynamic PCH generation is
inevitable.

I want to do like this:
1. Generate one or several PCH for major Clang release versions with no
building option.
2. When compiling the source, extract the -DXXX option out.
3. Call Clang's external AST source validation check function to check
the PCH with current environment. If passing, add back -D macro define
and everything is OK
4. If not, dynamically generating a PCH file under /tmp/ for current
environment and add the proper suffix for re-use later.


Please give some advice,
thanks



On Thu, 2013-12-05 at 15:37 +0800, Zhigang Gong wrote:
> On Thu, Dec 05, 2013 at 03:24:10PM +0800, junyan.he at inbox.com wrote:
> > From: Junyan He <junyan.he at linux.intel.com>
> > 
> > In clang, The PCH file will be used as an AST source, so
> > the check is strict. The macro define is also checked,
> > and if anything is different, the PCH is invalid and
> > the build processing will start from scratch.
> > Disable Clang's PCH valid check and do the compatible
> > check by ourself.
> > 
> > Signed-off-by: Junyan He <junyan.he at linux.intel.com>
> > ---
> >  backend/src/backend/program.cpp | 92 ++++++++++++++++++++++++++++++++++-------
> >  1 file changed, 77 insertions(+), 15 deletions(-)
> > 
> > diff --git a/backend/src/backend/program.cpp b/backend/src/backend/program.cpp
> > index 294bb16..30ad838 100644
> > --- a/backend/src/backend/program.cpp
> > +++ b/backend/src/backend/program.cpp
> > @@ -569,6 +569,9 @@ namespace gbe {
> >      clang::LangOptions & lang_opts = Clang.getLangOpts();
> >      lang_opts.OpenCL = 1;
> >  
> > +    clang::PreprocessorOptions prep_opt = Clang.getPreprocessorOpts();
> > +    prep_opt.DisablePCHValidation = 1;
> > +
> >      //llvm flags need command line parsing to take effect
> >      if (!Clang.getFrontendOpts().LLVMArgs.empty()) {
> >        unsigned NumArgs = Clang.getFrontendOpts().LLVMArgs.size();
> > @@ -645,32 +648,91 @@ namespace gbe {
> >      FILE *clFile = fopen(clName.c_str(), "w");
> >      FATAL_IF(clFile == NULL, "Failed to open temporary file");
> >  
> > -    bool usePCH = false;
> > +    bool usePCH = true;
> > +    bool findPCH = false;
> > +
> > +    /* Because our header file is so big, we want to avoid recompile the header from
> > +       scratch. We use the PCH support of Clang to save the huge compiling time.
> > +       We just use the most general build opt to build the PCH header file, so if
> > +       user pass new build options here, the PCH can not pass the Clang's compitable
> > +       validating. Clang will do three kinds of compatible check: Language Option,
> > +       Target Option and Preprocessing Option. Other kinds of options such as the
> > +       CodeGen options will not affect the AST result, so no need to check.
> > +
> > +       According to OpenCL 1.1's spec, the CL build options:
> > +       -D name=definition
> > +       If the definition is not used in our header, it is compitable
> > +
> > +       -cl-single-precision-constant
> > +       -cl-denorms-are-zero
> > +       -cl-std=
> > +       Language options, really affect.
> > +
> > +       -cl-opt-disable
> > +       -cl-mad-enable
> > +       -cl-no-signed-zeros
> > +       -cl-unsafe-math-optimizations
> > +       -cl-finite-math-only
> > +       -cl-fast-relaxed-math
> > +       CodeGen options, not affect
> > +
> > +       -Werror
> > +       -w
> > +       Our header should not block the compiling because of warning.
> > +
> > +       So we just disable the PCH validation of Clang and do the judgement by ourself. */
> 
> It's better to check whether current clang/llvm's version match with the one which is
> used to build the PCH header file. Different version may be incompatible to each other even
> it can pass all the above checks.
> 
> We may provide more than one versions' PCH files latter to satisfy the most popular
> clang/llvm versions.
> > +
> > +    if(options) {
> > +      char *p;
> > +      const char * incompatible_opts[] = {
> > +          "-cl-single-precision-constant",
> > +          "-cl-denorms-are-zero",
> > +          "-cl-std=",
> > +      };
> > +      const char * incompatible_defs[] = {
> > +          "GET_FLOAT_WORD",
> > +          "__NV_CL_C_VERSION",
> > +          "GEN7_SAMPLER_CLAMP_BORDER_WORKAROUND"
> > +      };
> > +
> > +      for (unsigned int i = 0; i < sizeof(incompatible_opts)/sizeof(char *); i++ ) {
> > +        p = strstr(const_cast<char *>(options), incompatible_opts[i]);
> > +        if (p) {
> > +          usePCH = false;
> > +          break;
> > +        }
> > +      }
> > +
> > +      if (usePCH) {
> > +        for (unsigned int i = 0; i < sizeof(incompatible_defs)/sizeof(char *); i++ ) {
> > +          p = strstr(const_cast<char *>(options), incompatible_defs[i]);
> > +          if (p) {
> > +            usePCH = false;
> > +            break;
> > +          }
> > +        }
> > +      }
> >  
> > -    if(options)
> >        clOpt += options;
> > +    }
> >  
> > -    if (options || !OCL_USE_PCH) {
> > -      /* Some building option may cause the prebuild pch header file
> > -         not compatible with the XXX.cl source. We need rebuild all here.*/
> > -      usePCH = false;
> > -    } else {
> > -      std::string dirs = PCH_OBJECT_DIR;
> > -      std::istringstream idirs(dirs);
> If the usePCH is false here, we do not need to find the pch header file.
> 
> > +    std::string dirs = PCH_OBJECT_DIR;
> > +    std::istringstream idirs(dirs);
> >  
> > -      while (getline(idirs, pchHeaderName, ';')) {
> > -        if(access(pchHeaderName.c_str(), R_OK) == 0) {
> > -          usePCH = true;
> > -          break;
> > -        }
> > +    while (getline(idirs, pchHeaderName, ';')) {
> > +      if(access(pchHeaderName.c_str(), R_OK) == 0) {
> > +        findPCH = true;
> > +        break;
> >        }
> >      }
> > -    if (usePCH) {
> > +
> > +    if (usePCH && findPCH) {
> >        clOpt += " -include-pch ";
> >        clOpt += pchHeaderName;
> >        clOpt += " ";
> >      } else
> >        fwrite(ocl_stdlib_str.c_str(), strlen(ocl_stdlib_str.c_str()), 1, clFile);
> > +
> >      // Write the source to the cl file
> >      fwrite(source, strlen(source), 1, clFile);
> >      fclose(clFile);
> > -- 
> > 1.8.3.2
> > 
> > _______________________________________________
> > Beignet mailing list
> > Beignet at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/beignet
> _______________________________________________
> Beignet mailing list
> Beignet at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/beignet