[Beignet] [PATCH] Add the PCH support when building the source.

[He Junyan]

On 07/16/2013 10:54 AM, Zhigang Gong wrote:
> This patch really accelerate the compiling speed dramaticlly. Thanks.
>
> One comment as below:
>
> On Mon, Jul 15, 2013 at 06:22:37PM +0800, junyan.he at inbox.com wrote:
>> From: Junyan He <junyan.he at linux.intel.com>
>>
>> Because the utest grows bigger, the runtime compiling time
>> seems a big problem to cause the utest_run very slow.
>> Most of the time wastes on re-parsing the big header file
>> such as ocl_stdlib.h.
>> Using the PCH feature of Clang can shorten the compiling time
>> a lot.
>> One shortcut: the PCH file in the /tmp/ will not be updated
>> automaticly when you modify the ocl_stdlib.h. You should remove
>> the pch file manually.
> I think this is not convenient for developers and may bring some
> confusing. I have two ideas to mitigate or solve this problem:
>
> 1. precompile the header file at the gbe's build time. Then we will get an up-to-date
> pch file embedded in the gbe library.
Do you mean generate the pch file when make ?
>
> 2. compute a hash value of the ocl_header.h at gbe's build time, and store this hash
> value into the pch file (not very sure whether pch file format can contain any comments
> like data, but at least we still can save this hash value to ocl_header.h.hash file at
> the same location of the ocl_header.h.pch). At the gbe runtime, we just need to compare
> current hash value with the one in the pch file to determine whether we need to update
> a new pch file.
>
> The first solution is the ideal solution. And the second is also not too bad, as
> we only compute the hash value at gbe's build time and it doesn't bring any overhead
> to the application for the hash value computing.
>
> Any thoughts?
>
>> Signed-off-by: Junyan He <junyan.he at linux.intel.com>
>> ---
>>   backend/src/backend/program.cpp |   54 +++++++++++++++++++++++++++++++++++----
>>   1 file changed, 49 insertions(+), 5 deletions(-)
>>
>> diff --git a/backend/src/backend/program.cpp b/backend/src/backend/program.cpp
>> index 2a4feb9..8f1cf61 100644
>> --- a/backend/src/backend/program.cpp
>> +++ b/backend/src/backend/program.cpp
>> @@ -229,18 +229,62 @@ namespace gbe {
>>                                             size_t *errSize)
>>     {
>>       char clStr[L_tmpnam+1], llStr[L_tmpnam+1];
>> +    std::string clOpt;
>> +    const std::string header = "/tmp/ocl_header.h";
>> +    const std::string header_pch = "/tmp/ocl_header.h.pch";
>>       const std::string clName = std::string(tmpnam_r(clStr)) + ".cl"; /* unsafe! */
>>       const std::string llName = std::string(tmpnam_r(llStr)) + ".ll"; /* unsafe! */
>> -
>> -    // Write the source to the cl file
>> +    FILE *clHeader = NULL;
>>       FILE *clFile = fopen(clName.c_str(), "w");
>>       FATAL_IF(clFile == NULL, "Failed to open temporary file");
>> -    fwrite(ocl_common_defines_str.c_str(), strlen(ocl_common_defines_str.c_str()), 1, clFile);
>> -    fwrite(ocl_stdlib_str.c_str(), strlen(ocl_stdlib_str.c_str()), 1, clFile);
>> +
>> +    /* Use the header_pch to save the tokenized file. */
>> +    if (::access(header_pch.c_str(), R_OK)) {
>> +      std::string cmd;
>> +      clHeader = fopen(header.c_str(), "w");
>> +      FATAL_IF(clHeader == NULL, "Failed to open header file");
>> +
>> +      fwrite(ocl_common_defines_str.c_str(), strlen(ocl_common_defines_str.c_str()), 1, clHeader);
>> +      fwrite(ocl_stdlib_str.c_str(), strlen(ocl_stdlib_str.c_str()), 1, clHeader);
>> +      fclose(clHeader);
>> +
>> +      cmd += "clang -cc1 ";
>> +      cmd += "-emit-pch ";
>> +      cmd += header;
>> +      cmd += " ";
>> +      cmd += "-x ";
>> +      cmd +=" cl ";
>> +
>> +#if LLVM_VERSION_MINOR <= 2
>> +      cmd +="-triple ";
>> +      cmd +="nvptx ";
>> +#else
>> +      cmd +="-triple ";
>> +      cmd +="spir ";
>> +#endif /* LLVM_VERSION_MINOR <= 2 */
>> +
>> +#if LLVM_VERSION_MINOR <= 1
>> +      cmd +="-triple ";
>> +      cmd +="ptx32 ";
>> +#else
>> +      cmd += "-ffp-contract=off ";
>> +#endif /* LLVM_VERSION_MINOR <= 1 */
>> +      cmd += "-o ";
>> +      cmd += header_pch;
>> +      ::system(cmd.c_str());
>> +    }
>> +
>> +    // Write the source to the cl file
>>       fwrite(source, strlen(source), 1, clFile);
>>       fclose(clFile);
>>   
>> -    buildModuleFromSource(clName.c_str(), llName.c_str(), options ? options : "");
>> +    clOpt += "-include-pch";
>> +    clOpt += " ";
>> +    clOpt += header_pch;
>> +    clOpt += " ";
>> +    if(options)
>> +      clOpt += options;
>> +    buildModuleFromSource(clName.c_str(), llName.c_str(), clOpt.c_str());
>>       remove(clName.c_str());
>>   
>>       // Now build the program from llvm
>> -- 
>> 1.7.9.5
>>
>> _______________________________________________
>> Beignet mailing list
>> Beignet at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/beignet