[Beignet] [PATCH] Add the PCH support when building the source.

[Zou, Nanhai]

Why not put the header and pch together with the library?


-----Original Message-----
From: He Junyan [mailto:junyan.he at inbox.com] 
Sent: Tuesday, July 16, 2013 12:23 PM
To: Zou, Nanhai
Cc: Zhigang Gong; Junyan He; beignet at lists.freedesktop.org
Subject: Re: [Beignet] [PATCH] Add the PCH support when building the source.

I notice that install the pch to /tmp/ also have some problem.
In some platform the /tmp/ dir is a memory storage and will lost all the content after reboot. This will cause problem.
I think the better way is to do the pch generation at runtime.
I think we can do like this:
Generate some serial number to ocl_stdlib_str.cpp as a comment when build the ocl_stdlib_str.cpp using ocl_stdlib.h. We also keep the tmp header file under /tmp/ which will be used to generate the pch file.
Then every time we compare the serial number to judge whether to generate the pch file again.


On 07/16/2013 12:03 PM, Zou, Nanhai wrote:
> I don't think expose PCH file is not acceptable.
> 1.  It is a standard piece for every kernel.
> 2.  It is platform independent.
>
> Why not just include it in our package?
>
> -----Original Message-----
> From: Zhigang Gong [mailto:zhigang.gong at linux.intel.com]
> Sent: Tuesday, July 16, 2013 12:01 PM
> To: He Junyan
> Cc: Zou, Nanhai; Junyan He; beignet at lists.freedesktop.org
> Subject: Re: [Beignet] [PATCH] Add the PCH support when building the source.
>
> On Tue, Jul 16, 2013 at 11:30:17AM +0800, He Junyan wrote:
>> I think this way maybe OK:
>> We generate the PCH file under beignet's folder when make and install
>> the PCH file to /tmp/ when use make install cmd.
> IMO, no, we don't need to install the PCH file. Just as we generate the ocl_header.h when we build the libgbe. We can call clang to generate the pch file at that time, and store the pch file in an internal array. Then when we need to use it, we just write it to a temporary file and invoke the clang to build the user application.
>
> I don't think there is a need to expose the PCH file. Right?
>
>>
>> On 07/16/2013 11:27 AM, Zou, Nanhai wrote:
>>> Hi,
>>> 	The patch really speed up the compiling.
>>> 	My suggestion is,
>>> 	1. Move ocl_header.h and ocl_header.h.pch into beignet folder
>>> 	2. Put the pch generation logic into cmake so that the build system can check dependency on ocl_header.h and the pch.
>>>
>>> Thanks
>>> Zou Nanhai
>>> -----Original Message-----
>>> From: beignet-bounces+nanhai.zou=intel.com at lists.freedesktop.org
>>> [mailto:beignet-bounces+nanhai.zou=intel.com at lists.freedesktop.org]
>>> On Behalf Of junyan.he at inbox.com
>>> Sent: Monday, July 15, 2013 6:23 PM
>>> To: beignet at lists.freedesktop.org
>>> Cc: Junyan He
>>> Subject: [Beignet] [PATCH] Add the PCH support when building the source.
>>>
>>> From: Junyan He <junyan.he at linux.intel.com>
>>>
>>> Because the utest grows bigger, the runtime compiling time seems a big problem to cause the utest_run very slow.
>>> Most of the time wastes on re-parsing the big header file such as ocl_stdlib.h.
>>> Using the PCH feature of Clang can shorten the compiling time a lot.
>>> One shortcut: the PCH file in the /tmp/ will not be updated automaticly when you modify the ocl_stdlib.h. You should remove the pch file manually.
>>>
>>> Signed-off-by: Junyan He <junyan.he at linux.intel.com>
>>> ---
>>>   backend/src/backend/program.cpp |   54 +++++++++++++++++++++++++++++++++++----
>>>   1 file changed, 49 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/backend/src/backend/program.cpp
>>> b/backend/src/backend/program.cpp index 2a4feb9..8f1cf61 100644
>>> --- a/backend/src/backend/program.cpp
>>> +++ b/backend/src/backend/program.cpp
>>> @@ -229,18 +229,62 @@ namespace gbe {
>>>                                             size_t *errSize)
>>>     {
>>>       char clStr[L_tmpnam+1], llStr[L_tmpnam+1];
>>> +    std::string clOpt;
>>> +    const std::string header = "/tmp/ocl_header.h";
>>> +    const std::string header_pch = "/tmp/ocl_header.h.pch";
>>>       const std::string clName = std::string(tmpnam_r(clStr)) + ".cl"; /* unsafe! */
>>>       const std::string llName = std::string(tmpnam_r(llStr)) +
>>> ".ll"; /* unsafe! */
>>> -
>>> -    // Write the source to the cl file
>>> +    FILE *clHeader = NULL;
>>>       FILE *clFile = fopen(clName.c_str(), "w");
>>>       FATAL_IF(clFile == NULL, "Failed to open temporary file");
>>> -    fwrite(ocl_common_defines_str.c_str(), strlen(ocl_common_defines_str.c_str()), 1, clFile);
>>> -    fwrite(ocl_stdlib_str.c_str(), strlen(ocl_stdlib_str.c_str()), 1, clFile);
>>> +
>>> +    /* Use the header_pch to save the tokenized file. */
>>> +    if (::access(header_pch.c_str(), R_OK)) {
>>> +      std::string cmd;
>>> +      clHeader = fopen(header.c_str(), "w");
>>> +      FATAL_IF(clHeader == NULL, "Failed to open header file");
>>> +
>>> +      fwrite(ocl_common_defines_str.c_str(), strlen(ocl_common_defines_str.c_str()), 1, clHeader);
>>> +      fwrite(ocl_stdlib_str.c_str(), strlen(ocl_stdlib_str.c_str()), 1, clHeader);
>>> +      fclose(clHeader);
>>> +
>>> +      cmd += "clang -cc1 ";
>>> +      cmd += "-emit-pch ";
>>> +      cmd += header;
>>> +      cmd += " ";
>>> +      cmd += "-x ";
>>> +      cmd +=" cl ";
>>> +
>>> +#if LLVM_VERSION_MINOR <= 2
>>> +      cmd +="-triple ";
>>> +      cmd +="nvptx ";
>>> +#else
>>> +      cmd +="-triple ";
>>> +      cmd +="spir ";
>>> +#endif /* LLVM_VERSION_MINOR <= 2 */
>>> +
>>> +#if LLVM_VERSION_MINOR <= 1
>>> +      cmd +="-triple ";
>>> +      cmd +="ptx32 ";
>>> +#else
>>> +      cmd += "-ffp-contract=off ";
>>> +#endif /* LLVM_VERSION_MINOR <= 1 */
>>> +      cmd += "-o ";
>>> +      cmd += header_pch;
>>> +      ::system(cmd.c_str());
>>> +    }
>>> +
>>> +    // Write the source to the cl file
>>>       fwrite(source, strlen(source), 1, clFile);
>>>       fclose(clFile);
>>> -    buildModuleFromSource(clName.c_str(), llName.c_str(), options ? options : "");
>>> +    clOpt += "-include-pch";
>>> +    clOpt += " ";
>>> +    clOpt += header_pch;
>>> +    clOpt += " ";
>>> +    if(options)
>>> +      clOpt += options;
>>> +    buildModuleFromSource(clName.c_str(), llName.c_str(),
>>> + clOpt.c_str());
>>>       remove(clName.c_str());
>>>       // Now build the program from llvm
>>> --
>>> 1.7.9.5
>>>
>>> _______________________________________________
>>> Beignet mailing list
>>> Beignet at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/beignet
>>
>>
>> _______________________________________________
>> Beignet mailing list
>> Beignet at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/beignet