Slow memory access when using OpenCL without X11
Lauri Ehrenpreis
laurioma at gmail.com
Sat Mar 9 06:54:12 UTC 2019
Even if it's using CPU for OCL (I know it's not doing this), why does
memcpy on CPU slow down permanently, if I'm not doing anything with OpenCL
after clCreateContext?
As you see from test program it just does clCreateContext and then a loop
of memcpy-s on CPU.
Also I found out that writing different values to
/sys/class/drm/card0/device/power_dpm_force_performance_level changes my
max memcpy speed on CPU:
echo "low" > /sys/class/drm/card0/device/power_dpm_force_performance_level
./cl_slow_test 1 5
got 1 platforms 1 devices
speed 731.810425 avg 731.810425 mbytes/s
speed 163.425583 avg 447.618011 mbytes/s
speed 123.441612 avg 339.559235 mbytes/s
speed 121.655266 avg 285.083252 mbytes/s
speed 123.806801 avg 252.827972 mbytes/s
echo "high" > /sys/class/drm/card0/device/power_dpm_force_performance_level
./cl_slow_test 1 5
got 1 platforms 1 devices
speed 3742.063721 avg 3742.063721 mbytes/s
speed 836.148987 avg 2289.106445 mbytes/s
speed 189.379166 avg 1589.197266 mbytes/s
speed 189.271393 avg 1239.215820 mbytes/s
speed 188.290451 avg 1029.030762 mbytes/s
echo "profile_standard" >
/sys/class/drm/card0/device/power_dpm_force_performance_level
./cl_slow_test 1 5
got 1 platforms 1 devices
speed 2303.955566 avg 2303.955566 mbytes/s
speed 2298.224121 avg 2301.089844 mbytes/s
speed 2295.585205 avg 2299.254883 mbytes/s
speed 2295.762939 avg 2298.381836 mbytes/s
speed 2288.766602 avg 2296.458740 mbytes/s
echo "profile_peak" >
/sys/class/drm/card0/device/power_dpm_force_performance_level
./cl_slow_test 1 5
got 1 platforms 1 devices
speed 3710.360352 avg 3710.360352 mbytes/s
speed 3713.660400 avg 3712.010254 mbytes/s
speed 3797.630859 avg 3740.550537 mbytes/s
speed 3708.004883 avg 3732.414062 mbytes/s
speed 3796.403076 avg 3745.211914 mbytes/s
However none of those is close to the memcpy speed I get when I don't do
clCreateContext (my test prog has first arg 0):
./cl_slow_test 0 5
speed 7299.201660 avg 7299.201660 mbytes/s
speed 9298.841797 avg 8299.021484 mbytes/s
speed 9360.181641 avg 8652.742188 mbytes/s
speed 9004.759766 avg 8740.746094 mbytes/s
speed 9414.607422 avg 8875.518555 mbytes/s
Also attached clinfo.txt. It shows that opencl is using GPU so device node
permissions are probably not the issue.
--
Lauri
On Fri, Mar 8, 2019 at 10:35 PM Alex Deucher <alexdeucher at gmail.com> wrote:
> I think you are probably using the CPU for OCL in the remote login
> case. When you log into the desktop, the permissions on the device
> nodes get changed dynamically to support accelerated rendering. You
> probably need to change the permissions on the device nodes manually
> if you are not logging into the desktop.
>
> Alex
>
> On Fri, Mar 8, 2019 at 2:43 PM Lauri Ehrenpreis <laurioma at gmail.com>
> wrote:
> >
> > Hi!
> >
> > I am using Ryzen 2400G with Gigabyte AMD B450 AORUS board. I have latest
> bios, ubuntu 18.04 and latest mainline kernel (5.0.0-050000-generic)
> installed. Also I have rocm-dev 2.1.96 but no rock-dkms installed.
> >
> > I found that when I log in over ssh and try to use OpenCL (doing
> clCreateContext is enough) then cpu memory accesses after that will slow
> down by 100x.
> > If I connect HDMI cable and log in to desktop mode then this does not
> happen. Also if I don't call clCreateContext then everything works properly.
> >
> > Attached the test program and kernel log also. Test works like that :
> > g++ cl_slow_test.cpp -o cl_slow_test -I /opt/rocm/opencl/include/ -L
> /opt/rocm/opencl/lib/x86_64/ -lOpenCL
> > lauri at rv:~$ ./cl_slow_test 0 5
> > speed 7003.145508 avg 7003.145508 mbytes/s
> > speed 8427.357422 avg 7715.251465 mbytes/s
> > speed 9203.049805 avg 8211.184570 mbytes/s
> > speed 9845.956055 avg 8619.877930 mbytes/s
> > speed 9882.748047 avg 8872.452148 mbytes/s
> > lauri at rv:~$ ./cl_slow_test 1 5
> > got 1 platforms 1 devices
> > speed 1599.803589 avg 1599.803589 mbytes/s
> > speed 1665.426392 avg 1632.614990 mbytes/s
> > speed 146.137253 avg 1137.122437 mbytes/s
> > speed 121.056877 avg 883.106018 mbytes/s
> > speed 122.428970 avg 730.970581 mbytes/s
> >
> > I also tried latest amd-staging kernel
> https://github.com/M-Bab/linux-kernel-amdgpu-binaries and it had the same
> issue.
> >
> > Can anyone point me into right direction?
> >
> > Br,
> > Lauri
> > _______________________________________________
> > amd-gfx mailing list
> > amd-gfx at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/amd-gfx/attachments/20190309/08f6080f/attachment.html>
-------------- next part --------------
Number of platforms 1
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.
Platform Version OpenCL 2.1 AMD-APP (2814.0)
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
Platform Host timer resolution 1ns
Platform Extensions function suffix AMD
Platform Name AMD Accelerated Parallel Processing
Number of devices 1
Device Name gfx902-xnack
Device Vendor Advanced Micro Devices, Inc.
Device Vendor ID 0x1002
Device Version OpenCL 1.2
Driver Version 2814.0 (HSA1.1,LC)
Device OpenCL C Version OpenCL C 2.0
Device Type GPU
Device Board Name (AMD) AMD Ryzen 5 2400G with Radeon Vega Graphics
Device Topology (AMD) PCI-E, 0b:00.0
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 11
SIMD per compute unit (AMD) 4
SIMD width (AMD) 16
SIMD instruction width (AMD) 1
Max clock frequency 1250MHz
Graphics IP (AMD) 9.2
Device Partition (core)
Max number of sub-devices 11
Supported partition types None
Max work item dimensions 3
Max work item sizes 1024x1024x1024
Max work group size 256
Preferred work group size (AMD) 256
Max work group size (AMD) 1024
Preferred work group size multiple 64
Wavefront width (AMD) 64
Preferred / native vector sizes
char 4 / 4
short 2 / 2
int 1 / 1
long 1 / 1
half 1 / 1 (cl_khr_fp16)
float 1 / 1
double 1 / 1 (cl_khr_fp64)
Half-precision Floating-point support (cl_khr_fp16)
Denormals No
Infinity and NANs No
Round to nearest No
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Single-precision Floating-point support (core)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Correctly-rounded divide and sqrt operations Yes
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 3129047040 (2.914GiB)
Global free memory (AMD) 3055710 (2.914GiB)
Global memory channels (AMD) 2
Global memory banks per channel (AMD) 4
Global memory bank width (AMD) 256 bytes
Error Correction support No
Max memory allocation 2659689984 (2.477GiB)
Unified memory for Host and Device Yes
Minimum alignment for any data type 128 bytes
Alignment of base address 1024 bits (128 bytes)
Global Memory cache type Read/Write
Global Memory cache size 16384 (16KiB)
Global Memory cache line size 64 bytes
Image support Yes
Max number of samplers per kernel 5597
Max size for 1D images from buffer 65536 pixels
Max 1D or 2D image array size 2048 images
Max 2D image size 16384x16384 pixels
Max 3D image size 2048x2048x2048 pixels
Max number of read image args 128
Max number of write image args 8
Local memory type Local
Local memory size 65536 (64KiB)
Local memory syze per CU (AMD) 65536 (64KiB)
Local memory banks (AMD) 32
Max number of constant args 8
Max constant buffer size 2659689984 (2.477GiB)
Preferred constant buffer size (AMD) 16384 (16KiB)
Max size of kernel argument 1024
Queue properties
Out-of-order execution No
Profiling Yes
Prefer user sync for interop Yes
Number of P2P devices (AMD) 0
P2P devices (AMD)
Profiling timer resolution 1ns
Profiling timer offset since Epoch (AMD) 0ns (Thu Jan 1 03:00:00 1970)
Execution capabilities
Run OpenCL kernels Yes
Run native kernels No
Thread trace supported (AMD) No
Number of async queues (AMD) 8
Max real-time compute queues (AMD) 8
Max real-time compute units (AMD) 11
printf() buffer size 4194304 (4MiB)
Built-in kernels
Device Extensions cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_media_ops cl_amd_media_ops2 cl_khr_subgroups cl_khr_depth_images cl_amd_copy_buffer_p2p cl_amd_assembly_program
NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) No platform
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) No platform
clCreateContext(NULL, ...) [default] No platform
clCreateContext(NULL, ...) [other] Success [AMD]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx902-xnack
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx902-xnack
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx902-xnack
More information about the amd-gfx
mailing list