OpenCL 1.0: The Road to Pervasive GPU Computing
by Derek Wilson on December 31, 2008 6:40 PM EST- Posted in
- GPUs
Why NVIDIA Thinks CUDA for C and Brook+ Are Viable Alternatives
While OpenCL is a high level API, it does require the programmer to perform certain tasks that don't have much to do with the parallel algorithm being implemented. OpenCL devices in the system need to be found and set up to properly handle the task at hand. This requires a lot of overhead like creation of a context, device selection, creating the command cue(s), management of buffers for supplying and collecting data on the OpenCL device, and dynamically compiling OpenCL kernels within the program. This is all in addition to writing kernels (data parallel functions) and actually using them in a program that does useful work.
The overhead and management work required is similar to what goes on with OpenGL. This makes sense considering the fact that both use GPUs, they can share data with eachother, and that the same standards body that manages OpenGL is now managing OpenCL. But the fact remains that this type of overhead is cumbersome and can be a real headache for anyone who is more interested in the algorithm. Like scientists working on HPC code who know the theory much better than the programming most of the time.
Both Brook+ and CUDA for C hide the complexity of setting up the hardware by allowing the driver to handle the details. This allows developers to write a kernel, use it, and forget about what's actually going on in the hardware for the most part. Going with something like this as a first move for both NVIDIA and AMD was a good move, as it allows developers to get familiar with the type of programming they will be doing in the future for data parallel problems without tacking higher levels of complexity than necessary.
NVIDIA, for one, believes a language extension as opposed to an API like OpenCL has major benefits and will always have a place in GPU computing (and especially in the HPC space where scientists don't want to be programmers any more than they need to). When asked if they would submit their language to a standards body, NVIDIA said that was highly unlikely as there are other language efforts out there and NVIDIA has been advancing CUDA for C much more rapidly than a standards body would.
On the down side, putting more control in the hands of the developer can result in better, faster code. There is a bit of a "black box" feeling to these solutions: you put code in and get results out, but you can't be sure what goes on in the middle to make it happen. OpenCL gives you better ability to fine tune the software and make sure that exactly what you want to happen happens. Despite NVIDIA's assertions that scientists interested in coding for HPC solutions will have a better experience with CUDA, the cost/benefit of ultra-fine tuning code for HPC machines leans heavily in favor of spending the time and money on optimizing. This means that OpenCL will likely be the choice for performance sensitive HPC applications. CUDA for C and Brook+ will likely have more of a place in just trying out ideas before settling on a final direction.
So there you have it. OpenCL will enable applications in the consumer space to take advantage of data parallel hardware, while Brook+ and CUDA may still have a place in the industry as well (but not on the consumer side of things). That is, until some other more popular standard data parallel language extensions come along and pushes both CUDA for C and Brook+ out of the market.
Facebook
Twitter
RSS
37 Comments
View All Comments
DerekWilson - Wednesday, December 31, 2008 - link
that is not possible -- when using the CPU to process data you need to copy it off the GPU ... when using the GPU to do processing, you need to copy data onto the GPU.What you don't need to do is to worry about copying data from an OpenGL buffer that resides on the GPU to another buffer in order to work on it with OpenCL.
In DX11, you can share buffers between the Pixel Shader and the Compute Shader. Both of these are processed on the graphics card. You can do graphics work and general purpose compute work on the same data ... this is useful for effects physics, visualization of calculations, or complex shaders that might not be possible in the constraints of HLSL.
With OpenGL + OpenCL, you can do the same thing -- share data between graphics buffers and OpenCL buffers. But these buffers reside on the GPU.
In both DX11 and OpenCL, data must be moved off the GPU to process it with the CPU.
If OpenGL and OpenCL did not have binary level buffer compatibility, worst case we would need to copy OpenGL buffers off the GPU, convert them, copy them into OpenCL buffers in the correct format, and then re-upload the data to the GPU. Alternately, we could modify the buffer on the GPU, but that would still require processing power and incur a performance penalty.
Jaybus - Friday, January 2, 2009 - link
I think kevinkreiser was advocating a shared memory architecture, where CPU and GPU could access the same physical RAM, so that there would be no need to copy buffers. However, I disagree with such an approach, because that is only eliminating the buffer copy overhead by forcing the use of a global mutex or some other method of shared memory arbitration. The bottleneck would then become memory contention, offsetting any performance gained by eliminating the copy.Loki726 - Friday, January 2, 2009 - link
PCIe latency is incredibly large compared to memory copy latency. For example, a synchronous copy of a single byte from CPU memory to GPU memory on an 8800GT using CUDA takes around 100k cpu cycles to complete, where non-cached CPU memory copies of the same size are in the order of 100-1000s of cycles. PCIe transfers only become fast when copying large chunks of data.You are right that a shared memory architecture would require synchronization via some mechanism (mutex or other), but this would still be much faster than a DMA copy over PCIe for small data sizes if it was implemented correctly. There is no reason it should be any slower than sharing data between two threads in an SMP.
I think the reason why no one builds systems like this is because low latency access to a shared DRAM would require complex protocols between the GPU and CPU memory controllers to ensure memory consistency and coherence, and no one builds CPUs and GPUs that closely integrated.
DerekWilson - Saturday, January 3, 2009 - link
Some people built / build systems like this -- they are called game consoles ;-)Loki726 - Saturday, January 3, 2009 - link
Good point Derek. The Xbox 360 supports tightly integrated CPU-GPU communication:"The bus design and the CPU L2 provide added support that allows the GPU to read directly from the CPU L2 cache."[1]
[1] Andrews, J. and Baker, N. 2006. Xbox 360 System Architecture. IEEE Micro 26, 2 (Mar. 2006), 25-37
Wwhat - Monday, January 5, 2009 - link
Whatever happened to the hypertransport bus on motherboards and making graphics card for it? That would nicely cover both issues, plus since intel also is going in that direction they might agree with AMD at some far point in the future on a universal direct CPU transport bus connector.Or perhaps the graphicscard makers should consider making a universal socket on their graphicscards that connects to the motherboard to a dedicated connector designed for DMA between a shared memory space with the CPU, a cache designed for shared GPU/CPU use, the advantage would be that people would yet again be forced to buy a new motherboard, and chipset, and that will keep the money rolling in ;]
Personally I think they should sit down in some room alone with themselves and think a bit until they realise having everybody doing their own propriety interfaces and systems is NOT a nice and positive and helpful and even economical way to go about thing and that making a plan then talking in a group with the 'opposition' and then tweaking it before releasing isn't such a bad idea and might actually lead to MORE profit and innovation.
Loki726 - Monday, January 5, 2009 - link
The interface you are thinking of is called HTX and there are some specialized products that use it. Hypertransport may be an open spec, but the memory transfer and coherence protocols used by AMD are not open. So it is not possible for a third party vendor to sit down and implement an HTX card that could work cooperatively with an AMD processor without negotiating a license from AMD. Intel's equivalent Quickpath is similar, but not even an open spec. PCIe is not an open spec either, but is controlled by a consortium that offers third parties pretty much equal opportunities to obtain a license.Someone correct me if I'm wrong, but I'm not sure if dramatically reduced CPU/GPU memory copy latency would be useful for graphics applications. Games seem to scale just fine with the PCIe. Obviously there will be specific cases where it will be useful, but in general, the industry hasn't had a problem getting huge speedups over CPUs without it.