<HTML>
<HEAD>
<TITLE>Re: [Paraview] PV & multiple video cards</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>I have two comments. First, I think Burlen’s original idea should be implemented before and potentially added independently of the Windows solution. Setting the display screen based on process id is pretty trivial. In my experience, solutions involving OpenGL extensions tend to be harder to implement than you think and fraught with difficulties to support.<BR>
<BR>
Second, I’m disheartened by how labyrinthine it is to control which GPU an OpenGL context uses. I’ll refrain from starting a Windows rant (and for what it’s worth, I understand Mac is just as bad). Instead, I’d like to point out that Kitware has been working with Microsoft’s HPC development to support visualization, and perhaps they could encourage Microsoft to provide a better solution. After all, HPC visualization is not really supported if you cannot make use of every GPU available.<BR>
<BR>
-Ken<BR>
<BR>
<BR>
On 6/22/10 6:49 PM, "luc Renambot" <<a href="renambot@gmail.com">renambot@gmail.com</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>That would be cleanest solution, but Nvidia provides it only for Windows and only for Quadro cards.<BR>
<BR>
Luc<BR>
<BR>
On Jun 22, 2010, at 2:37 PM, Berk Geveci wrote:<BR>
<BR>
> By the way, if we wanted to go all the way on this, we would also<BR>
> support Windows. I was told that there is an nvidia-specific function<BR>
> that controls the GPU a context uses. I believe that it is this:<BR>
><BR>
> <a href="http://developer.download.nvidia.com/opengl/specs/WGL_nv_gpu_affinity.txt">http://developer.download.nvidia.com/opengl/specs/WGL_nv_gpu_affinity.txt</a><BR>
><BR>
> I have no idea if ATI supports this.<BR>
><BR>
> -berk<BR>
><BR>
><BR>
> On Tue, Jun 22, 2010 at 4:16 PM, Moreland, Kenneth <<a href="kmorel@sandia.gov">kmorel@sandia.gov</a>> wrote:<BR>
>> I think this is a good suggestion. I don’t think the fact that it relies on<BR>
>> processes being assigned in a particular order is a big deal. The default<BR>
>> allocation is usually to place contiguous ranks in the same node anyway.<BR>
>> The tiled display options also have heavy reliance on the order in which<BR>
>> processes are assigned, and I don’t recall complaints about that.<BR>
>><BR>
>> The only change I would suggest is to not have an option named<BR>
>> --number-of-gpu-per-host change the display parameter. The option name<BR>
>> should reflect the fact that it is going to override the display environment<BR>
>> variable. Offhand, I cannot think of a single option name that reflects<BR>
>> both the fact that you are changing the display and providing the number of<BR>
>> screen ids to use. Perhaps you would have two options, the<BR>
>> --number-of-gpu-per-host as you described and a something like<BR>
>> --set-display-server-id-by-process-id. Since one does not make sense<BR>
>> without the other, you could give an error if one is used independently.<BR>
>><BR>
>> -Ken<BR>
>><BR>
>><BR>
>> On 6/22/10 12:31 PM, "burlen" <<a href="burlen.loring@gmail.com">burlen.loring@gmail.com</a>> wrote:<BR>
>><BR>
>> Hi,<BR>
>><BR>
>> I realized that my suggestion relies on processes being assigned in a<BR>
>> particular order. So something slightly more complicated would need to<BR>
>> be done to determine the number of processes running on each host. Still<BR>
>> it would be pretty simple way to make folks lives easier. Let me know<BR>
>> what you think.<BR>
>><BR>
>> Burlen<BR>
>><BR>
>> burlen wrote:<BR>
>>> Hi,<BR>
>>><BR>
>>> currently we suggest users rely on details particular to specific MPI<BR>
>>> implementations<BR>
>>><BR>
>>> (<a href="http://paraview.org/Wiki/Setting_up_a_ParaView_Server#Multiple_GPUs_Per_Node">http://paraview.org/Wiki/Setting_up_a_ParaView_Server#Multiple_GPUs_Per_Node</a>)<BR>
>>> to set up PV server on clusters with multiple graphics cards per node.<BR>
>>> It seems to me that this reliance on non standard implementation<BR>
>>> details isn't necessary for the most common case and it can introduce<BR>
>>> some complication to get PV running on various installations.<BR>
>>><BR>
>>> There is something simple that we could do to make users life easier.<BR>
>>> ParaView could handle a common configuration of multiple graphics<BR>
>>> cards seamlessly with very little effort via an additional command<BR>
>>> line option.<BR>
>>><BR>
>>> Two common ways to address multiple video devices in X11 via the<BR>
>>> DISPLAY variable are, 1) by server id (":[server].0") or 2) by screen<BR>
>>> id (":0.[screen]"). It depends on how X11 is setup, but I think the<BR>
>>> latter is the most common.<BR>
>>><BR>
>>> If the user were to tell PV how many video devices are available on<BR>
>>> each node via the following command line variable, PV could assign<BR>
>>> rendering contexts cyclically across the available devices by setting<BR>
>>> the DISPLAY variable for the user.<BR>
>>><BR>
>>> --number-of-gpu-per-host=N<BR>
>>><BR>
>>> as each server starts he will make the computation to set the DISPLAY<BR>
>>> variable so that render contexts are cycled across the available devices.<BR>
>>><BR>
>>> For example, something like the folowing could be added to<BR>
>>> Filters/vtkPVMain.cxx<BR>
>>><BR>
>>> if (numGpuPerHost)<BR>
>>> {<BR>
>>> int screenId=LocalProcId%numGpuPerHost<BR>
>>> setenv("DISPALY=:0.%i",screenId)<BR>
>>> }<BR>
>>><BR>
>>> This would conflict with the "-display" option, if both are present PV<BR>
>>> would exit and print some error message.<BR>
>>><BR>
>>> Do you guys think this would be a reasonable approach to handle a<BR>
>>> common use case?<BR>
>>> Burlen<BR>
>>><BR>
>><BR>
>> _______________________________________________<BR>
>> Powered by www.kitware.com<BR>
>><BR>
>> Visit other Kitware open-source projects at<BR>
>> <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><BR>
>><BR>
>> Please keep messages on-topic and check the ParaView Wiki at:<BR>
>> <a href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a><BR>
>><BR>
>> Follow this link to subscribe/unsubscribe:<BR>
>> <a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><BR>
>><BR>
>><BR>
>><BR>
>><BR>
>> **** Kenneth Moreland<BR>
>> *** Sandia National Laboratories<BR>
>> ***********<BR>
>> *** *** *** email: <a href="kmorel@sandia.gov">kmorel@sandia.gov</a><BR>
>> ** *** ** phone: (505) 844-8919<BR>
>> *** web: <a href="http://www.cs.unm.edu/~kmorel">http://www.cs.unm.edu/~kmorel</a><BR>
>><BR>
>><BR>
>> _______________________________________________<BR>
>> Powered by www.kitware.com<BR>
>><BR>
>> Visit other Kitware open-source projects at<BR>
>> <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><BR>
>><BR>
>> Please keep messages on-topic and check the ParaView Wiki at:<BR>
>> <a href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a><BR>
>><BR>
>> Follow this link to subscribe/unsubscribe:<BR>
>> <a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><BR>
>><BR>
>><BR>
> _______________________________________________<BR>
> Powered by www.kitware.com<BR>
><BR>
> Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><BR>
><BR>
> Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a><BR>
><BR>
> Follow this link to subscribe/unsubscribe:<BR>
> <a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><BR>
<BR>
_______________________________________________<BR>
Powered by www.kitware.com<BR>
<BR>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><BR>
<BR>
Please keep messages on-topic and check the ParaView Wiki at: <a href="http://paraview.org/Wiki/ParaView">http://paraview.org/Wiki/ParaView</a><BR>
<BR>
Follow this link to subscribe/unsubscribe:<BR>
<a href="http://www.paraview.org/mailman/listinfo/paraview">http://www.paraview.org/mailman/listinfo/paraview</a><BR>
<BR>
<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
</SPAN></FONT><FONT SIZE="2"><FONT FACE="Consolas, Courier New, Courier"><SPAN STYLE='font-size:10pt'><BR>
**** Kenneth Moreland<BR>
*** Sandia National Laboratories<BR>
*********** <BR>
*** *** *** email: <a href="kmorel@sandia.gov">kmorel@sandia.gov</a><BR>
** *** ** phone: (505) 844-8919<BR>
*** web: <a href="http://www.cs.unm.edu/~kmorel">http://www.cs.unm.edu/~kmorel</a><BR>
</SPAN></FONT></FONT><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
</SPAN></FONT>
</BODY>
</HTML>