Running VisIt on Comet

VisIt is a free interactive parallel visualization and graphical analysis tool for viewing scientific data on Unix and PC platforms. One of the best ways to get started using VisIt on Comet is in Client-Server mode. Below are step-by-step instructions for:

Top

Setting Comet's Host Profile in VisIt

  1. Check for available versions of VisIt on Comet

    VisIt requires matching versions on the client (your computer) and the server (Comet).
    For example: VisIt 2.9.x matches with VisIt2.9.0 and VisIt 2.9.1, etc.
    Login to Comet and use the list command to identify installed versions of VisIt.

    % ls /opt/visit

    As of this tutorial, the default version of VisIt on Comet is 2.9.1. Make sure that your computer has one of the matching versions of VisIt installed on Comet, if not then download and install a compatible version from the VisIt website.
  2. Open VisIt from the Options menu and choose Host Profiles

    Host Profile
  3. In the Host profiles dialog box:

    Click New on the lower left of the window
    Enter the following in the Host Settings tab

    Host nickname: SDSC_Comet
    Remote host name: comet.sdsc.edu
    Path to VisIt Installation: /opt/visit
    Username: Enter your SDSC user name
    Tunnel data connections through SSH: Check box to enable
    SSH Command: ssh (Check box to enable)
    SSH Port: 22 (Check box to enable)

    Launch Profiles
  4. Change the tab to Launch Profiles in the upper right of the window.
    Create a profile by clicking New in the middle of the window.
    Enter the following in the Settings tab:

    Profile name: normal
    Timeout (minutes): 480
    Number of threads per task: 0

    Launch Profiles

 

  1. Choose the Parallel tab and configure as follows:

    Launch parallel engine Check box to enable
    Parallel launch method: srun
    Partition / Pool / Queue: compute
    Default number of processors: 24
    Default number of nodes: 1
    Default Bank / Account: YOUR ALLOCATION ACCOUNT
    Default Time Limit: 00:30:00 (set this as per your needs)

    Launch Profiles - Parallel
  2. Click Apply on the lower left of the window and then click Dismiss.

    Caution: Your settings are not yet saved so the next step is very important and easily missed by many users.
  3. From the Options menu, choose Save Settings. If you miss this step, your host profile will not be saved!

    Save Settings

Top

Connect to Comet from the VisIt application on your computer

  1. Connect to Comet from the VisIt client on your computer.

    Start the VisIt application on your computer.
    In the VisIt main window under Sources click the Open button.

    Note: In VisIt Open performs a function to connect to a remote resource as well.

    Open
  2. Select the drop down list of hosts at the top.

    In the Host field, use drop down button on right and choose SDSC Comet.
    You may get a prompt to allow VisIt to connect to Comet.

    Select Host
  3. If you are asked for your password, enter it and click OK. Once connected to Comet you should be able to see a listing of your home folder on Comet similar to the following figure.

    Now you can browse and open the desired files for your visualization.

 Top

Visualize an Example Data in VisIt

  1. First, be sure you are connected to Comet as described above.

    Now load a test data as follows:

    Change Path to /opt/visit/data .
    Hit Enter Key to load content in the above folder.
    In the right file list panel choose noise.silo .
    Click OK button on bottom right.

    Select File
  2. Under the Plots section in the main VisIt window:

    Choose Pseudocolor > hardyglobal

    Pseudocolor
  3. Click Draw in the Plots section.

    Draw
  4. In the prompted dialog window change the following as needed:

    Num nodes to 1
    Num 
    Time limit to 10 minutes
    Bank to your allocation account

Wait

Click OK

  1. You will get another window showing that the job has been submitted and it is waiting for the job to run and start.

    If the machine has nodes available your job should start in a short time.
    If your job does not run then login in to Comet using ssh and check the wait time for your job.

    % qstat -u YOUR_USER_NAME

    Wait
  2. Once the job runs you should see an image as shown below.

    Now you are all set to start visualizing your data using VisIt on Comet.

    Plot

 Top

Running VisIt in Batch Mode on Comet

The VisIt application relies on complex integration with job scheduler and several dependent libraries making these instructions not easily transferable to other XSEDE resources. This section provides a simple way to run VisIt in batch mode on Comet.

  1. Login to Comet
  2. Download the batch scripts archive and unzip it in your home directory:
    % cd ~/
    % wget http://www.sdsc.edu/us/visservices/downloads/software/visit/batch_visit.zip
    % unzip batch_visit.zip
    % cd batch_visit
  3. Render Script: Examine the render.py script. This python script will perform the following tasks:
    1. Start parallel compute engines on specified nodes and cores
    2. Open a multi_rect3d data file
    3. Create a volume-rendering plot for variable d using raycasting
    4. Save the rendered image to batch_visit directory
    5. Delete the plot, close the file, close the compute engine and then terminate
  4. Job Script: Examine the submit.sh script. This job submission shell script requests nodes and cores for rendering and then runs the render.py script. Note that the following parameters may need to be modified:
    1. The script currently requests 2 nodes with 16 cores for 5 minutes. This may need to be increased for more computationally intensive visualization tasks.
    2. Change your allocation account in the line #PBS -A gue998. If you are unsure of which accounts you are authorized to use, run the show_accounts command.
  5. Test Run: Submit the job script using qsub on Comet

    % qsub submit.sh

    Once the job completes, you should see myBatchRender0000.png and myBatch_render* log files in your batch_visit directory.


    Volume rendered image for variable d

 Top

Benchmark VisIt Rendering Performance on Comet for Identifying Best Configuration

Often it is necessary to identify best node to core ratio for visualizing large data. This section provides instruction to determine best performing configuration for rendering with VisIt on Comet.

  1. Login to Comet
  2. Download the benchmark scripts archive and unzip it in your home directory:
    % cd ~/
    % wget http://www.sdsc.edu/us/visservices/downloads/software/visit/bench_visit.zip
    % unzip bench_visit.zip
    % cd bench_visit
  3. Render Script: Examine the render_bench.py script. This python script will perform the following tasks:
    1. Start parallel compute engines on specified nodes and cores
    2. Open a multi_rect3d data file
    3. Create a volume-rendering plot for variable d using raycasting
    4. Save the rendered image to bench_visit directory
    5. Measure the total rendering time for a given node and core configuration
    6. Delete the plot, close the file, close the compute engine and then terminate
  4. Launch Script: Examine the launch_bench.sh script. This shell script will automatically create and submit several batch jobs requesting different numbers of nodes and cores for benchmark rendering. Note that the following parameters may need to be modified:
    1. Change your allocation account in line 13: myAccount=PROJECT. If you are unsure of which accounts you are authorized to use, run the show_accounts command.
    2. The number of nodes in line 7 may need to be increased for very large visualization problems.
  5. Test Run:
    1. Change permissions and execute the launch_bench.sh script.
      % chmod a+x launch_bench.sh
      % ./launch_bench.sh
    2. A script named myBench_setup will be created in bench_visit directory
    3. Finally, the myBench_setup script is automatically submitted for different configurations in a way such that only one job runs at a time to avoid data file read contention.
    4. Once the job completes you should see several images named myBenchRender*.png and myBench_nodes*cores* log files in the bench_visit directory.
    5. Examine the render time consumed by each configuration
      % tail -n 3 myBench_nodes*.o*
    6. While the test case chosen here is trivial, this method could be very useful with large data sets.

 Top

If you are unable to connect to Comet through VisIt, check the following:

  1. Make sure that the VisIt application version on your computer is compatible with a VisIt version available on Comet.
    Check versions of VisIt installed on Comet by using the list command:

    % ls /opt/visit

    For example: VisIt2.3.X version will work with VisIt2.3.0 and VisIt2.3.1 and VisIt2.3.2
  2. Check that the Path to Visit Installation is set correctly in your Comet Host profile.
    Refer to how to create a Host Profile for Comet documentation. 
  3. Ensure that your computer or network firewall is not blocking the VisIt server to connect back to the VisIt client on your computer. 
  4. In some cases erasing the local and remote visit settings can help resolve issues.
    Note: You will lose all your Host Profiles and will need to recreate them.
    Delete the ".visit" folder on your computer's home or "My Documents" folder.
    Delete the ".visit" folder on Comet in your home folder.

    % rm -rf ~/.visit
      
  5. If the Metadata Server repeatedly crashes, try enabling SSH tunneling in the host profile.

For more information refer to VisIt manuals and tutorials for further guidance.