+ All Categories
Home > Documents > Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015....

Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015....

Date post: 01-Oct-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
23
@
Transcript
Page 1: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

@

Page 2: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

Table of Contents 1. Render farm introduction

A. What is a render farm B. What a render farm will not do C. Example of render farm performance

2. SVA’s render farm A. Basic design and functionality of SVA’s render farm B. Workers, slots, and instances C. Render power on SVA’s render farm

3. Render policy and etiquette A. What is supported on the render farm B. Why there are rules for the render farm C. How renders are wrangled

4. Submitting your job to the render farm A. Schedule a render test B. Upload your files to network C. Test your scene

i. How to reliably test render times D. Clean your scene E. Qube! operations and submission F. Understanding reservations G. How to monitor jobs in Qube! H. Cache your scene I. General guidelines for scene submission J. General guidelines for Mental Ray K. General guidelines for V­Ray L. General guidelines for Arnold

5. Common errors and how to troubleshoot them 6. Appendix and resources

Page 3: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

1. Render farm introduction A. What is a render farm? A render farm is group of computers that work together to render scenes (or frame sequences) at a faster rate than using a single workstation. It is designed to distribute computationally complex tasks, such as 3D renders, and spread them across multiple computers. Basically, the render farm sends your render job to multiple computers so that multiple frames can be rendered simultaneously. B. What a render farm will not do:

A render farm will not fix scene errors. In fact, it will magnify errors. Make sure your scene is optimized and checked for errors before submitting to the render farm.

A render farm will not speed up the render time for a single frame. In fact, a single workstation may render a single frame faster than using the render farm. The benefit of using the render farm is that multiple frames are rendered at the same time—drastically reducing the amount of time it takes to render a scene or frame sequence.

C. Example of render farm performance

Scene Size Single Workstation Render Time (16 threads per frame)

Render Farm Render Time (1 thread per frame/20 slots)

Time Savings Using the Render Farm

1 Frame 3 minutes 10 minutes ­7 minutes

2 Frames 6 minutes 10 minutes ­4 minutes

3 Frames 9 minutes 10 minutes ­1 minute

4 Frames 12 minutes 10 minutes 2 minutes

20 Frames 1 hour 10 minutes 50 minutes

200 Frames 10 hours 1 hour 40 minutes 8 hours 20 minutes

2160 Frames (total frames used in average 90 sec.thesis)

108 hours 18 hours 90 hours

By leveraging the power of distributed computing, the render farm outperforms the single workstation exponentially. In the job illustrated in the above chart, it takes 3 minutes/frame to render on a local workstation and 10 minutes/frame to render on the render farm. However, the render farm can render 20 frames simultaneously every ten minutes, accounting for the

Page 4: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

performance increase. Using this example, the render farm saves over 90 hours in render time for an entire thesis project. *Your Performance May Vary.

2. SVA’s render farm A. Basic design and functionality of SVA’s render farm The render farm at SVA is called Qube!!. For the sake of simplicity, SVA refers to their render farm as Qube!! because it shares the name of the render farm management software we use. Qube!! is based on the client/server model and consists of four components:

1. Supervisor: the server that controls job submissions and how resources are allocated for each job.

2. Client: your local workstation where you submit jobs and monitor status. 3. Workers: also known as nodes, these are the computers that actually perform the

render computations. The dedicated core farm workers are located in the 3rd floor render closet. In addition to the dedicated core farm, all of the computing power from the workstations in computer labs can be harnessed to increase the overall power of the farm.

4. Network File Server (\\genesis\bfacad\shotgun\): where your files are stored when you are ready to render. Every component of your project, from project file to texture file, must be copied to \\genesis\bfacad\shotgun\ before render. Completed renders are also copied to \\genesis\bfacad\shotgun\

B. Workers, slots, and instances In the Qube!! render farm, workers perform all of the processing needed to render a scene. The workers’ computational resources are broken down at a more granular level. It is helpful

Page 5: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

to know these terms to gain a full understanding of the processing available for your job. Workers: Also known as nodes, these are the physical computers or CPUs available to

you for rendering. Most workers contain multiple processors with multiple threads. The higher the amount of threads allow the computer to render more frames simultaneously.

Slots: the amount of threads used to process a job. Instances: the amount of frames you request to render simultaneously. Each instance

is rendered using a slot. C. Render power on SVA’s render farm To calculate render capability on Qube!!, you can use this equation: Workers x Threads per Worker = Slots = Instances = Amount of Frames Rendered Simultaneously Core Render Farm Workers (3rd Fl Network Closet)

Number of Workers Number of Threads/Worker (avg)

Total Slots

20 20 392

Using only our core farm, a single student could theoretically render 320 frames simultaneously at one thread per frame. Or, in a more likely scenario, 16 students could all render 20 frames simultaneously at one thread per frame. Extended Render Farm Workers (Lab Computers)

Number of Workers Number of Threads/Worker (avg)

Total Slots

80 16 1280

Utilizing lab computers dramatically increases the amount of available workers. However, these workers are not dedicated to just rendering. These computers are only available overnight or in the rare occasions the lab is not being used for a class. Burst Render Farm Workers (Lab Computers­March Only)

Number of Workers Number of Threads/Worker (avg)

Total Slots

100 12 1200

The burst render farm workers are an extension of the extended render farm workers in that it uses additional lab computers overnight or in the rare occasions the lab is not being used for a class. These are only in use for one month during peak production. As a student submitting jobs, you should not notice any difference on whether your jobs are

Page 6: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

being processed by the core workers, the extended workers, or the burst workers. You will notice that the amount of slots on Qube!! may expand and contract as the extended and burst workers become available. Independent of available slots, it is important to remember that the time to render per frame depends on the complexity of the scene and how well the scene is optimized. The render farm will not make your render time per frame any faster, it will just allow you to render more frames at the same time.

3. Render policy and etiquette A. What is supported on the render farm

Supported 3D software Maya

Supported render engines Arnold VRay Mental Ray

Supported render time/size limitations Limit of 20 minutes per frame (using a 2­thread render with all passes included)

8 GB of RAM per scene Limit of 720p frame size (1280x720) 2K limit for texture files at 72dpi

Other considerations You must perform a test render in the fall semester before you are authorized to use the render farm.

Only save your Maya files as .MA You must use assigned storage

\\genesis\bfacad\shotgun\). You must login as yourself. Don’t

submit jobs under others’ names. Geometry /Alembic caching is

encouraged to speed up render times.

Local rendering is strongly discouraged.

BACKUP YOUR DATA! Rule of thumb: one backup = no backup.

Do not use live sync for cloud storage services such as Dropbox or Google Drive

Page 7: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

Any jobs that do not meet the above criteria may be paused, killed and/or the user may be banned from using the render farm. Rendering jobs that fall outside of these guidelines are at the discretion of the BFA Computer Art System Administrators. B. Why there are rules for the render farm While SVA’s Qube!! render farm is very powerful, it does have limitations—especially when you consider how many frames need to move through the farm to process all of the thesis projects. Since Qube!! is a finite resource, it is important to abide by the rules and limitations of the render farm. Using the render farm, we have approximately three months to render what it would take a single workstation 4 years to render:

An average thesis is approximately 90 seconds in duration. There are 24 frames per second. There are approximately 50 thesis projects. 90 x 24 x 50 = 108,000 frames that need to be rendered through the farm. Taking into

account re­renders, this number can easily double to 216,000 frames. The bottom line is that there are many jobs going through the farm at the same time. It is important to follow policy to ensure that everyone has a chance to get their thesis project rendered. C. How renders are wrangled Because rendering 3D scenes is a complicated process, your jobs are “wrangled” on the render farm. Essentially this means that a system administrator or render wrangler checks for errors and manages render priorities. When you submit jobs on the farm, they are not necessarily processed in the order they are submitted. Our goal is to get jobs in and out of the render farm as quickly as possible. In general, render priority is given using the following criteria:

1. Render speed (40%)­how long each frame takes to render. A job that renders 2 min/frame on the farm will get higher priority than a job that is rendering 20 min/frame on the farm.

2. Job size (30%)­the amount of frames in a job. A job of 30 frames will be given higher priority than job of 500 frames.

3. Thesis progress (20%)­how close the thesis project is to being completed. A job that this is the last shot of a thesis project will receive a higher priority than a job that is a

Page 8: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

first shot of a thesis project. 4. Submission order (10%)­when a job is submitted to the render farm. Earlier jobs go

first.

4.Submitting your job to the render farm A. Schedule a render test Before you use the render farm, you have to schedule a render test with Richard Hagen. Schedule a test in person or by emailing Richard at [email protected]. These render tests are typically done in the first week of December but you can schedule one earlier if you wish. You should have a scene ready that is representative of a scene in your thesis including some sample textures and a basic lighting scheme. We will go over how to submit a scene to Qube! during this test. B. Upload files to the network In order to render on a network, all your files must be on centralized network storage. You will need to create a master folder (named after your project or your name) on \\genesis\bfacad\shotgun\ share and then copy all the assets you will need for rendering into that folder. If you have been using Shotgun, please speak with an administrator at this step so that we can have Shotgun reate the appropriate folder structure for you. Since Shotgun has an excellent pipeline integration toolkit, you should take advantage of what they offer. The shotgun integration tools take care of a lot of the asset management that this document will go into. Another excellent software to use is Open Pipeline, if you do not wish to use the toolkit from Shotgun. However, please note that the two pipeline tools are not compatible and Open Pipeline will not track your usage on the shotgun website ­ you will then have to manually track all of your progress in Shotgun. C. Test your scene Once the files are uploaded, you should then Set Project to the

Page 9: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

\\genesis\bfacad\shotgun\ share folder and load your scene. If you are using Shotgun Integration, this step is done for you ­ all you have to do is load the project through Shotgun instead of the file>open menu. Check for missing textures, cache files or other assets. Do test renders to make sure everything works. If it doesn’t render locally, it isn’t going to work on the farm either.

How to reliably test render times Try sending a render to your own station via Qube!. You do this by submitting a

job to Qube! as normal, but then going to the “Qube! Worker Selection” and selecting your machine from the list. This will make Qube! send the render to your computer only. You can also test a scene by going to your actual file and right­clicking “render” on the file. You should then see command­line output. Time this process with a timer or by looking at file­creation timestamps.

D. Clean your scene Now that you have your scene working, it is time to optimize your scene. Maya’s render.exe and batchrender.exe programs (which Qube! uses to render with) do NOT like Unknown or Unused nodes anywhere in your scene. A good method to properly clean a scene is to first IMPORT ALL REFERENCES. While the render farm supports referencing, if your references are unoptimized, the render will likely fail. It is easier to import everything then clean the scene, then to go to all your references individually and clean them one at a time. After your references are imported, go to your Hypershade and Delete Unused Nodes. After that, clean your scene using File > Optimize Scene Size. Be warned that the Optimize Scene Size function can destroy your scene if you are using customized data structures ­ if this happens simply undo the operation and go through each Optimize function until you eliminate the one that breaks your scene. Note that using caching technologies such as Alembic or Autodesk Geocache will also greatly optimize

Page 10: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

your scene. Delete anything in your scene that is not contributing to the render. The farm does not support live dynamics, remove them or cache them. The farm will fail with scenes that have references that are themselves referenced (i.e. do NOT build a renderable scene with more than ONE level of referencing!). Beware of overly long (256 character) namespaces. No Spaces in Filenames or file paths! E. Qube! operation and submission 1) Initialize the Maya Plugin (has to be done only once per login per machine) a)Load Qube!. b)Run the AppUI installer: File > Install App UI > Install Maya App UI. If you do not see the Qube! menu inside Maya, close and relaunch Maya after you do this.

You should see this in all versions of Maya once complete:

2) Submitting your render job a) Load your scene from\\genesis\bfacad\shotgun\..etc..

Page 11: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

b) Check your scene as detailed above (check for textures, scene optimize, etc) c) Go to the Qube! menu in Maya and select Submit Render Job. Don’t select Submit Batch Render unless you require the manual, granular control that mode offers. The next window will appear. There are lots of options; listed are the default settings:

Name: the name will default to your scene file name. Please add what renderer you are using to the name. Priority: Leave at the default of 5003 Instances: Please use a default of 20. Render wranglers will adjust it accordingly. Range: This should be automatically filled out from your Render Globals. You can change it to the frame range you need. You can use a span (i.e. 1­100) or specify frame numbers with commas (1,2,5,10,100). Use Preview Frames: If you would like certain frames to render first, click this checkbox and fill out which frames Qube! should render first. i.e. ­ your range is 1­100, but you want Qube! to render frame 52 first.

Page 12: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

Skip the section “Render App Auto Threads & Uses the Entire Worker” and go to “Render App Uses a Specific Number of Threads” Slots = Threads ­ click this box Specific Thread Count: type in a default of 2. If you want to use 3 or 4 threads, speak with a Render Wrangler first.

Scene File: This should be automatically populated by the submission script. Please check to see that it starts with \\genesis\bfacad\shotgun\ and doesn’t start with anything else such as c:\myfolder or x:\myfolder Project Dir: This should be automatically populated by the submission script. Please check to see that it starts with \\genesis\bfacad\shotgun\ as mentioned above. Cameras: Please select a renderable camera here. Only select one. If you have more than one camera in the scene, you will have to submit a separate render job for each camera. Layers: Leave blank to use settings from your Render Layers settings. Click on Browse and then select layers to override the scene settings. Enable Default Light: Leave blank.

Renderer: Select your renderer here. This forces Maya to render with the proper renderer and also allows Render Wranglers to know what you are using.

Page 13: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

Render Threads: Leave at ­1 Ignore Errors: Check if you are using Mental Ray. Vray and Arnold users should leave this unchecked unless told otherwise. Distributed Rendering: leave blank. If you need to use VRay DR, speak with Richard or a Render Wrangler and we will plan out how to set up a DR session through Maya. Maya Executable: leave blank. Render Directory: Please type in the path to where you are rendering your files. Frame Padding: You can enter 3 or 4 here if you want to override the Render Globals setting.

Please skip to the Qube! Worker Selection portion of the submission window. Hosts & Omit Hosts: Clicking “Browse” on Hosts and clicking rendernodes will only render to those hosts that you have selected. Clicking “Browse” on Omit Hosts will ban those specific hosts from rendering your job. Use these if you find certain computers do not render your project successfully. Be sure not to omit too many computers, else your job will not render. Reservations: Click Browse and you will see this window:

If not present, type “2” into host.processors. Type “4000” into the host.memory field as a good default. Type in a HIGHER host.memory if you know that your job

Page 14: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

needs more. If your job needs 8GB, type in “8000”, etc. You can check on how much RAM your scene needs by checking the Task Manager in Windows (press CTRL+SHIFT+ESC) and find the Maya process. This step is very important, pleae don’t skip it and please be honest about how much memory your scene takes when loaded. If you need help with this step, ask Richard. Flags: Please press “Browse” and check the following:

auto_mount, auto_wrangling, disable_windows_job_object, migrate_on_frame_retry Press Submit! Whew! To save on doing this every single time, you can save these settings as a default by pressing “Set Defaults” before you submit. It will be saved to your local user profile on the computer you are sitting at and will load up with the same settings. You can also pre­create a filled out submission form and press the little blue Floppy Disk Icon to save an .xja file which can be resubmitted later by using the Qube! menu Submit> Job From File. F. Understanding reservations The reservations setting is very important: Without setting this properly, the renderers will fight for resources and crash. The “host.processors” number is the amount of threads that each frame will use. The “host.memory” number tells the computer to set aside a chunk of RAM for the scene to load and for the render framebuffer. These

Page 15: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

numbers also act as a filter to automatically eliminate any render nodes that do not have enough resources to fulfill the rendering task. If you do not set those numbers properly, your render might get posted to a machine that does not have enough resources and crash the machine. G. How to monitor jobs in Qube Once your job is submitted, you can track its progress by watching the time graph for the selected job. You should also follow the status of the rendering via the alphabetical shorthand listed under the “State Frames” column. H. Cache your scenes We highly recommend that you use the caching technologies that are included with Maya. The two major types are Autodesk Geocache and Alembic Cache. Used properly, either of these caching methods will help to cut down on scene errors, simplify your scene and will also speed up both your viewport and rendering performance. Network rendering works better when you give the renderer less to process. Live rigs, histories and references should be eliminated from the scenes that you wish to submit to render; caching is a way to streamline this process. I. General guidelines for scene submission

Optimize your scenes regularly. If you’ve optimized it, but then worked on it again… optimize it again. Caching helps with this.

Simplify your scene however you can. Caching helps a lot with this. If there is something in your scene that is NOT contributing to a render

(such as Hidden layers / geometry) then please DELETE IT. Hiding or setting assets to non­renderable does nothing for optimization. If you don’t need it in the render, get rid of it for that scene.

Save revisions of your scene. Save a “render” version that’s been cached, baked and optimized, and a “live” version for continued editing.

Check your RAM levels when the scene has been loaded and with a rendered frame in the framebuffer.

No live dynamics. Cache / bake everything that is animated with a dynamics solution.

Make sure your scene can load & render without having to load it in via openPipeline. OpenPipeline is a great tool, but you have to be sure that the scene can load without it being resident. Before you send a scene

Page 16: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

to render make sure it can load in via Maya menu: file>open. Did you set proper render settings? Check the render directory,

filetype, frame padding, renderable camera, resolution and renderable layers.

No renders larger than 1280x720. Higher just wastes your time because we will crunch everything to 720p before the screenings anyway.

No spaces in folder names or file names!

Each shot is unique. Some shots need different renderer settings. What works great for one shot may not necessarily work for another shot. Don’t just “set it and forget it”.

Don’t set reflections to be too deep. Don’t make mirrors that reflect mirrors that refract through glass that reflect off mirrors, etc.

Don’t reference into references into references. You will end up creating an unrenderable and unrepairable scene if you do this.

Render Passes / Render Elements / AOV’s are NOT “free” to render. Use wisely! Don’t just use them all for the sake of having them “just in case”

J. General guidelines for Mental Ray

Use Maya’s built in Render Diagnostics (only works with Mental Ray) before you send to the farm

Use ray tracing and not scanline mode. Don’t forget to set the Quality > Framebuffer to either RGBA Float or

RGBA Half.

K. General guidelines for V­Ray Check your default subdivision levels. Never set higher than 10. The

default value for subdivisions in V­ray is 256 which is way too high.

If you activate subdivisions on a per­object basis, the SAME “256” number appears under “Max Subdivs” on each mesh. If you are tweaking the scene with per­object subdivision, then BE SURE to take

Page 17: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

Level of Detail into account. Set small / distant objects to “1”.

V­Ray’s final gather and monte carlo “brute force” radiosity are specific to V­Ray. Sometimes a brute force render in Vray can be faster and cleaner than an interpolated render. Test, test, test.

Watch out with the “Render Viewport Subdivision” checkbox.

If you check this box, Vray will render Maya’s subdivision settings ON TOP OF the Vray subdivision settings. This can make your RAM requirements and render times to skyrocket. Usually, people will use this if they only want Maya’s subdivisions and turn all the Vray subdivs off.

Set Vray’s Max Render Threads to “2” before sending it to the farm.

Try V­Ray’s lightcache mode. For scenes that are taking a lot of time to render, this can slash render times by a large margin. Unlike Mental Ray, V­Ray can EASILY cache animated lighting. The only consideration is that you have to pre­render the lightcaches. Ask Richard for a Distributed Render setup to speed this along.

Page 18: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

Render to VRIMG files rather than to EXR. VRIMAGE files work far more reliably than EXRs when rendering through V­Ray. You have to convert them later with the command­line program “vrimg2exr” which is easily setup through Deadline.

If your scene is failing, try using the .VRSCENE save in the render settings. We will have to use Deadline to render this type of file. This filetype strips out unneeded elements from your scene and greatly stabilizes a render job.

L. General guidelines for Arnold

Arnold is a raytraced “brute­force” monte carlo radiosity renderer. V­Ray and Mental Ray use approximation methods like Final Gather and Global Illumination which are not present in Arnold.

Use Arnold’s subdivision settings. Look in the object attributes for

“Arnold” and then tweak “iterations”. Don’t crank this up past 4. These settings have a direct link to render times and RAM usage. Too high a setting will crash the renderfarm.

For large texture maps, convert to Arnold’s .tx format. Doing that will

make Arnold work more efficiently.

Use only the built­in Arnold shaders. If you’ve found a third­party shader, we can’t support it at this time.

Page 19: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

The Sampling and Ray Depth settings are the core of Arnold and are directly related to render times. Use them wisely and carefully. Optimizing these settings on a shot­by­shot basis will greatly improve your efficiency. Start all your shots at value 1 for all sampling settings and only increase them slowly, one by one with a test render for each iteration. Increase AA samples before the other samples. If you can get a good image with low settings, use low settings. Don’t increase settings unless you absolutely need to.

Using lots of AOV’s eats up RAM. Being frugal here will pay off in

shorter renders. Select what you will actually USE. IN Arnold, selecting “tiled” for EXRs will help with RAM usage. If your renders are failing with the regular Maya Job submission, we will

have to use the Arnold .ass/kick mode for rendering. We will use Deadline for this. Using .ass / kick is in fact the preferred method of rendering Arnold scenes in all studios, so don’t be afraid of using it.

5. Common errors and how to troubleshoot them

My Job didn’t submit! Check to see if you are logged in as yourself. Are you submitting from a Mac? You have to setup the submission

script manually, adhering to Windows file paths.

Page 20: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

Are you submitting with the proper settings as outlined above?

My render is pending! Most likely, the farm is at capacity. The amount of cores to render on

fluctuates throughout the day. When classes are in full swing and every computer is being used, there is much less capacity to go around and you will have to wait longer. This is why optimizing your scenes and getting fast render times is a key concept.

Your RAM requirements might be too high. If your job needs 16GB to load or render, many machines simply will not accept the job.

There might be something wrong with the scene that is causing it to hang upon load. Speak to Richard to see how to begin troubleshooting a bad scene like this.

Are you using spaces in your folder or filenames? You shouldn’t be. Beware of corrupted maya scene files! Only save in .MA (ascii)

format!

Qube! is failing my renders! The most common cause of failed frames are missing assets. Check

all your paths for your assets to make sure they are relative and not pointing to a letter drive like C:\ or X:\

Do you have unknown or unused nodes? Go to hypershade > edit > delete unused nodes. Then go to file > optimize scene size.

Sometimes a frame will render fine but because the word “error” pops up, Qube! might fail a frame. Check to see if the frame rendered alright. Even if one render element or AOV fails, Qube! will mark the whole frame as “failed”.

Check your scene path and project path in the submission script. Do you have live dynamics in the scene? Is your scene crashing the render node it is rendering on? Check the

log files and look for “Exception Caught ­ Did Maya Crash Earlier” ­ this means that the command­line version of Maya is dying when it tries to render your scene.

Remember: Qube! cannot fix errors in your scene. It will, instead, make any errors WORSE.

My job reads “KILLED” but I didn’t kill it!

Did you submit your job as yourself? If so, then a render wrangler killed the job. Usually, we will kill a job because it isn’t rendering well or if

Page 21: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

it is causing problems on the farm. Many times we will resubmit the job to re­render with better settings, so check to see if your job has been resubmitted. If you didn’t submit a job as yourself, then whoever’s account “owns” the job could have killed it.

Qube! didn’t render / save the frames! Check to see that you have selected to render an animation. An easy

error to make is selecting “name.ext (single frame)” and not “name.#.ext”. Check to see that you told Qube! to render the right sequence of

frames. Check to see that your render paths are correct. You might be victim of crashing nodes, in which the render engine

crashes, but the Qube! worker thinks that it has completed the job. They are very easy to spot by looking at the render times for the missing frames (look for very low render times) Sometimes a render node itself is to blame and not the scene file, in which case you would Omit Host upon re­rendering the bad frames.

Use the Qube! render logs to see where Qube! might have put frames. Sometimes, when there is heavy network congestion, Qube! will put frames on the local rendernode or outside of the folder you told it to go.

My renders don’t work at all!

Talk to an administrator or render wrangler and we will look through your scene to see how best to address the problem.

6. Appendix and Resources Deadline

Typically, seniors will use Qube, but for special needs cases, we will transfer your job to Deadline. Such cases include converting VRIMAGE files to EXR files, rendering Vray .vrscene files, rendering Arnold .ass files, rendering Mantra scenes and rendering posters using tile mode.

Backing up Files

This thesis represents a year of your life. Treat the data accordingly. Back it up to as many devices as you can as often as you can. You will use the storage on \\genesis or a terrablock for when you work here, but do not use school storage as your only source of data backup. You MUST have your own storage. The IT rule of thumb, ”3 copies of anything

Page 22: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

you care about - Two isn't enough if it's important.”. http://www.hanselman.com/blog/TheComputerBackupRuleOfThree.aspx

Remember that your work is stored as magnetic (or in the case of flash, electro­chemical) bits. If the machine holding those bits fails, the bits vanish as if they never existed. Backups are your lifeblood.

How To Figure Out Memory

Load your scene and render a frame. Right­click on the taskbar and select Start Task Manager (or press ctrl+shift+esc) to get to the Task Manager. Find the Maya.exe process. Read the amount of memory being used.

Round that number off to the nearest Gigabyte of RAM. In this example, we see that the Maya scene with a rendered framebuffer takes up 627MB of RAM. I would type in 1000 ~ 2000 into the host.memory section of the reservation tab in the Qube! submit window.

Qube! GUI filter

The Qube! GUI has a feature to filter out only the jobs you have submitted. You access this by pressing the “User” button at the top. When in doubt, unfilter the job list results to see if your job was resubmitted by an administrator.

Render Scheduling

Please schedule a meeting with Richard Hagen between December 1st to December 5th, 2014 in order to get permission to use the render farm. You have to furnish a scene to be tested on the farm. It should be something from your thesis, and should have animation, lighting and surfacing so it can be properly tested.

Final Thought

There are no prizes for longest render. There’s no pride in having bloated scenes that eat loads of RAM. There is no joy in waiting a half hour for your scene to load because you have forgotten to optimize your scene. Time is of the essence! Don’t forget you have to do sound for your

Page 23: Table of Contents - SVA Computer Artsvacomputerart.com/images/Renderfarm_Primer-2015.pdf · 2015. 1. 12. · A render farm will not speed up the render time for a single frame. In

piece. Don’t forget you have to do compositing, touch up, color grading and editing to your short. Those take a lot of time too. You also want to give yourself enough time to fix errors, tweak the shots and hone the overall look beyond a first pass result. Rendering is just the first step of a long finishing process. Give yourself a firm deadline for when your renders have to be done. This deadline needs to be at least two weeks before your thesis is due for Output.


Recommended