mental ray 101: Essentials

Part 1

Jennifer O'Connor

College of Lake County



mental ray 101: The Essentials


Version:  December 2008

Jennifer O’Connor

Engineering, Math, and Physical Sciences Department

Produced for CAD179, CAD 279 and ARC 216

College of Lake County, Grayslake, Illinois

joconnor at or jenni at

There are a number of new features in the new mental ray version 3.5+ that make it easier to make great renderings.  We’ll take a look at a few of the new features, and some of the things I’ve been using a lot lately. 

There have been a lot of things written about mental ray, and I’ll try not to duplicate too much here.  My goal is to be succinct, and to cover some of the essentials to give you an overview, and get you going with this powerful renderer in 3ds Max release 9/2008/2009 and Autodesk VIZ 2008.  The best way to learn is to teach, I believe, so this is an exercise in learning as much as anything else.

Essential Settings

“Essential Settings” – the things that you should look for and learn first when learning mental ray – will have Blue Bold text associated with them saying, cleverly, “Essential Settings”.  Although I do not cover all the settings of the renderer I tried to stick to the things I use the most, and there are certainly things here that are aimed to the other geeks in the world.

I will also have Very Important Notes within blue-bordered text boxes.  Other notes will simply be offset with a bold “NOTE:” right before.

Additional Reading

For more details on these features I highly recommend two new books on the subject, "Rendering with mental ray & 3ds Max" by Joep van der Steen, and "Realistic Architectural Visualization with 3ds Max and mental ray" by Roger Cusson and Jamie Cardoso.

A PDF version of this document, and the Max scene files used, are at  The 'Light Gallery' scene provided is a slightly modified version of a mental ray sample provided with 3ds Max.

In Part One we'll look at the general features of the renderer, and Indirect Illumination.  In Part Two we'll look at Lighting, including the Daylight System and Exposure Controls, and in Part Three we'll look at Materials, including the new Car Paint material, Arch & Design, and Water materials, among others. Part Four will cover GI and Caustics.

This document talks about 3ds Max release 2009 and newer, but much subject matter also pertains to 3ds Max 9/2008, Autodesk VIZ 2008, and to some degree Maya 8.5+.

mental ray is a registered trademark of mental images GmbH. 3ds Max, Autodesk VIZ and Maya are registered rademarks of Autodesk, Inc.




mental ray 101: The Essentials. 2

What is mental ray?. 5

Enabling mental ray. 6

mental ray General Settings. 8

Distributed Rendering. 9

Memory Management in mental ray. 11

Use Placeholder Objects. 12

Use mental ray Map Manager 12

Memory Limit 12

Conserve Memory. 13

Direct Illumination. 14

Indirect Illumination. 14

Indirect Illumination in mental ray. 16

Final Gather 17

Visual Diagnostic Tool 20

mr Sky Portals and Final Gather 29

Flicker-Free Animation with FG Lock. 30

Rendering Long Animations and High-Res Stills. 31

Controlling FG Calculation. 36

Global Illumination. 37

Caustics. 38

Importons and Irradiance Particles. 40

Quality Settings. 43

Samples per Pixel (SPP) 43

Spatial Contrast 44

Quality and Time. 46

Filters and Quality. 46

Other Important Settings. 46

Part Two, Three and Four 47

Thanks! 47



What is mental ray?

mental ray (mr) is a third-party render engine for 3ds Max, VIZ and Maya which provides a number of enhancements over the default render engine, including Indirect Illumination (bounced light), Caustic effects (the concentration or reflection of light from a transparent or reflective surface), and procedural effects ranging from water ripples to sub-surface translucency (as with human skin), among many other things.  mental ray is produced by mental Images in Germany, an NVIDIA company.

The default Scanline renderer in Max and VIZ work on an image one horizontal line one at a time, from top-to-bottom.  mental ray, in contrast, renders scenes in ‘Buckets’; each processor core in your machine takes a small rectangular region of the rendered image (a bucket) and processes that portion before moving on to the next available bucket.  Brackets appear around each bucket as it is processing, and when the bucket completes it jumps to the next easiest bucket to render (by default).  Here are some completed buckets, and four that are in-process on a quad-core machine….

Figure 1: mental ray bucket rendering

Buckets work independently of one another, and complete asynchronously depending on how long each bucket takes to render in a given CPU core.  mr will then pick up the next available bucket to work on, and so on, until the image is completed.  This image (above) shows a Spiral bucket method rather than the default, a method that starts in the center and works outwards to complete the image.  The default method appears to randomly walk the image, working on the parts that are easiest to accomplish, and this method giving you a faster preview of your overall image.


Enabling mental ray

In 3ds Max Design 2009, mental ray is the default rendering engine, and the materials in the material editor are, by default, mental ray's "Arch & Design" and "ProMaterials".  For legacy scenes, and if you are using 3ds Max 2009 or earlier, you may need to select the mental ray renderer manually.

In the sample scenes provided with this document, mental ray is already selected.

In 3ds Max, F10 brings up the Render Scene dialog.  Choose the Common tab, roll-down the Assign Renderer section, and click the button '...' to choose mental ray:

You can also choose a Preset from the bottom of the Render Scene dialog (press F10) that includes a few mental ray options:

As with all settings, it is best to look at what is changed with each preset both to learn what they do, and to see what you may need to adjust to get the look you need. 

Another way to set mental ray as your default renderer is to go to the Customize pull-down menu, and select the "Customize UI and Defaults Switcher..." option, and then choose an option with mental ray:

As you switch between tool options you’ll see that a number of settings are changed.  This tool changes your complete Max/VIZ configuration, in this case to use mental ray (and the other settings shown) to a new default.

The right side of the dialog box optionally changes your UI.  I tend to customize my UI quite a bit, and if you do the same then be certain to save your current UI before switching things here.  The 3ds Max dialog has a few additional options over the VIZ version.  The "DesignVIZ mental ray" option is the default in 3ds Max 2009 Design.


mental ray General Settings

For mental ray general settings, go to the Customize and Preferences... dialog, then the mental ray tab:

I turn off the ‘Clear Frame Window…” button (Off by default for Max 2009) as I like to see bucket progress with the last render in the view so I can compare.  Otherwise it blanks the Rendered Frame Window (RFW).

Prior to Max 2009, if you do a region render, it will blank the area around the region when the render completes, and you'll need to save the region to a separate file and use a compositing program to paste that region into your original render.  Adobe Photoshop or Corel PhotoPaint skills are usually required in this field.

In Max 2009 mental ray does do Region without clearing the Rendered Frame Window, eliminating the need for compositing regions in another application. 

You can get a feel for what mental ray is doing while rendering by looking at the log that is created.  This also allows you to see any errors or warnings that are produced.  Turn on the ‘Show/Log’ buttons (previous image) and press OK, and then go to the Render pull-down and select the “mental ray Message Window…” option to see the messages as they are generated.  Usually I keep that closed, but it will open automatically if there are any errors during the render.  As you are learning mental ray it can be interesting and informative to see what messages are being generated, including warnings you wouldn’t know about otherwise.  This can help you to optimize and troubleshoot your scenes.


Distributed Rendering

Because mental ray subdivides of the image into buckets, it can distribute the rendering to other machines, potentially allowing every computer on your network to work on individual buckets at the same time (with some limitations).  This is known as Distributed Bucket Rendering, or DBR.

This is just one of the big advantages to using mental ray over the Scanline renderer, the ability to use additional hardware to get your image completed faster. 

A little about how it works under-the-hood…

When you click on the Render button your scene is translated into a format that mental ray understands, a 'mib' format, and that translated scene data is passed on to the renderer.  3ds Max can optionally cache (store) the data to the hard drive for later reuse.

In distributed rendering, the translated scene data and associated bitmaps are sent to other machines where each machine’s processor core(s) will handle the rendering of an individual bucket.  The buckets are managed from the one copy of 3ds Max or Maya that you are using.  At the remote computers you need to have either a copy of 3ds Max installed, or the stand-alone satellite version of mental ray.   3ds Max allows you to use 8 distributed CPUs without purchasing additional stand-alone licenses..

To enable Distributed Rendering, you open the Render dialog in Max (press F10), and with mr as your renderer, you go to the Processing tab and the Distributed Bucket Rendering roll-down section:

NOTE: According to the mental ray documentation with 3ds Max, you must turn on the “Use mental ray Map Manager” translator option in the “Processing” tab when using DBR.  Check to make sure all your maps look okay in test renders, though!  If you see seams, then keep it off.  More on this option later.

NOTE: You must use the “Use Placeholder Objects” translator option when using DBR.  More on this option later.

Before it will work, you need to add the name or IP address of the remote machines to the 'rayhosts' list by clicking the Add button.  Above I have six of my machines listed with five selected for use (in blue), giving me five total machines that will be used for the render (including my local machine).  If you are running 3ds Max 64-bit, then the target machines must also be x64.

Here is a matrix of the mr capabilities of 3ds Max and VIZ from Autodesk:


VIZ 2008

3ds Max 9-2009

Number of mental ray satellites



Local Processing cores limit for mental ray



Unlimited network rendering or integrated mental ray via backburner



64-bit architecture



Additional licenses are relatively cheap, and the pricing model seems more reasonable than other third-party renderers such as VRay or Brazil, but you’ll need to do some math on that.  

Maya Complete 2008 gives two mr processor licenses, and Unlimited gives eight licenses.

NOTE: VIZ users that need a lot of mental ray power, or have very large scenes requiring more memory, or want to network render mr scenes, should upgrade to 3ds Max rather than use VIZ 2008.  Since VIZ is now discontinued, an upgrade to 3ds Max is probably inevitable.  The additional functionality that 3ds Max brings to mental ray makes the upgrade a good value, in my humble opinion.

NOTE: My experience so far is that Archvision RPC content may show up as red boxes if you use distributed rendering.  Archvision is aware of this, and hopefully this will be fixed some time soon.  In this case, use the Split Scanlines option in Network Rendering to use the other computers. 

NOTE: The bandwidth used on the network with distributed rendering seems pretty reasonable, although we do use gigabit adapters with gigabit switches.  I still tend to Backburner network render to remote machines rather than use distributed rendering, and use the Backburner “Split Scanlines” option for large images.  This way multiple machines still work on the one image, and progress is saved along the way if there are issues with a machine or memory.


Memory Management in mental ray

Since the topic of memory settings have come up with DBR, we need to go over a few essential options.  As mentioned in the DBR section, (with caveats) two options should always be enabled when using DBR:

·         Use Placeholder Objects

·         Use mental ray Map Manager

Use Placeholder Objects

The ‘Use Placeholder Objects’ option allows mental ray to store your objects as empty bounding-boxes – placeholders – and  swap out parts of your scene to conserve memory.  DBR will still work if you don’t turn this on, however mental images says that this is a ‘must’. 

The use of this option causes the ‘Translator’ – the mental ray process that converts your scene into mental ray data – to only send to the renderers bounding-boxes (placeholders) of the scene geometry.  Once a bucket hits a piece of geometry that is only a placeholder, then the actual translated geometry is passed to the renderer.  If needed, placeholders can be removed from memory.  This can save a lot of memory and network bandwidth, particularly if geometry is off-screen.

Use mental ray Map Manager

The “Use mental ray Map Manger’ option causes mental ray to only read bitmaps from disc when they are needed.  It then converts them into a mental ray format and they are held in memory.  mental ray can also remove the map from memory as need.

Turning this Off prevents mr from off-loading the bitmaps, and all maps are held in memory.  This can be faster, as 3ds Max also holds bitmaps between rendering jobs.

Because the algorithms used are slightly different between 3ds Max and mental ray, you generally would not switch between these two methods for a particular image.

My experience with the mr Map Manager is that I can get ‘seams’ in some bitmaps, and I cannot always use this feature.  An alternative to this may be to use the 3ds Max Bitmap Pager.

Memory Limit

This is an Essential Setting.

This should be set somewhere near the available memory limit on your computer, figuring in resident applications, 3ds Max and mental ray.  This is the threshold where mental ray will begin to remove placeholders (when used) and also bitmaps from memory to make room for more.  The default is 640mb, and this should be set as high as you can.  For 4gb machines I’ll set this to 3000mb.

Conserve Memory

The ‘Conserve Memory’ option causes mental ray to work harder to minimize memory use, at the expense of additional time in the translator reading placeholders and reading/translating maps.

Other Memory Optimizations…

We will cover other memory issues as we hit topics throughout the document.


Direct Illumination

As its name implies, Direct Illumination is the light that shines directly on a surface from a light, and is not bounced (indirect) illumination.  Here is a night scene with incandescent ceiling ‘can’ lights at 75 watts, and a cylindrical white light in the floor lamp at about 150 watts.:

Figure 2: Rendered in mr with only Direct Lighting from Photometric Spot lights in ceiling and a Photometric Cylinder Area Light in floor lamp.

This scene is the mr_LightGallery_Direct.max  scene.  With only Direct Illumination in a scene, the images is somewhat dark despite a low exposure setting, and areas in shadow are completely dark (although the cylinder light helps to fill those shadow areas somewhat). 


Indirect Illumination

Indirect Illumination is the light in your scene that is reflected off, and/or refracted through, objects.  

In the real-world, light strikes a surface, some of it is absorbed and the remainder will become the color of that surface and is scattered to other surfaces. Some of it will pass through objects and be refracted. The light is partially absorbed, reflected/refracted and re-colored again and again until the light is fully absorbed by the surfaces.  As it bounces, the changed color of the light will affect the color of the other objects it strikes, so, for instance, a bright red carpet will give you a pink tint throughout your room, and light refracted through colored glass will become the color of the glass.

In the following image, you can see the increased illumination in the scene from the bounced lighting, and also the color splash from the flooring onto the lower edges of the walls and the ceiling: 

Figure 3: mr_LightGallery scene with Indirect Illumination enabled.

This image used the Indirect Illumination “Final Gather Medium” preset and Global Illumination, and has identical exposure settings as the previous image.

As you can see, visible areas of your rendering that are in-shadow and not in direct light still receive illumination bounced from all other visible surfaces.   Those previously dark areas will be illuminated to some degree, and your image is brighter.   You also have light splashed off of surfaces, and this is most noticeable on the ceiling.

Perhaps 80% of the light that illuminates a space is from Indirect Illumination, and simulating this process is essential for producing the highest-quality images.


Indirect Illumination in mental ray

Indirect Illumination in mental ray is produced by one or more of these technologies:

*  Final Gather (FG)

*  Global Illumination (GI)

*  Caustics

*  Importons

*  Irradiance Particles (IP)

*  Ambient Occlusion (AO)

Ambient Occlusion is an Indirect Illumination effect produced inside of mental ray materials, and are covered in Part 3.  In mental ray AO is used for detail enhancement and helps to produce contact-shadows and fine details in objects.

‘Global Illumination” is often used as a generic term for Indirect Illumination, but in mental ray it is a type of indirect illumination.  Caustics are the effect of light photons that have been reflected off or refracted through an object, and produces effects such as magnifying light through glass, or light reflection patterns off of a water surface.

GI and Caustics will only be briefly discussed later in this document.  Importance and Irradiance Particles are new for Max 2010 (my guess) and Maya 2008SP1, and these, along with GI and Caustics, are Part 4 of this series.

Without using methods of calculating indirect light, brighter and more realistic renderings must be produced by adding lights to your scene to simulate that missing 'bounced' indirect illumination.

Final Gather

Final Gather (FG) gives you a free and near zero-effort way to add good-quality Indirect Illumination to your scenes.  It is often used alone, or combined with Global Illumination (GI) for highest quality results.

Final Gather includes ‘Bounced’ light in your scene, and not Reflected or Refracted illumination.  Its original purpose was as a ‘clean-up’ tool for GI but is now a good tool by itself.

So, why is Final Gather in the ‘mental ray 101: Essentials’ document, and not a more advanced chapter?  First, it is easy to use and second, it adds greatly to your rendered images and should be used for just about every scene.


FG works by taking the viewpoint of a small area in your scene, it shoots random rays out into your scene from that area, it collects the illumination and colors of the surfaces it samples, and stores that FG data for when the surfaces are being rendered.  At render time the FG illumination and color value is then added to the Direct illumination to give a final brightness and color.

FG is often used with the GI technology (described later) on interior scenes, and often used alone for exterior scenes.  However, Final Gather works very well by itself on interior scenes; That said, for interior scenes adding Global Illumination (GI) can greatly speed overall rendering time and improve the brightness and quality of your images.

Final Gather does require your scene to be properly illuminated with some form of Direct Lighting for it to produce results, and for a pleasing and accurate simulation you'll need perhaps a minimum one-third to one-half of the scene lit with Direct light. 

NOTE: mental ray materials that use Self Illumination can produce indirect illumination when used with FG.

Interior scenes that are lit through a small window, or have minimal lighting from small directed light sources (a ‘film noir’ scene lit from a desk lamp, for instance), may not produce good results without some help.  Even outdoor scenes that do not have a lot of direct light (only skylight, for instance) may not work as expected, or/or may take a lot of time. 

In this case, adding lights for Direct illumination, mental ray "mr Sky Portal" lights to windows and openings, fill lights to help illumination, or (even better) just switching on GI, may help considerably.  We'll discuss these options in more detail later.

To try the FG feature yourself, first open the mr_LightGallery_Direct.max scene and render to see Direct lighting.  Then...

Enabling Final Gather

In the Render Dialog (F10), choose the Indirect Illumination tab, and turn on the Enable Final Gather checkbox.   Then choose a FG Precision Preset, such as Draft or Low, and increase the number of Diffuse Bounces.  We’ll describe these as we go along.

That is all there is to it!  Render the scene and see the results for yourself…

NOTE:  If your final gather pass is taking too long, then turn down the preset to Draft, or change the Diffuse Bounces to 0 or 1, or reduce the rendered image size.

In general, the High and Very High presets take a Very long time, and I do not generally use them in a production rendering. I use Medium for most final renders.  If you don’t get good results with Medium on interior scenes, then see if you need mr Sky Portals, fill lights, or the addition of GI.  Proper lighting is always the first thing!

How Final Gather Works

Final Gather takes the viewpoint of the surface being rendered, shoots a finite number of rays randomly out into the scene from that point, and collects illumination and color from the objects it sees.  As the rays travel through your scene some of the illumination is absorbed, and some of it is re-colored at a new intensity and will propagate through your scene for multiple bounces.  The FG rays follow the inverse-square law for light, and produce a natural decay over distance.

At render time mental ray averages the FG points in a small area around each pixel to give that pixel an Indirect Illumination value.   The more points it stores (Draft to Low to Medium, and so on), and the more rays it shoots, the more accurate the FG solution and the closer to reality the rendering. 

Open the mr_LightGallery_Indirect.max scene, Render, and observe the results…..

Figure 4: The completed render, with Daylight System and mr Sky Portal added, Medium Final Gather preset and 2 Diffuse bounces.

One thing that you will notice when you render with Final Gather enabled is that it does the rendering essentially in two passes.  The first pass is the low-resolution Final Gather pass, and after that completes, it goes back and renders the buckets for your final image using the FG information to add indirect illumination to every pixel rendered.

This is what the scene looks like during the Final Gather pass.....