All Hail Camera.RenderWithShader

“one day there was no internet in the office, so I did not know what bugs to fix… so I played around with this instead :)” – Aras Pranckevičius

That day of internet outage is probably one of the happiest days for me using the Unity3d engine. On that fateful day Camera.RenderWithShader was born which lived on to become the backbone of almost all of our post processing effects. The best way to explain how this makes a world of difference in our projects is to just give examples of how we handled post processing effects before and after replacement shaders.

Jetpack Brontosaurus

What We Did

We haven’t posted a graphics postmortem for Jetpack Brontosaurus yet. This post will simplify the techniques we used in order to demonstrate how we accomplished the “multiple dimension” effect. Each object in Jetpack Brontosaurus has at least three renderers associated with it. One for the nightmare/death dimension, one for the dream/living dimension, and one for the mask that determines which dimension gets rendered to the screen. The renderers are divided into each dimension by using different layers. We have a different camera for each dimension. One camera is set to render all the renderers in the nightmare dimension, one camera is set for the dream dimension, and one for the mask. These cameras output their renders to separate render textures. Then we combine the two dimensions based on the color in the mask.

The implementation required each of the three renderers to have its own game object, material, and layer. As a result, the complexity of the scenes increased, producing redundant information and human errors.

What We Could Have Done

If we had the ability to use replacement shaders during Jetpack Brontosaurus production, everyone’s life would have been easier. Most of the renderers in Jetpack Brontosaurus used a vertex color shader. Because the renderers shared the same shader, using a replacement shader is an easy change.

An important thing to note here is that all of the tags have the same Key/Value pair.

The dream replacement shader:

Shader “Bronto/Dream Replace” {
   SubShader {
      Tags {“RenderEffect”=“Multidimensional”}
      Pass {
         ColorMaterial AmbientAndDiffuse
         Lighting Off
         SetTexture [_DreamTex] {
            Combine texture * primary, primary
         }
      }
   }
}

The death replacement shader:

Shader “Bronto/Death Replace” {
   SubShader {
      Tags {“RenderEffect”=“Multidimensional”}
      Pass {
         ColorMaterial AmbientAndDiffuse
         Lighting Off
         SetTexture [_DeathTex] {
            Combine texture * primary, primary
         }
      }
   }
}

The mask replacement shader:

Shader “Bronto/Mask Replace” {
   SubShader {
      Tags {“RenderEffect”=“Multidimensional”}
      Pass {
         Lighting Off
         Color [_MaskColor]
      }
   }
}

Now we have one shader that has texture and color information for all of dimensions in a single material. No need for more than one renderer per object anymore!

The shader used by the material for the renderers (this shader is also used for rendering the object to the scene view):

Shader “Bronto/Multidimensional Object” {
   Properties {
      _DreamTex (“Dream Dimension Texture”, 2D) = “white” {}
      _DeathTex (“Death Dimension Texture”, 2D) = “white” {}
      _MaskColor (“Mask Color”, Color) = (1,0,0,1) // Alpha used for interpolating between the two textures in the scene view
   }
   SubShader {
      Tags {“RenderEffect”=“Multidimensional”}
      Pass {
         ColorMaterial AmbientAndDiffuse
         Lighting Off
         SetTexture [_DreamTex] {
            Combine texture
         }
         SetTexture [_DeathTex] {
            constantColor [_MaskColor]
            combine previous lerp(constant) texture
         }
         SetTexture [_DeathTex] {
            combine previous * primary
         }
      }
   }
}

For ingame rendering the material shader is never used. The only shaders used are the replacement shaders.

Now we have a script on the scene’s camera that renders the scene with each replacement shader and then composites them. The scene’s camera should be set to render nothing in its culling mask.

#pragma strict
@script ExecuteInEditMode
@script RequireComponent (Camera)

// The culling mask that should be used for rendering
var cullingMask : LayerMask;

// The replacement shaders
var dreamReplacementShader : Shader;
var deathReplacementShader : Shader;
var maskReplacmentShader : Shader;

// The magic composite material
var dimensionCompositeMaterial : Material;

// The render textures for each dimension
private var dreamRT : RenderTexture;
private var deathRT : RenderTexture;
private var maskRT : RenderTexture;

// The camera that renders the replacement shaders (Don’t access this directly, use GetPPCamera())
private var ppCamera:Camera;

/**
* Handle any needed pre processing
*/

function OnPreCull() {
   // Start from nothing
   CleanRenderTextures();
   
   // Reference to ppCamera’s camera
   var cam:Camera = GetPPCamera();
   
   // Set up camera
   cam.CopyFrom(this.camera);
   cam.cullingMask = this.cullingMask;
   cam.clearFlags = CameraClearFlags.Skybox;
   cam.backgroundColor = Color(0.0,0.0,0.0,0.0);
   
   // Render Dream Dimension
   dreamRT = RenderTexture.GetTemporary(Screen.width, Screen.height, 16);
   cam.targetTexture = dreamRT;
   cam.RenderWithShader(this.dreamReplacementShader, “RenderEffect”);
   
   // Render Death Dimension
   deathRT = RenderTexture.GetTemporary(Screen.width, Screen.height, 16);
   cam.targetTexture = deathRT;
   cam.RenderWithShader(this.deathReplacementShader, “RenderEffect”);
   
   // Render Death Dimension
   maskRT = RenderTexture.GetTemporary(Screen.width, Screen.height, 16);
   cam.targetTexture = maskRT;
   cam.RenderWithShader(this.maskReplacementShader, “RenderEffect”);
}

/**
* Post Processing magic
* @param source
* @param destination
*/

function OnRenderImage(source:RenderTexture, destination:RenderTexture)
{
   // We do nothing with the source render texture, the camera didn’t do anything to it anyway!
      
   // Magically composite the render textures together into the final render
   // The shader used in the dimensionCompositeMaterial for compositing these textures is outside the scope of this post
   // Will have a post about CG full screen post processing effects sometime in the future
   RenderTexture.active = destination;
   dimensionCompositeMaterial.SetTexture(“_DreamRender”, dreamRT);
   dimensionCompositeMaterial.SetTexture(“_DeathRender”, deathRT);
   dimensionCompositeMaterial.SetTexture(“_MaskRender”, maskRT);
   GL.PushMatrix ();
      GL.LoadOrtho ();
      for (var i:int = 0; i < dimensionCompositeMaterial.passCount; i++) {
         dimensionCompositeMaterial.SetPass (i);
         DrawQuad();
      }
   GL.PopMatrix ();
   
   // Clean up
   CleanRenderTextures();
}

/**
* Cleanup if we get disabled
*/

function OnDisable()
{
   CleanResources();
}

/**
* Camera that renders the replacement shaders
* ppCamera getter
* @return
*/

private function GetPPCamera():Camera
{
   // Create the shader camera if it doesn’t exist yet
   if(!ppCamera) {
      ppCamera = new GameObject(“PPCamera”, Camera);
      ppCamera.camera.enabled = false;
      ppCamera.hideFlags = HideFlags.HideAndDontSave;
   }
   
   return ppCamera.camera;
}

/**
* Cleanup all resources used for Post Processing
*/

private function CleanResources()
{
   if(ppCamera)
   {
      DestroyImmediate(ppCamera);
   }
   CleanRenderTextures();
}

/**
* Cleanup Temporary RenderTexture resources
*/

private function CleanRenderTextures()
{
   if(deathRT != null) {
      RenderTexture.ReleaseTemporary(deathRT);
      deathRT = null;
   }
   if(dreamRT != null) {
      RenderTexture.ReleaseTemporary(dreamRT);
      dreamRT = null;
   }
   if(maskRT != null) {
      RenderTexture.ReleaseTemporary(maskRT);
      maskRT = null;
   }
}

Blush

Glow

The Pro Standard Assets package that comes with Unity has a simple glow effect. Their implementation uses the alpha channel of the destination render texture to decide where to render glow. This limits us a couple ways. We can not have multicolored glow effects and we can not use the alpha channel for anything else. We decided to just bite the bullet and have a separate render texture for rendering glow.

Now we’ve freed up the destination render texture’s alpha channel for something else to use and we can have glow any color we like. Other than the additional memory needed for a 32bit render texture there’s a large disadvantage to doing it our way. Glow is no longer occluded by other geometry. A workaround for this is to have objects that you don’t want to glow render black to the glow render texture. Since glow is an additive pass, anything that is black will do nothing to the original image.

The Glow Replace Shader:

Shader “Blush/Glow Replace” {
   SubShader {
      Tags { “RenderEffect”=“Glow” }
      Pass {
         Fog { Mode Off }
         Color [_Glow_Color]
      }
   }
}

For every object we wanted to glow we added two things to the object’s original shader.
– Tag: “RenderEffect” = “Glow”
– Property: _Glow_Color (“Glow Color”, COLOR) = (1,1,1,1)

Distortion

Distortion is handled in a few steps.
1. Render the scene to a render texture.
2. Render a 2 dimensional “normal” map to a render texture.
3. Draw the scene’s render texture to the screen offsetting each texels’ texture coordinate by the amount specified by the 2 dimensional “normal” map texture.

Blush used a constantly oscillating full screen normal map to distort the scene. This distortion helped established an underwater feel. We used replacement shaders to render directly to this normal map to further modify the distortion. The artists used these shaders on particles to provide distortion effects on fast-moving tentacles, enemies, and other places.

The Particle Distortion Replace Shader:

Shader “Squiddy/Post Processing/Distortion Replace” {
   Properties {
      _BumpMap (“Bump (RGB)”, 2D) = “bump” {}
   }
   SubShader {
      Tags { “RenderEffect”=“Distort” }
      Pass {
         Lighting Off
         ZWrite Off
         Blend SrcAlpha OneMinusSrcAlpha
         BindChannels {
            Bind “Color”, color
            Bind “Vertex”, vertex
            Bind “Texcoord”, texcoord
         }
         SetTexture[_BumpMap] {combine texture, texture * primary}
      }
   }
}
This entry was posted in Graphics and tagged , , , . Bookmark the permalink.

3 Responses to All Hail Camera.RenderWithShader

  1. Mathieu says:

    Speaking of Jetpack Brontosaurus, is it just me or the game doesn’t work anymore. Nothing happens after loading…
    http://www.blurst.com/jetpack-brontosaurus/play

    Anyway, thanks for sharing all this information!

  2. Can you post the tutorial of the raycast vehicles that you mention on Unite, please? It will be very useful for me and surely for a lot of people of the Unity community. Thanks!

  3. Hizral says:

    Hi, my name is Hizral and im trying to apply the “blush” type of glow and it seem I have a problem.

    1. how do I use the shader? can anyone teach me how to use the shader.
    2. when I copy and paste your shader I receive “Shader has errors or is not
    supported by your graphic card”

Comments are closed.