Tutorial: Cocos Shader Series - Alpha Testing and Blending

Cocos Shader Series - Alpha Testing and Blending

Series Chapters

  1. Basic Introduction
  2. Build A Triangle
  3. Draw More Things
  4. Add Some Texture
  5. Draw Shaders With Cocos Creator
  6. Change a shader with a texture map
  7. Use a Noise Map to Make a Dissolve Texture
  8. Building With WebGL
  9. Alpha Testing and Blending
  10. Blend Testing

In the previous chapters, what we know is the rendering on a flat surface. It is easy enough to draw things directly on the screen, and the content of the drawing is relatively simple. But in 3D games, we need to consider a lot more. For example, there are many character models in the direction of the player’s view in a crowd, and each character model needs to be drawn. How to prevent objects entering the camera from being occluded far away? For another example, there are many shops with glass windows on both sides of the street. How can I see the scene through the glass windows?

To achieve this function, you need to involve the last stage of the rendering process: alpha testing and blending. At this stage, the main job of the GPU is to operate on a piece-by-piece basis, combining their colors in some form to obtain the final pixel color displayed on the screen. There are two main tasks involved: testing and merging the fragments. The test step determines whether the fragment will eventually be displayed. The main tests in WebGL include tailoring testing, transparency testing, template testing, and depth testing. These tests are all highly configurable. Considering that the tailoring test is not more flexible than the template test, this time, the tailoring test content is not involved. The whole test process is as follows:

It can be seen from the figure that the color buffer output by the fragment shader is not the color buffer presented on the final screen, and the final color buffer used for output can be obtained after being affected by the template, depth, and mixing test. This chapter focuses on template testing and in-depth testing. Hybrid testing will be introduced in the next chapter.

Note: Since this part of the content is supplementary knowledge, the critical point is to understand its concept and application in Cocos Creator 3.x.

Stencil Test

The essence of Stencil is hollow, and a specific shape can be easily drawn. The core of the template test is to hold a template buffer, and each pixel/fragment has a template value. Usually, each template value is 8 bits (represented by a mask). That is, there can be 256 different values so that you can discard or keep this fragment by setting the template value we want. A simple template test example is as follows:

Generally, when the user enables the template buffer, the template value of all fragments in the entire template buffer is set to 0, and all fragments are discarded. Then set the template value (greater than 0) and the comparison function of the specific area. The GPU will read the stencil value set by the user and then compare that value with the stencil value at that location in the stencil buffer by the comparison function and finally decide whether to keep or discard the fragment to form the skeleton or mask effect. In stencil testing, there are two very important methods, stencilFunc and stencilOp. The former is used to control the stencil test method and get the test results, and the latter decides how to deal with the data in the buffer based on the results.

void stencilFunc(GLenum func, GLint ref, GLuint mask):
  • func: Specify the template test comparison function. The default is Always.


  • ref: The reference value used for template testing.
  • mask: Specifies the operation mask. During the test, ref and mask will be ANDed first, then ref and the value in the stencil buffer will be ANDed, and finally, the result will be obtained according to the comparison function.

Just looking at these descriptions may not help you to understand the meaning here. Here is an example:

gl.stencilFunc (gl. GEQUAL ,  1 ,  0xff );  // Here, the reason why the mask adopts hexadecimal is because the representation of data in the computer ultimately exists in binary form, but the binary is too long to write so that It can be solved in hexadecimal or octal. The larger the base, the shorter the expression length of the number.

This configuration means to compare the value 1&0xff with stencil buffer&0xff to determine whether the conditions of GEQUAL are met, and if the conditions of GEQUAL are met, the test passes; otherwise, the test fails. Therefore, we just want to compare ref and stencil buffer here, and the mask cannot be an interference factor, so set it to 0xff (11111111), let each bit be 1, and the “AND” calculation will keep the original value. If you want to disable the template, you can set the value of 0x00 so that the values in the template buffer are all 0.

What to do with the stencil buffer after the stencil test passes or fails? You need to use:

void stencilOp(GLenum fail, GLenum zfail, GLenum zpass);
  • fail: Specify the behavior when the current template test fails. The default is KEEP.

  • zfail: Specify the behavior when the current mask test passes but the depth test fails. The allowed and default values are the same as fail.
  • zpass: Specify the behavior when the current stencil test passes and the depth test also passes, or when the stencil test passes and the depth test is not turned on. The allowed and default values are the same as fail.

The content here is easier to understand. Usually, we will keep the current value when the test fails and replace the template buffer value with the set value when the test passes.

gl.stencilOp (gl. KEEP , gl. KEEP , gl. REPLACE );

The mask test is disabled by default and needs to be manually turned on when using it.

// By default, the template test is disabled. You need to enable the template test manually
 gl. enable (gl. STENCIL_TEST ); 
// It is also necessary to clear the template buffer before each iteration
 gl. clear (gl. COLOR_BUFFER_BIT  | gl. STENCIL_BUFFER_BIT );

By the way, if you want to try to write a template test on WebGL, if you encounter a situation where the template test does not take effect, you can check whether it is required to include a template buffer when requesting the context.

const  gl = canvas. getContext ( "webgl" , {stencil:  true  });

Depth test

Depth testing is an indispensable and essential part of 3D games. It can help achieve the occlusion effect of objects on 3D rendering. If there is no depth test, the rendering of the front and back objects may be disordered or flickering.

The core of the depth test is similar to the template test. It also holds a depth buffer. The depth buffer is like a color buffer (the storage buffer of the final pixel color value, and the pixel color presented on the final device is read from here.) The depth value of each segment is also stored in the form of 16, 24, or 32-bit floats. The default precision is 24 in most systems. When the depth test is turned on, the depth value of each fragment currently rendered will be tested against the content of the depth buffer. If the test passes, the depth buffer will update the new depth value. If the test fails, the fragment will be discarded.

The depth buffer is run in the screen space after the fragment shader has run (also after the stencil test). The screen space coordinates are related to the gl.viewport setting. WebGL will directly use the GLSL built-in variable gl_FragCoord to access it from the fragment shader directly. The x and y components of gl_FragCoord represent the screen space coordinates of the fragment. At the same time, it also contains a z component, which is used to store the actual depth value, and finally, use it to compare with the content in the depth buffer.

The depth buffer also has an important function void depthFunc(GLenum func) is used to set the depth comparison function. The comparison parameters are the same as those used by the template buffer comparison function. The default parameter is LESS. The depth test is also disabled by default, and it also needs to be turned on manually.

const  gl = canvas. getContext ( "webgl" , {stencil:  true , depth:  true  }); 
gl. enable (gl. DEPTH_TEST );

When the depth test is passed, the z value of the current segment will be stored in the depth buffer. The z (depth) value of the current clip is a direct value between 0.0 and 1.0. See the z value of the object in the scene from the observer’s point of view. This value is converted to the value between 0.0 and 1.0 after the coordinate transformation of the standard equipment after the projection matrix is applied.

Application in Cocos Creator 3.x

Based on the above content, I believe that everyone should have a basic understanding of template testing and depth testing principles. Next, let’s try how to apply this part in Cocos Creator. The bottom layer of Cocos Creator initializes the template/depth, etc., by default. The implemented module is the webgl1/webgl2-device.ts module in the engine source code. Because here I am testing using the backend of WebGL1, In Cocos Creator folder, find resources -> 3d -> engine -> cocos -> core -> WebGL -> webgl-device.ts module in the version installation directory, you can see the following initialization content:

Note: Although there is a slight difference in the API part, this is because WebGL provides more than one method for setting, but the concept is basically the same.

The default configurations listed here are mainly for 3D objects. Since most 2D objects contain transparent pixels, the underlying 2D pipeline does not handle the depth part, which eliminates the need for depth testing, as seen on previous 2D Effects like builtin-sprite, where the depth testing part was manually turned off. Therefore, in the following practice of depth template testing, 3D objects are used. Try to test the depth correlation by placing 2 models in the scene. The model is O1, and the scene model is O2. The models and the camera are placed as follows.

From the “Cocos Shader Series: Basic Introduction (5)”, there is a way to write about the template depth buffer. You can see that each pass can configure the template/depth buffer. The rough wording is as follows:

CCEffect %{ 
  techniques: -name 
  : opaque 
    passes:       -vert: 
      frag: ... 
      properties: ... 
          deprhTest:  false

This is usually written to set the data when the pass is initialized, but if it needs to be modified, Cocos Creator 3.x also wraps the Effect writing style. The PipelineStates property can be seen under the pass of each material on the property inspector panel, which can be easily configured visually.


The configurable parameters and description are as follows:

Next, modify some configurations of the character model O1:

The material closes depthTest and applies it. It can be observed directly on the editor that since there is no depth test, the drawing content of O1 is covered by the ground, and the rendering order of the decorations on the body is also disordered.

Turn on the depthTest for the material, adjust the depthFunc to GREATER, and apply it. At this point, you can find that no matter how you find it, the model cannot be seen. Typically, because the depth test function used by the background model is still LESS, it will cover the depth buffer content, but not all can be covered, so the part that is not covered may not be completely invisible to the character model. This is mainly because we clear the depth buffer in every frame. The default value of the cleared depth buffer is 1.0, which represents the maximum depth value. Therefore, the character model cannot be farther than the maximum.

Restore all modifications, and then perform a template test. The template test requires two basic operations, one is to clear all the fragments, and the other is to set the template value of a specific area. Then, the objects that need to be drawn only need to be drawn on a specific template value. I am here to implement an effect that only shows the upper body of the character model. Prepare two quads, each with its own material (Effect uses the default builtin-standard), the A-side is closest to the camera, and the character disappears; the B-sides are only the size of the upper half of the character, located in the upper half of the character’s position, compared to the A second closer to the camera, do the character only display area settings; the final character behind these two facets. Placement is as follows:

Then, do the following:

  1. Turn on both the front and back stencilTest of material A (the material of face A). What the front and back sides are will be discussed in the next chapter. Set stencilFunc to NEVER and stencilFailOp to ZERO, and click Apply. The configuration here makes the stencil test never pass, execute the fail function, and set the stencil buffer of all model drawing areas to 0.
  2. Turn on both the front and back stencilTest of material B (the material of face B), set stencilFunc to NEVER, stencilFailOp to REPLACE, and stencilRef to 1, and click Apply. The configuration here makes the stencil test never pass, execute the fail function, and set the stencil buffer of all model drawing areas to ref.
  3. Turn on both the front and back stencilTest of the character material, set stencilFunc to EQUAL, stencilRef to 1, stencilReadMask, and stencilWriteMask to the ref, stencilFailOp, stencilZFailOp and stencilPassOp to KEEP, and click Apply. The configuration here can only pass the template test when the ref value equals the template test value. After the test passes, replace the corresponding fragment in the template buffer with stencilRef & stencilWriteMask.

The final results are as follows:

Only the upper body of the model is shown. You can try to observe the effect from different angles.

Template and depth testing content ends here, and interested developers can add different combinations to achieve special effects. In the next chapter, we’ll learn about blended testing (BlendState) and face culling (CullMode).


  1. Template test

  2. Depth test

  3. Optional Pass Parameters