Hey

Introduction

---

The thought of procedural generation has fascinated me since I first heard about it. I mean.. imagine walking through a virtual environment you set up yourself, and bump into creatures and scenery you did not explicitly create or place there. To generate the meshes would be one step, and representing their surfaces another. This project focuses on the latter. In particular, methods for having the surface varying based on the surroundings are discussed. Further, as an UV-map would not be available for procedural generated meshes, one interesting aspect is to, despite that, achieve a varying surface based on an area on the mesh. While a non-procedural mesh is used for testing, it can be applied for a procedurally created mesh as well. To really point out the benefits of using a procedural shader, I chose to imitate the skin of a chameleon. See Figure 1 for reference. This animal has the interesting capability of camouflaging according to its surroundings or mood, depending on species. Further, it has a detailed cell pattern in its skin which opens up for the possibility to play around with different noise functions.

Aim

The purpose of this project is to achieve a procedural texture in which colors depend on the surrounding. A control panel should be available to the end-user, for creative adjustments regarding parameters such as cell size, color variations and amount of procedural bump mapping.

To be used: Unity + HLSL shader code.

Limitations

This project covers an alternative for UV mapping. While an UV map can be generated automatically it often needs manual justifications and will add substantial complexity to this solution, it will not be discussed.
Computational complexity is an important aspect in the field of procedural generation. It will be lightly discussed, although optimization can be seen as potential further development. Further, level of detail (LOD) depending on camera position is not considered within the scope of this work.

Theory

---

Characteristics of chameleons

First, because animals are cool, let us quickly cover some main characteristics of the chameleon skin which are to be imitated. The chameleon skin is said to contain small nanocrystals, which reflect different colors as the skin is more or less tense at the moment [1]. However, a virtual version works just fine with setting the final color directly, no crystals needed. Like in real life, the transition is smooth and has a duration of about a second or two and this is a behavior to keep.

UV-maps

A UV map can be interpreted as the 3D model stretched out into a flat 2D image, a procedure often referred to as UV-unwrap and illustrated in Figure 2. Each face in the polygon then has a corresponding face on the UV map and thus it is possible to attach attributes for different parts of the mesh easily. [2]
A common use of the UV map is to place 2D textures according to these flattened faces, while another is to mark areas with values representing other characteristics. As an example, these characteristics could correspond to the length, thickness or bend direction of fur. Also, a few integer values can be used to label areas as “leg”, “t-shirt”, or “hair”, to then attach separate surface attributes in the shader based on the label. Especially this last example would be useful in the creation of a procedural texture, in a case when the object has distinct variations in different areas. Procedural alternatives to UV-maps, other than actually generate the maps procedurally, seem not to be a widely researched subject.

Noise

Gradient noise is a technique used to produce natural structures in computer graphics, in particular to obey for the irregularity in the majority of all natural objects - things are simply not flat and exact. Grass grows in tufts, dust covers surfaces and kitchen clinker has details of imperfections. The list of examples can continue infinitely. A fragment shader processes each fragment individually and with the same code, yet the desired output should be continuous and coherent in its entirety. Gradient noise has proven its ability to meet these needs.
The first known implementation of a gradient noise function is Perlin noise developed by Ken Perlin in 1983, which since then has increased the appearance of realism in computer graphics. Along with the classic Perlin noise, further development has led to similar implementations as Simplex noise and Cellular Noise with its extension Voronoi noise, see Figure 3-5. Common for all these implementations is that pseudo-randomness is typically applied when a random value is needed, as the same result will be achieved every time the program runs. [3]
An interesting aspect to keep in mind is the implicitly of the noise functions. As described in the following sections, the basic idea seems to be based on a grid or, as in simplex noise, in shapes that are patched close together. These grids and shapes are however not often defined in advance nor stored in data structures. Instead, when the noise function is called it evaluates only the closest surrounding of the input point in real-time, and math define the shapes or grid cells.

Classic Perlin noise

The main idea of Perlin noise is to first generate regularly spaced points, each with function value zero and with a pseudo-random gradient assigned. Then, a continuous function is retrieved with interpolation between the points given the gradients. Figure 6 shows this setup in 1D. Similarly for noise in higher dimensions, the points are regularly spaced in each direction and interpolation is computed between the closest points. However, this particular interpolation step brings up the computational complexity for Perlin noise in higher dimensions, as well as complicated analytic derivatives. [4]

Simplex Noise

As a response to the computational disadvantages of Perlin noise, Simplex noise was presented in 2001 by Ken Perlin. Instead of a grid where each point is equally distanced to its neighbors, Simplex noise applies tessellation of N-space. That is, the most simple and compact that can fill an entire space without overlaps is chosen and referred to as Simplex Grid in this context, or simply simplex. For the 2D case, the simplest shape is an equilateral triangle, with the motivation that three corners are better than the four corners of a square. In 3D, the simplex shape is a somewhat skewed tetrahedron. It is however convenient to work with them as if they were not skewed, to be able to simply move unit steps along each axis when it is time to traverse the simplices. Thus, implementations of simplex noise typically includes transformations back and forth between a skewed and non-skewed coordinate space, see figure 7. Notice also how the shapes are oriented to fill the entire space - this adds a computational step the determine which kind of simplex the point lies in, which is crucial for a proper traversal. Given a point P within a simplex, only the corners of this simplex contributes to the resulting value. This assumption relies on a radial attenuation which is set to decay the contribution for a corner point to zero, before the boundary of a neighbor simplex. Further, the contributions are not interpolated but rather straight summarized, which lowers the computational complexity. See high-level algorithm steps below in Algorithm 1. [4]

        // Algorithm 1 - Simplex Noise, high-level implementation steps

        Selecting simplex:
        - Skew the input space
        - Determine which simplex cell the point is in. Integer parts of the skewed point tell which cell the point is in.
        - Skew the cell origin back to (x,y,z) space
        - Calculate the x, y, z distances from the cell origin
        - Compare the magnitude of coordinates relative to the cell origin to determine which type of simplex the point is in.

        Traverse simplex:
        - Given which type of simplex the point is in, move unit steps along each axis.
        - Retrieve gradient for each simplex corner, e.g. with the use of hashed gradients based on cell coordinates.
        - Calculate the contribution from each simplex corner and add to the final value, scale to stay inside range [-1,1]

      

Cellular Noise

Cellular Noise was presented in 1996 by Steven Worley. It is a noise function based on the idea for random feature points and the fact that for any position p, it is closer to a specific feature point than any other. Points with an equal distance to two neighboring cells then contribute to a natural edge halfway between. The distance from x to the closest feature point can then be used to color the surface as shades of gray, where white would represent the center of the cell and positions closer to an edge gradually darken. [5]

        // Algorithm 2 - Cellular Noise, high-level implementation steps

        - With the domain considered to be a grid with center points at each integer coordinate:
        - Determine in which cell the current point p lies within, preferably with a floor operation.
        - Scan the set of neighboring 3x3 cells with a nested loop
        - A feature point is generated inside each of these 9 cells, with a random displacement from its center
        - Find the feature point with the minimum distance from p
        - The minimum distance is returned and used for shading ( sometimes along with the second smallest distance )

      

Hierarchical Voronoi noise

As an extension of cellular noise, Voronoi noise tessellates the space to distinct cells. This feature may be convenient for procedural generations of brickwork or very appropriately, reptile skin. Classic implementations, however, produces a constant density of cells, which may not match the features of the true material. Inigo Quilez informally presented Hierarchical Voronoi noise in 2015 and distributed code under MIT License. This implementation is mostly equal to the Voronoi noise explained above, with the exception that depending on a random value, some cells split themselves into smaller cells. Thus, the same snippet of code that deals with the neighbor scan appears several times and runs depending on the level of the hierarchy. Unlike the classic Voronoi noise where the position for a neighbor cell is retrieved by a constant tile offset equal to 1, this offset is reduced by half each time the level hierarchy increases. [6]

2D noise versus 3D noise

2D noise may seem convenient to use for a surface texture, as the common method to give a 3D object its appearance is to map a 2D texture to it. One advantage is that information retrieved by the noise function, as the vector to the closest point, will actually refer to a point on the surface. However, a UV-map would be needed to map the 3D coordinates of the mesh to the corresponding texture pixel. Without the UV-map, there is no suited input to the 2D noise function. Noise in 3D comes with some pros and cons as well. Unlike the 2D version, there are no visible seams due to UV mapping which is good. One notable downside though is that while the noise is a volume, the surface we see is actually just a cross-section. That is, the points we would define as the center points in a cell pattern are (most usually) not actually the actual center points. While the surface is not explicitly defined, approximations can be obtained with the use of linear algebra. That is, the vector to the center should be replaced with the orthogonal component in affected equations. See Algorithm 3 and Figure 8-10.

        // Algorithm 3 - Transformation to surface, code snippet

        float3 p_parallel = dot(vecToCenter, i.normal)*i.normal;
        float3 p_orth = vecToCenter - p_parallel;
      

Method

---

The project included the creation of a procedural cell pattern, the use of bump mapping for the illusion of three-dimensional cells, and a solution to generate colors to camouflage the object to its surroundings.

Procedural pattern

Two versions of the main cell pattern were created. One with classic Voronoi noise and one with Hierarchical Voronoi noise. Both versions can be combined with a second Vononoi noise for additional surface details, and a simplex noise for large color speckles. Adjustable parameters were introduced for cell size ( noise scale ) and level of uniformness, i.e. scale of random displacement of the feature points. Algorithm 4 shows a code snippet where the fragment position is used to retrieve noise data and how it can be used to color the surface. Note the necessity that the positions used in the fragment shader are transformed into local coordinate space, as the chameleon is supposed to move. Figure 11 shows the effect of alternating albedo according to the vector that points to the cell center, as well as the normal - which is further discussed in the next section about bump mapping. More specifically regarding the Hierarchical Voronoi noise, the implementation is heavily based on the code presented by Inigo Quilez [6]. Below are the modifications done for the purpose of this work:

  - from 2D to 3D
  - the level is set to a function input instead of a global preset value
  - a vector to the closest feature point is saved
  - a random number is generated, such that all points within a specific cell has the same number

The shader has two different modes depending on whether the camouflage the effect is on or off. A set of colors is defined to reflect the default appearance of the chameleon. Below are these colors labeled, along with a description of how this value is overridden in camouflage mode:

  - base color, overridden with values from the render texture - sampled with noise values.
  - color spots, overridden by the average value of the current render texture.
  - angle color ( normal dependent color, like top or bottom colors ), overridden by either sampled value or average color, and either lightened or darkened

       // Algorithm 4 - Create noise, code snippet

       float3 objPos = mul(unity_WorldObject, float4(i.worldPos, 1.0)).xyz;
       float3 value = objPos / _CellSize;
       float3 vecToCenter; // to pass by reference
       float3 noise = VoronoiNoise(value, vecToCenter);

       vecToCenter = p_orth; // see code snippet in Algorithm 3
       albedo = lerp(albedo, _BorderColor, _BorderGradientStrength*length(vecToCenter*vecToCenter) );

     

Note:

Before Hierarchical Voronoi noise came to mind, Apollonian sphere packing was given a chance. This algorithm is often implemented recursively and in the time of writing, I can not tell if it is suitable for a 3D shader function. It was successfully implemented as a 2D shader function, however, the pattern was not filling the entire space but rather centered to a small area. As Hierarchical Voronoi was convincingly more convenient, Apollonian sphere packing was discarded.

Procedural Bump Mapping

To create the illusion of three-dimensional cells in the skin, procedural bump mapping is used. As the Voronoi noise function returns a vector to the closest feature point, it can be used to modify the normal as in Algorithm 5. When the light calculation is then performed for this normal, the illusion is created that the surface is directed in a different direction than is actually the case and shadows and highlights are created, see Figure 12-14. Recall to the figures in the section about 2D noise versus 3D noise. The vector used should first be transformed according to Algorithm 3.

      // Algorithm 5 - Procedural bump mapping, code snippet

      // duplicate row and change variables accordingly to add detailed bump mapping
      vecToCenter = p_orth; // see code snippet in Algorithm 3
      i.normal += -vecToCenter*_NormalBump;

      i.normal = normalize(i.normal);
    

Procedural UV-map imitation

To get ahead with this task, it was all about to take advantage of the limited information available in the fragment shader and the limited input parameters. Normals were considered to be a convenient choice, with the intuition that the animal's belly, back, and sides should be possible to separate and that this may be sufficient for many cases. Four user parameters were introduced for this purpose and their usage is shown in Algorithm 6.

User parameters:
1. _Vector: A vector to compare the object's normals with, e.g. to evaluate how much a normal deviate from the positive y-direction.
2. _AngleLim: A parameter that tells how much the deviation should be to be included in this surface area, in the range [0, pi] rad
3. _AngleColor: What color this area should have if it is in this surface area
4. _AngleBlend: A parameter to justify the sharpness of the blending

      // Algorithm 6 - Normal dependent color, code snippet

      float angle = acos(dot(i.normal, _Vector)/(length(i.normal)*length(_Vector)));
      float smooth_lerp = smoothstep(_AngleLim, _AngleLim + _AngleLim/_AngleBlend, abs(angle) );
      albedo = lerp(albedo, _AngleColor, smooth_lerp);
    

Note that these input parameters and code snippets can be duplicated to customize another surface area. Further, another parameter ( _LimY ) was introduced to specify a limit for the surface area, based on the y-position of the fragment. The implementation of this feature would need refinements though, as it's not general enough.

Camouflage effect

The targeted appearance had the following ambitions:
- the colors of the animal should not change with camera orbiting
- surrounding objects should not be obviously recognized in the skin as a mirror surface, rather their colors should be represented in a more abstract way.

    // Algorithm 7 - Render Texture color, code snippet

    float2 noise_uv = float2(noise.y, noise.y);
    float3 camou_color = tex2D(_RenderTexture, noise_uv);
    albedo = lerp(albedo, camou_color, noise_value );

  

Test environment

To test the camouflage effect, a scene was created in Unity with a number of colorful areas, see Figure 15. The user can control the chameleon to move and then activate the camouflage at the desired location. 3D models for the chameleon and most other assets to create an interesting environment are obtained from the Unity Asset Store or other sources under creative commons.

User control.
As described above in the Noise section, the points are regularly spaced and this can be adjusted if a float parameter is first scaling the fragment position. This is done for all of the three noise functions used. A compelling visual result can be achieved by the combination of one larger Voronoi noise as the main pattern, along with a smaller scaled Voronoi noise representing the surface details. Further, a simplex noise adds a pleasant color contribution. Figure 16 shows the available shader properties. The cell pattern and details become clearer as the boundary gradient strength increases and likewise the strength of the bump effect can be tuned. The script Camo Controller was created to switch between default mode and camouflage mod. An Enter keydown event triggers the shader property "Camou On" to toggle on, and it also starts the shader property "Camou Timer" to start a transition. For reference to the implementation, the camouflage timer value is used to lerp between the default colors and the camouflage colors.

Results

---

The final shader succeeds to mimic the colors of the environment and it has a cell pattern like those associated with reptiles. Figures 17a and 17b present the chameleon in the exact same setting, except for that camouflage mode is toggled on in Figure 17b. Figure 18 shows the transition between the two modes. Figure 19 shows how the UV-map imitation can be customized in real-time. Moreover, the shader was tested with other meshes to test its capacity. Figure 20 shows it applied to a frog mesh and in particular comparison with an actual texture mapping. Note that the incentive with work was not to replace UV-mapping as a better option, but rather to develop a solution analogous to it when writing pure procedural shaders. Note that several images are captured with classic Voronoi noise, however, Figure 21 shows the final shader that applies Hierarchical Voronoi noise. Figure series 22a-22f shows how the procedural texture emerges, step by step.

Discussion

A major challenge was to achieve a roundness in the cells, while also have them packed but not intersecting. With the use of classic Voronoi noise, this was somewhat improved when a disturbance was added to the length calculation, as it became less edgy. Hierarchical Voronoi noise accommodates this need with a convincing result, however, the program slows down significantly with 3 levels of hierarchy. To optimize for computational complexity is indeed within the scope of further work if it is to be used in a serious production. This also applies to LOD, as it is crucial that the level of noise details decreases with distances far from the camera. For some settings of the light sources, some flaws of the bump mapping became visible and the cells then appear too pointy. One can imagine that it is possible to fix with, for example, squaring the vector, however, it gave no good results - maybe because the vector already is an approximation. It sure has the potential for improvements.

Also regarding further work, the panel for user parameters can be presented more elegant. Specifically, to take full advantage of the UV-map alternative solution, the UI for this needs to be carefully considered. As soon as the user wants more customized surface areas than one or two, the number of parameters will intimidate the user. One idea to look further into is if the texture2d input type can be used to store all these parameters and be managed from a UI separated from the shader.

References

---

[1] livescience. 2015. Chameleons' Color-Changing Secret Revealed. [ONLINE] Available at: https://www.livescience.com/50096-chameleons-color-change.html . [Accessed 13 January 2020].
[2] medium. 2018. All About Texture Mapping. [ONLINE] Available at: https://medium.com/imeshup/all-about-texture-mapping-8b3447c81cd6 . [Accessed 13 January 2020].
[3] thebookofshaders. Noise. [ONLINE] Available at: https://thebookofshaders.com/11/ . [Accessed 13 January 2020].
[4] prof. Stefan Gustavsson, LiU Simplex noise demystified
http://staffwww.itn.liu.se/~stegu/simplexnoise/simplexnoise.pdf
[5] Worley, Steven. “A cellular texture basis function.” SIGGRAPH '96 (1996).
[6] iquilezles. 2014. A chellenge – hierarchical voronoi. [ONLINE] Available at: https://www.iquilezles.org/blog/?p=3357 . [Accessed 13 January 2020]. | + link to code