Problem

Short solution to a simple problem I came across. I downloaded a ready made effect from the Asset store and found a gray scale texture with no easing before the edge. I'm not sure how to describe it so a picture might be the best example.

As can be seen here, when rendered with transparency the edges of this texture are very clearly visible against the black background because the texture is drawn all the way to the edges. If used in a particle system it will look very obvious.


The best solution

If you are creating the textures yourself, the solution to this problem is to not draw all the way to the edges if the images. But that's of course not what this blog entry is about.


Fixing it in the shader

The goal is to find a way to solve the issue of softening the edges of the image in the shader instead. As mentioned above, it would be simpler and more efficient, compute power wise, to just go back and fix the source image but that is not always possible.


The source for this effect is an Unlit Shader.

The

fixed4 soften(fixed4 color)
function is left empty for now as is where the different solutions will be implemented.

Shader "Demonstration/SoftAlphaClipping"
{
   Properties
   {
       _MainTex ("Texture", 2D) = "white" {}
      _Softness("Softness", Range(0, 1)) = 0
   }
   SubShader
   {
      Tags {
         "Queue" = "Transparent"
         "IgnoreProjector" = "True"
         "RenderType" = "Transparent"
         "ForceNoShadowCasting" = "True"
      }

       LOD 100

       Pass
       {
         Blend SrcAlpha OneMinusSrcAlpha
         Fog { Mode Off }
         ZWrite Off

           CGPROGRAM
           #pragma vertex vert
           #pragma fragment frag

           #include "UnityCG.cginc"

           struct appdata
           {
               float4 vertex : POSITION;
               float2 uv : TEXCOORD0;
           };

           struct v2f
           {
               float2 uv : TEXCOORD0;
               float4 vertex : SV_POSITION;
           };

           sampler2D _MainTex;
           float4 _MainTex_ST;
         fixed _Softness;

           v2f vert (appdata v)
           {
               v2f o;
               o.vertex = UnityObjectToClipPos(v.vertex);
               o.uv = TRANSFORM_TEX(v.uv, _MainTex);
               return o;
           }

         fixed4 soften(fixed4 color)
         {
            // TODO: Implement soften function here
            return color;
         }

           fixed4 frag (v2f i) : SV_Target
           {
               fixed4 col = tex2D(_MainTex, i.uv);
            col = soften(col);
            return col;
           }
           ENDCG
       }
   }
}


Add alpha clipping

First I tried just alpha clipping to the shader based on the

_Softness
variable. Clip is a function in CG that discards pixels from being written to the back buffer entirely if any parts of its input is <= 0.
clip(color.a - _Softness)
makes the
_Softness
variable act as a clipping threshold. Any pixel with an original alpha value <=
_Softness
will be discarded.

fixed4 soften(fixed4 color)
{
 clip(color - _Softness);
 return color;
}


The result is a good start but it leaves a sharp edge where pixels have been "clipped". The issue here is one of contrast. The color space becomes squished as

_Softness
increases. In the end only very light colors remain.



Subtract from the color

In this attempt I subtracted _Softness from the value of color.

fixed4 soften(fixed4 color)
{
 color -= _Softness;
 return color;
}

This looks better because there is no sharp edge but instead the entire image becomes darker and darker the as

_Softness

 increases. The issue here is still one of contrast but the exact opposite from when using clip. The color space is squished between black and a progressively darker shade of gray until, at the very end, only the very dark colors remain.



Alpha clipping and color range normalization

This solution starts with clip like the first solution but also normalizes the remaining pixels down into a color range of 0 -> 1. This means that the darkest color remaining after clip are still black and the lightest remains white and hence the contrast issues are solved.


Consider a

_Softness
of 0.5. The normalization of values can then be calculated as
color = (color - 0.5) * (1 / 0.5)
.

fixed4 soften(fixed4 color)
{
 clip(color.a - _Softness);
 fixed factor = 1 / (1 - _Softness);
 color.a -= _Softness;
 color.a *= factor;
 color.a *= color.a;
 return color;
}


This code is not that easy to read and its also redundant. There is a ready made cg function to handle the color range correction for us named smoothstep.

fixed4 soften(fixed4 color)
{
  color = smoothstep(_Softness, 1, color);
  return color;
}


Finally the output value is squared (

color.a^2
). This step is not necessary but it increases the contrast quite a lot.


Conclusion

So there we have a simple solution to, maybe, trivial problem. The main thing to bring with you from this is color space normalization as it can come in handy in a lot of situations.