Normals and Transforms

Transforms are where otherwise correct intersection code starts producing black acne and inverted lighting. The points moved. The normals did not move correctly.

Points, directions, and normals are different geometric objects under a matrix. Treat them as such.

Under non-uniform scale, normals use n' = normalize((M⁻¹)ᵀ n). Directions do not.

Transform rules

Let M be object-to-world and M_linear its linear 3×3 part.

Quantity	Transform
Point	p' = M p
Direction	d' = M_linear d
Normal	n' = normalize((M_linear⁻¹)ᵀ n)

Why the special normal rule? A normal is defined by orthogonality to surface tangents. Applying the same matrix used for positions destroys that relationship under non-uniform scale. The inverse transpose restores n · t = 0 for transformed tangents.

Uniform scale is forgiving. Non-uniform scale and shear are not. Rotation alone is easy. Scaling only X or only Y is where beginners ship lighting bugs.

Ray in object space vs object in world space

Two valid strategies:

1. Transform geometry into world space once and intersect there.
2. Keep geometry local, transform the ray into object space, intersect, map the hit back.

Instancing prefers (2). Many instances share one mesh and carry different matrices.

o_local = M⁻¹ o_world
d_local = M_linear⁻¹ d_world

Do not casually normalize d_local if you still need a world-space t without conversion. Either keep a careful metric conversion or recompute:

p_world = M p_local
n_world = normalize((M_linear⁻¹)ᵀ n_local)

Cache (M_linear⁻¹)ᵀ on the instance. Inverting per hit is wasteful and an excellent source of NaNs on singular matrices.

Front faces after transforms

A negative determinant flips handedness. A front face may become a back face. Evaluate front_face from the final world-space normal and ray direction after transformation. Doing it earlier is wrong for scaled instances and ruins refraction side tests.

Skinning and animation

Skinning blends several matrices. Normals still require the inverse transpose of the blended linear part, or an equivalent carefully constructed normal matrix. Shortcuts appear as seams when lights move.

For rigid motion, normals may ride the rotation. As soon as authoring tools apply non-uniform scale, the full normal matrix path must exist in code, not in a comment.

Debug sequence

1. Render world-space normals as RGB.
2. Render front_face as white/black.
3. Move the light, not the camera, and watch the terminator.
4. Apply rotation only, then translation only, then non-uniform scale.

When step 4 breaks, stop editing the BRDF.

Common errors

• Applying translation to normals
• Forgetting to renormalize after the inverse transpose
• Mixing object-space normals with world-space lights
• Sharing one inverse matrix across instances
• Assuming the UI scale was uniform because the handles looked even

When lighting looks half inverted, inspect normals before materials. The normal image is the argument settler.

Determinant sign

if det(M_linear) < 0:
    handedness flips

Mirrored instances are common in level design. If your front-face test ignores the determinant, one tree in a mirrored row will light inside-out. Detect mirrors explicitly or always evaluate front face in final world space.

Normal matrix cache

Store both M and N = (M_linear⁻¹)ᵀ on each instance at creation or when the transform dirty flag is set. Do not invert in the inner hit loop. Singular or near-singular scales should fail loudly during update, not during rendering as silent NaNs in the frame buffer.