If you are gonna use stencil shadows in Ogre, you should carefully read the manual page about this shadowing technique, and be sure to fully understand what is explained there. Besides, you can find some useful hints and tips regarding this topic that I have discovered while adding shadows to my scenes.
- Any object (mesh) casting shadows must contain an edge list, which can be generated either when exporting the mesh in Blender (or any other modelling tool) or calling Mesh::buildEdgeList before the mesh is used in run-time. If you are using Blender, make sure that the option Edge Lists is enabled when exporting your model (like I explained in a previous post).
- Current Ogre release (v1.8.1) has a bug when computing stencil shadows if a light source is placed inside an object. Read this thread. Solutions: a) do not place a light source inside an object (with edge list), b) apply the patch posted in the previous thread to your Ogre sources (this patch will be included in the next maintenance release), c) consider switching to the texture shadows technique.
- Stencil shadows project the object geometry. The shadow caster class (OgreShadowCaster) has a 16-bit index limit on the number of vertexes. This limit will only be shown to the user in debug mode with an assert. In release mode, this limit will not trigger anything, but shadows will not work properly. In addition, the performance may be seriously penalized, if big meshes are used with this technique due to the geometry projection. Therefore, if shadows do not work properly, make sure none of your meshes exceeds the 16-bit limit executing in debug mode. If this requirement is not meet, you can split the mesh in different pieces (that will help Ogre to cull non visible parts), or reduce the number of vertexes.