Thermal noise in the mirror coatings limits the accuracy of today’s most optical precision measurement experiments. Unlike the more commonly discussed thermal phase noise, the crystalline coating can generate thermal birefringent noise due to its anisotropic nature. In this study, we propose that the nondiagonal anisotropic photoelastic effect induced by the Brownian motion of mirror coating layers may contribute to this noise. Employing a standard model for the coating surface, we calculate the spectrum of the nondiagonal anisotropic Brownian photoelastic noise to be 1.2 × 10−11p63f−1/2/Hz1/2. Further experiments are warranted to validate the influence of this effect and reduce its uncertainty. Our findings highlight that for high-precision experiments involving optical resonant cavities targeting signals imprinted in optical polarizations, this noise could emerge as a limiting factor for experimental sensitivity.