The purpose of this paper is to find some conditions under which the tangent bundle TM has a dualistic structure. Then, we introduce infinitesimal affine transformations on statistical manifolds and investigate these structures on a special statistical distribution and the tangent bundle of a statistical manifold too. Moreover, we also study the mutual curvatures of a statistical manifold M and its tangent bundle TM and we investigate their relations. More precisely, we obtain the mutual curvatures of well-known connections on the tangent bundle TM (the complete, horizontal, and Sasaki connections) and we study the vanishing of them.