Let $G$ be a $p$-group of order $p^n$ and $Phi$=$Phi(G)$ be the Frattini subgroup of $G$. It is shown that the nilpotency class of $Autf(G)$, the group of all automorphisms of $G$ centralizing $G/ Fr(G)$, takes the maximum value $n-2$ if and only if $G$ is of maximal class. We also determine the nilpotency class of $Autf(G)$ when $G$ is a finite abelian $p$-group.