At first note the following :
[tex]\begin{gathered} a^{-m}=\frac{1}{a^m} \\ (a^m)^n=a^{m\cdot n} \\ \frac{a^m}{a^n}=a^{m-n} \\ a^m\cdot a^n=a^{m+n} \end{gathered}[/tex]
So, for the given expression :
[tex]\begin{gathered} (\frac{5\cdot a^{-1}\cdot b^3}{3\cdot b^{-3}\cdot a^{-5}})^{-2} \\ \\ =(\frac{5}{3}\cdot a^{-1--5}\cdot b^{3--3})^{-2} \\ \\ =(\frac{5}{3}\cdot a^4\cdot b^6)^{-2} \\ \\ =(\frac{5}{3})^{-2}\cdot(a^4)^{-2}\cdot(b^6)^{-2} \\ \\ =(\frac{3}{5})^2\cdot a^{-8}\cdot b^{-12} \\ \\ =\frac{9}{25}\cdot\frac{1}{a^8}\cdot\frac{1}{b^{12}} \\ \\ \\ =\frac{9}{25a^8b^{12}} \end{gathered}[/tex]
