[tex]\begin{gathered} \cos ^2(x)+2cos(x)=5\cos (x)+4 \\ \text{Subtract 5cos(x)-4 from both sides:} \\ \cos ^2(x)-3\cos (x)-4=0 \end{gathered}[/tex][tex]\begin{gathered} Let\colon \\ y=\cos (x) \end{gathered}[/tex]
Rewrite the equation as:
[tex]y^2-3y-4=0[/tex]
The factors of -4 that sum to -3 are -4 and -1, therefore:
[tex]\begin{gathered} (y-1)(y-4)=0 \\ y=1 \\ or \\ y=4 \end{gathered}[/tex]