Let [tex]x[/tex] be some angle so that [tex]\tan x=-\dfrac35[/tex].
Recall that
[tex]\sin^2x+\cos^2x=1\implies\dfrac{\sin^2x}{\cos^2x}+\dfrac{\cos^2x}{\cos^2x}=\dfrac1{\cos^2x}\implies\tan^2x+1=\sec^2x[/tex]
[tex]\implies\sec x=\pm\sqrt{1+\tan^2x}=\pm\dfrac{\sqrt{34}}5[/tex]
The identity above gives us two possible answers, but they can't both be right: [tex]\dfrac{\sqrt{34}}5\neq-\dfrac{\sqrt{34}}5[/tex]. The caveat is that [tex]\arctan[/tex] and [tex]\tan[/tex] are not mutual inverse functions. While
[tex]\arctan y=x\implies y=\tan x[/tex]
is true, the converse is not. The [tex]\arctan[/tex] function can only return values between [tex]-\dfrac\pi2[/tex] and [tex]\dfrac\pi2[/tex].
To find out which value is correct, we have to use what we know about the [tex]\tan[/tex] function. [tex]\tan x[/tex] is negative only when [tex]-\dfrac\pi2<x<0[/tex]; over this domain, we expect to have [tex]\cos x[/tex] be positive, which in turn means [tex]\sec x[/tex] should be positive.
So we have
[tex]\sec\left(\arctan\left(-\dfrac35\right)\right)=\sec x=+\dfrac{\sqrt{34}}5[/tex]
