Let \(f'\)(x)=(x+cos x) (x-tan x)
By product rule,
f'(x) = (x + cos x)\(\frac{d}{dx}\)(x - tan x)+(x - tan x) \(\frac{d}{dx}\)(x + cos x)
= (x + cos x) [ \(\frac{d}{dx}\)(x) - \(\frac{d}{dx}\)(tan x)] + (x-tan x)(1— sin x)
= (x + cos x)[1-\(\frac{d}{dx}\)tan x]+(x-tan x) (1-sin x) ...(i)
Let g(x) = tan x. Accordingly, g(x+h) = tan(x+h)
By first principle.
g'(x)= \(\lim_{h\rightarrow 0}\) \(\frac{g(x+h)-g(x)}{h}\)
= \(\lim_{h\rightarrow 0}\)\(\frac{tan(x+h)-tan\,x}{h}\)
= \(\lim_{h\rightarrow 0}\) \(\frac{1}{h}\)[\(\frac{sin(x+h)}{cos(x+h)}-sin\,x\frac{h}{cos\,x}\)]
= \(\lim_{h\rightarrow 0}\) \(\frac{1}{h}\) [\(\frac{sin(x+h)cos\,x-sin\,xcos(x+h)}{cos(x+h)cos\,x}\)]
=\(\frac{1}{cos\,x}\)\(\lim_{h\rightarrow 0}\) \(\frac{1}{h}\)[\(\frac{sin(x+h-x)}{cos(x+h)}\)]
=\(\frac{1}{cos\,x}\)\(\lim_{h\rightarrow 0}\) \(\frac{1}{h}\)[\(\frac{sin\,h}{cos(x+h)}\)]
=\(\frac{1}{cos\,x}\) (\(\lim_{h\rightarrow 0}\) \(sin\frac{h}{h}\)).(\(\lim_{h\rightarrow 0}\) \(\frac{1}{cos(x+h)}\))
=\(\frac{1}{cos\,x}\).1.\(\frac{1}{cos\,x}\)(x+0)
=\(\frac{1}{cos^2x}\) = sec2x ...(ii)
Therefore, from (i) and (ii), we obtain
\(f'\)(x)=(x+cosx)(1-sec2x)+(x-tan x) (1-sin x)
=(x+cos.x) (-tan2x)+(x−tan x)(1−sin x)
=−tan2x(x+cos x)+(x−tan x)(1−sin x)