Answer:
n = 9
Step-by-step explanation:
Let's first prove that for any constants k > 0, [tex]n\geq 1[/tex]
[tex]\lim_{x\to \infty}\frac{\log x^k}{x^n}=0[/tex]
The derivative
[tex](\log (x^k))'=\frac{kx^{k-1}}{x^k}=\frac{k}{x}[/tex]
and the derivative [tex](x^n)' = nx^{n-1}[/tex]
Now, applying L'Hôpital's rule we find that
[tex]\lim_{x\to \infty}\frac{\log x^k}{x}=\lim_{x\to \infty}\frac{k}{nx^n}=0[/tex]
Now, let f be the function
[tex]f(x)=x^8log(x^3)+x^6log(x^5)[/tex]
It is easy to see that f(x) is [tex]O(x^n)[/tex] only if [tex]n\geq 9[/tex]
If [tex]n\geq 9[/tex]
[tex]\frac{f(x)}{x^n}=\frac{x^8log(x^3)+x^6log(x^5)}{x^n}=\frac{log(x^3)}{x^{n-8}}+\frac{log(x^5)}{x^{n-6}}[/tex]
but both n-8 and n-6 are greater than one, so
[tex]\lim_{x \to \infty}\frac{f(x)}{x^n}=0[/tex]
and f is [tex]O(x^n)[/tex]
On the other hand, if [tex]n \leq 8[/tex] then
[tex]\frac{f(x)}{x^n}=x^{8-n}log(x^3)+x^{6-n}log(x^5)[/tex]
but 8-n is greater or equal than one, so
[tex]\lim_{x \to \infty}\frac{f(x)}{x^n}=\infty[/tex]
and so f(x) in not [tex]O(x^n)[/tex]