Answer: [tex]\frac{\sqrt[4]{10xy^3}}{2y}[/tex]
where y is positive.
The 2y in the denominator is not inside the fourth root
==================================================
Work Shown:
[tex]\sqrt[4]{\frac{5x}{8y}}\\\\\\\sqrt[4]{\frac{5x*2y^3}{8y*2y^3}}\ \ \text{.... multiply top and bottom by } 2y^3\\\\\\\sqrt[4]{\frac{10xy^3}{16y^4}}\\\\\\\frac{\sqrt[4]{10xy^3}}{\sqrt[4]{16y^4}} \ \ \text{ ... break up the fourth root}\\\\\\\frac{\sqrt[4]{10xy^3}}{\sqrt[4]{(2y)^4}} \ \ \text{ ... rewrite } 16y^4 \text{ as } (2y)^4\\\\\\\frac{\sqrt[4]{10xy^3}}{2y} \ \ \text{... where y is positive}\\\\\\[/tex]
The idea is to get something of the form [tex]a^4[/tex] in the denominator. In this case, [tex]a = 2y[/tex]
To be able to reach the [tex]16y^4[/tex], your teacher gave the hint to multiply top and bottom by [tex]2y^3[/tex]
For more examples, search out "rationalizing the denominator".
Keep in mind that [tex]\sqrt[4]{(2y)^4} = 2y[/tex] only works if y isn't negative.
If y could be negative, then we'd have to say [tex]\sqrt[4]{(2y)^4} = |2y|[/tex]. The absolute value bars ensure the result is never negative.
Furthermore, to avoid dividing by zero, we can't have y = 0. So all of this works as long as y > 0.