Question

use the divergence theorem to calculate the flux of f across s, where f=zi yj zxk and s is the surface of the tetrahedron enclosed by the coordinate planes and the plane

Answer 1

I guess you're missing some plus signs, so that

$\vec f(x, y, z) = z \, \vec\imath + y \, \vec\jmath + zx \, \vec k$

You're also missing the plane that defines the tetrahedron, but no big deal; I'll use the general equation $ax+by+cz = 1$, where each of a, b, and c are positive. Then S is the boundary of the region R, where

$R = \left\{ (x, y, z) : 0 \le x \le \dfrac1a \text{ and } 0 \le x \le \dfrac{1-ax}b \text{ and } 0 \le z \le \dfrac{1-ax-by}c \right\}$

By the divergence theorem, the flux of $\vec f$ across S is

$\displaystyle \iint_S \vec f \cdot d\vec s = \iiint_R \mathrm{div}(\vec f) \, dV$

We have

$\mathrm{div}(\vec f) = \dfrac{\partial(z)}{\partial x} + \dfrac{\partial(y)}{\partial y} + \dfrac{\partial(zx)}{\partial z} = 1 + x$

Then the flux is equal to the volume integral,

$\displaystyle \iiint_R (1 + x) \, dV = \int_0^{1/a} \int_0^{(1-ax)/b} \int_0^{(1 - ax - by)/c} (1 + x) \, dz \, dy \, dx \\\\ = \frac1c \int_0^{1/a} \int_0^{(1-ax)/b} (1 + x) (1 - ax - by) \, dy \, dx \\\\ = \frac1{2bc} \int_0^{1/a} (1 + x) (1 - ax)^2 \, dx \\\\ = \frac1{6abc} \left(\frac1{4a} + 1\right) = \boxed{\frac{1+4a}{24a^2bc}}$