In my textbooks and lectures (I'm a second year physics student) I often come across statements such as $``dV=4\pi r^2dr\text{ can be thought of as the spherical volume element between }r\text{ and }r+dr."$
Is this meant only symbolically, or is it a precise statement? For instance, I get \begin{align}\int_r^{r+dr}4\pi r^2dr&=\frac{4}{3}\pi\left[r^3\right]_r^{r+dr}\\&=\frac{4}{3}\pi(r^3+dr^3+3rdr^2+3r^2dr-r^3)\\&=\frac{4}{3}\pi(dr^3+3rdr^2+3r^2dr)\\&=4\pi r^2dr+\text{increasingly smaller terms.}\end{align}
Moreover, a Taylor expansion of the spherical volume function at a point $r'$ around a general point $r$ gives $V(r')\approx\frac{4}{3}\pi r^3+4\pi r^2(r'-r)+\cdots.$ Since the volume of a spherical shell between $r$ and $r+dr$ is generally (?) the volume of a sphere with radius $r+dr$ minus the volume of a sphere with radius $r$, we get $\text{volume between }r\text{ and }r+dr\approx\frac{4}{3}\pi r^3+4\pi r^2dr-\frac{4}{3}r^3\approx4\pi r^2dr.$
This leads me to think that what $4\pi r^2dr$ really is, is a good approximation (in fact, a first order Taylor expansion) of the volume between $r$ and $r+dr$, which can furthermore be assumed to be perfectly valid in the limit $dr\to 0$.
Maybe ... I'm just being pedantic?