In order to find an efficient inversion algorithm for 3D helicopter-borne electromagnetic (HEM) data interpretation, we present in this paper two popular nonlinear optimization algorithms-non- linear conjugate gradients (NLCG) and limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) and compare their effectiveness and efficiency. For better recovering the depth of anomalous targets, large smoothing parameters in the model covariance matrix are employed at the beginning of inversion process. When the misfit decrement becomes very slow, small smoothing parameters are used to obtain focusing and accurate results. Numerical results show that these two methods have similar memory requirements, but L-BFGS method performs better than NLCG in terms of time-cost and the model resolution, making the L-BFGS method more suitable for large scale optimization problems. Further model experiments show that the current iterative algorithms may not suit for the inversion of large HEM datasets, technologies like the matrix factorization may be needed in the future for the fast modeling and inversion for airborne EM platform.