This paper investigates the statistical behavior of a sequential adaptive gradient search algorithm for identifying an unknown Wiener-Hammerstein system (WHS) with Gaussian inputs. The WHS nonlinearity is assumed to be expandable in a series of orthogonal Hermite polynomials. The sequential procedure uses 1) a gradient search for the unknown coefficients of the Hermite polynomials, 2) an LMS adaptive filter to partially identify the input and output linear filters of the WHS, and 3) the higher order terms in the Hermite expansion to identify each of the linear filters. The third step requires the iterative solution of a set of coupled nonlinear equations in the linear filter coefficients, An alternative scheme is presented if the two filters are known a priori to be exponentially shaped. The mean behavior of the various gradient recursions are analyzed using small step-size approximations (slow learning) and yield very good agreement with Monte Carlo simulations. Several examples demonstrate that the scheme provides good estimates of the WHS parameters for the cases studied.