| NEWS | R Documentation |
change VarCorr method signature (for compatibility
with upstream nlme changes)
several glmer.nb bugs fixed (generally not
changing results, but causing warnings and errors e.g.
during bootstrapping)
fixes to some lmList bugs (Github #320)
minor documentation, vignette updates
minor fix to plot.merMod with id specified
bootMer now handles separate offset term properly
(Github #250)
This release is primarily a version bump for the release of the paper in J. Stat. Software.
updated CITATION file.
We export set of about a dozen printing utility functions
which are used in our print methods.
bootMer now allows the use of re.form.
fixed reordering bug in names of getME(.,"Ztlist")
(terms are reordered in decreasing order of the number of
levels of the grouping variable, but names were not being
reordered)
fixed issue with simulation when complex forms (such as nested random effects terms) are included in the model (Github #335)
explicit maxit arguments for various functions
(refit, mkGlmerDevfun, ...)
terms and formula methods now have
random.only options
getME gains a glmer.nb.theta option.
It is now (an S3) generic with an "merMod" method in
lme4 and potentially other methods in dependent packages.
simulate now works for glmer.nb models
(Github #284: idea from @aosmith16)
prediction and simulation now work when random-effects
terms have data-dependent bases (e.g., poly(.) or
ns(.) terms) (Github #313, Edgar Gonzalez)
logLik for glmer.nb models now
includes the overdispersion parameter in the
parameter count (df attribute)
lmList handles offsets and weights better
lots of fixes to glmer.nb (Github #176, #266, #287,
#318). Please note that glmer.nb is still somewhat
unstable/under construction.
import functions from base packages to pass CRAN checks
tweak to failing tests on Windows
getME gains a "Tlist" option
(returns a vector of template matrices from which the blocks of
Lambda are generated)
hatvalues method returns the diagonal of the hat
matrix of LMMs
nlminbwrap convenience function allows use of
nlminb without going through the optimx package
as.data.frame.VarCorr.merMod gains an order
option that allows the results to be sorted with variances first
and covariances last (default) or in lower-triangle order
allow more flexibility in scales for
xyplot.thpr method (John Maindonald)
models with only random effects of the form 1|f have
better starting values for lmer optimization (Gabor
Grothendieck)
glmer now allows a logical vector as the response
for binomial models
anova will now do (sequential) likelihood ratio
tests for two or more models including both merMod and
glm or lm models (at present, only for GLMMs
fitted with the Laplace approximation)
deviance() now returns the deviance, rather than half the
negative log-likelihood, for GLMMs fitted with Laplace
(the behaviour for LMMs and GLMMs fitted with nAGQ>1 has
not changed)
convergence warning and diagnostic test issues are now
reported in print and summary methods
update now (attempts to) re-evaluate the original fit
in the environment of its formula (as is done with drop1)
refit of a nonlinear mixed model fit now throws an
error, but this will hopefully change in future releases (related
to bug fixes for Github #231)
lmList now returns objects of class lmList4,
to avoid overwriting lmList methods from the recommended
nlme package
names of random effects parameters in confint
changed (modified for consistency across methods);
oldNames=TRUE (default) gives ".sig01"-style names,
oldNames=FALSE gives "sd_(Intercept)|Subject"-style names
confint(.,method="Wald") result now contains rows
for random effects parameters (values set to NA)
as well as for fixed-effect parameters
simulate and predict now work more
consistently with different-length data, differing
factor levels, and NA values (Github #153, #197, #246, #275)
refit now works correctly for glmer
fits (Github #231)
fixed bug in family.merMod; non-default links
were not retrieved correctly (Alessandro Moscatelli)
fixed bootMer bug for type=="parametric",
use.u=TRUE (Mark Lai)
gradient scaling for convergence checks now uses the Cholesky factor of the Hessian; while it is more correct, this will lead to some additional (probably false-positive) convergence warnings
As with lm(), users now get an error for non-finite
(Inf, NA, or NaN)
values in the response unless na.action is set to
exclude or omit them (Github #310)
the nloptr package is now imported;
a wrapper function (nloptwrap) is provided so that
lmerControl(optimizer="nloptwrap") is all that's necessary
to use nloptr optimizers in the nonlinear optimization
stage (the default algorithm is NLopt's implementation
of BOBYQA: see ?nloptwrap for examples)
preliminary implementation of checks for scaling
of model matrix columns (see check.scaleX in
?lmerControl)
beta is now allowed as a synonym for fixef
when specifying starting parameters (Github #194)
the use of deviance to return the REML criterion
is now deprecated; users should use REMLcrit() instead
(Github #211)
changed the default value of check.nobs.vs.rankZ to
"ignore" (Github #214)
change gradient testing from absolute to relative
fix confint(.,method="boot") to allow/work
properly with boot.type values other than "perc"
(reported by Alan Zaslavsky)
allow plot() to work when data are specified in a different
environment (reported by Dieter Menne)
predict and simulate work for matrix-valued
predictors (Github #201)
other simulate bugs (Github #212)
predict no longer warns spuriously when
original response was a factor (Github #205)
fix memory access issues (Github #200)
This version incorporates no changes in functionality, just modifications to testing and dependencies for CRAN/backward compatibility.
change drop1 example to prevent use of old/incompatible
pbkrtest versions, for 2.15.3 compatibility
explicitly require(mlmRev) for tests to prevent cyclic
dependency
bump RcppEigen Imports: requirement from >0.3.1.2.3 to
>=0.3.2.0; Rcpp dependency to >= 0.10.5
improved NA handling in simulate and refit
made internal handling of weights/offset
arguments slightly more robust (Github #191)
handle non-positive-definite estimated fixed effect
variance-covariance matrices slightly more generally/robustly
(fall back on RX approximation, with a warning,
if finite-difference Hessian
is non-PD; return NA matrix if RX approximation is
also bad)
Added output specifying when Gauss-Hermite quadrature was used to fit the model, and specifying number of GHQ points (Github #190)
Models with prior weights returned an incorrect sigma and deviance (Github issue #155). The deviance bug was only a practical issue in model comparisons, not with inferences given a particular model. Both bugs are now fixed.
Profiling failed in some cases for models with vector random effects (Github issue #172)
Standard errors of fixed effects are now computed
from the approximate Hessian by default (see the
use.hessian argument in vcov.merMod); this
gives better (correct) answers when the estimates of
the random- and fixed-effect parameters are correlated
(Github #47)
The default optimizer for lmer fits has been
switched from "Nelder_Mead" to "bobyqa" because we have
generally found the latter to be more reliable. To switch
back to the old behaviour,
use control=lmerControl(optimizer="Nelder_Mead").
Better handling of rank-deficient/overparameterized
fixed-effect model matrices; see check.rankX option
to [g]lmerControl. The default value is
"message+drop.cols", which automatically drops redundant
columns and issues a message (not a warning). (Github #144)
slight changes in convergence checking; tolerances can
be specified where appropriate, and some default tolerances
have changed (e.g., check.conv.grad)
improved warning messages about rank-deficiency in X and Z etc. (warnings now try to indicate whether the unidentifiability is in the fixed- or random-effects part of the model)
predict and simulate now prefer
re.form as the argument to specify which random effects
to condition on, but allow ReForm, REForm, or
REform, giving a message (not a warning) that they are
deprecated (addresses Github #170)
small fixes for printing consistency in models with no fixed effects
we previously exported a fortify function identical
to the one found in ggplot2 in order to be able to define a
fortify.merMod S3 method without inducing a dependency on
ggplot2. This has now been unexported to avoid masking
ggplot2's own fortify methods; if you want to
add diagnostic information to the results of a model, use
fortify.merMod explicitly.
simulate.formula now checks for names associated
with the theta and beta parameter vectors. If
missing, it prints a message (not a warning); otherwise, it
re-orders the parameter vectors to match the internal
representation.
preliminary implementation of a check.scaleX argument
in [g]lmerControl that warns about scaling if some columns
of the fixed-effect model matrix have large standard
deviations (relative to 1, or to each other)
The gradient and Hessian are now computed via finite
differencing after the nonlinear fit is done, and the results
are used for additional convergence tests. Control of the
behaviour is available through the check.conv.* options
in [g]lmerControl. Singular fits (fits with estimated
variances of zero or correlations of +/- 1) can also be tested for,
although the current default value of the check.conv.singular
option is "ignore"; this may be changed to "warning"
in the future. The results are stored in @optinfo$derivs.
(Github issue #120; based on code by Rune Christensen.)
The simulate method will now work to generate
simulations "from scratch" by providing a model formula,
a data frame holding the predictor variables, and a list
containing the values of the model parameters:
see ?simulate.merMod. (Github issue #115)
VarCorr.merMod objects now have an as.data.frame
method, converting the list of matrices to a more
convenient form for reporting and post-processing. (Github issue #129)
results of fitted(), predict(),
and residuals() now have
names in all cases (previously results were unnamed, or
named only when predicting from new data)
the anova method now has a refit argument
that controls whether objects of class lmerMod should be
refitted with ML before producing the anova table.
(Github issues #141, #165; contributed by Henrik Singmann.)
the print method for VarCorr objects
now has a formatter argument for finer control
of standard deviation and variance formats
the optinfo slot now stores slightly more
information, including the number of function evaluations
($feval).
dotplot.ranef.mer now adds titles to sub-plots by default,
like qqmath.ranef.mer
fitted now respects na.action settings (Github
issue #149)
confint(.,method="boot") now works when there are
NA values in the original data set (Github issue #158)
previously, the code stored the results (parameter values, residuals, etc.) based on the last set of parameters evaluated, rather than the optimal parameters. These were not always the same, but were almost always very close, but some previous results will change slightly (Github issue #166)
when using the default method="profile",
confint now returns appropriate upper/lower bounds
(-1/1 for correlations, 0/Inf for standard deviations)
rather than NA when appropriate
in a previous development version, ranef returned
incorrect conditional variances (github issue #148). this is
now fixed
prediction now works when new data have fewer factor levels than are present in the original data (Github issue #143, reported by Rune Haubo)
the existence of a variable "new" in the global environment
would mess lme4 up: reported at
http://stackoverflow.com/questions/19801070/error-message-glmer-using-r-what-must-be-a-character-string-or-a-function
confint.merMod and vcov.merMod are
now exported, for downstream package-author convenience
the package now depends on Matrix >=1.1-0 and RcppEigen >=0.3.1.2.3
new rename.response option for refit (see BUG
FIXES section)
eliminated redundant messages about suppressed fixed-effect correlation matrices when p>20
most inverse-link functions are now bounded where
appropriate by .Machine$double.eps, allowing fitting
of GLMMs with extreme parameter values
merMod objects created with refit did not
work with update: optional
rename.response option added to refit.merMod, to allow
this (but the default is still FALSE, for
back-compatibility) (reported by A. Kuznetsova)
fixed buglet preventing on-the-fly creation of index variables,
e.g. y~1+(1|rownames(data)) (reported by J. Dushoff)
predict now works properly for glmer models
with basis-creating terms (e.g. poly, ns)
step sizes determined from fixed effect coefficient standard
errors after first state of glmer fitting are now bounded,
allowing some additional models to be fitted
refit() now works, again, with lists of
length 1, so that e.g. refit(.,simulate(.)) works.
(Reported by Gustaf Granath)
getME(.,"ST") was returning a list
containing the Cholesky factorizations that get repeated in
Lambda. But this was inconsistent with what ST represents in
lme4.0. This inconsistency has now been fixed and
getME(.,"ST") is now consistent with the definition of the
ST matrix in lme4.0. See
https://github.com/lme4/lme4/issues/111 for more
detail. Thanks to Vince Dorie.
Corrected order of unpacking of standard
deviation/correlation components, which affected results
from confint(.,method="boot"). (Reported by Reinhold
Kliegl)
fixed a copying bug that made refitML()
modify the original model
check.numobs.* and check.numlev.* in
(g)lmerControl have been changed (from recent development
versions) to check.nobs.* and
check.nlev.* respectively, and the default values of
check.nlev.gtreq.5 and check.nobs.vs.rankZ
have been changed to "ignore" and "warningSmall"
respectively
in (g)lmerControl, arguments to the optimizer
should be passed as a list called optCtrl, rather than
specified as additional (ungrouped) arguments
the postVar argument to ranef has been
changed to the (more sensible) condVar ("posterior variance"
was a misnomer, "conditional variance" – short for "variance of the
conditional mode" – is preferred)
the REform argument to predict has been changed
to ReForm for consistency
the tnames function, briefly exported, has been
unexported
getME(.,"cnms") added
print method for merMod objects is now more
terse, and different from summary.merMod
the objective method for the respMod
reference class now takes an optional sigma.sq parameter
(defaulting to NULL) to allow calculation of the
objective function with a residual variance different from
the profiled value (Vince Dorie)
Because the internal computational machinery has changed,
results from the newest version of lme4 will not be numerically
identical to those from previous versions. For reasonably well-
defined fits, they will be extremely close (within numerical
tolerances of 1e-4 or so), but for unstable or poorly-defined fits
the results may change, and very unstable fits may fail when they
(apparently) succeeded with previous versions. Similarly, some fits
may be slower with the new version, although on average the new
version should be faster and more stable. More numerical
tuning options are now available (see below); non-default settings
may restore the speed and/or ability to fit a particular model without
an error. If you notice significant or disturbing changes when fitting
a model with the new version of lme4, please notify the maintainers.
VarCorr returns its results in the same format as before (as a
list of variance-covariance matrices with correlation and stddev
attributes, plus a sc attribute giving the residual standard
deviation/scale parameter when appropriate), but prints them in a
different (nicer) way.
By default residuals gives deviance (rather than Pearson)
residuals when applied to glmer fits (a side effect of matching glm
behaviour more closely).
As another side effect of matching glm
behaviour, reported log-likelihoods from glmer models
are no longer consistent with those from pre-1.0 lme4,
but are consistent with glm; see glmer
examples.
More use is made of S3 rather than S4 classes and methods: one
side effect is that the nlme and lme4 packages are now much more
compatible; methods such as fixef no longer conflict.
The internal optimizer has changed. [gn]lmer now has an
optimizer argument; "Nelder_Mead" is the default for [n]lmer,
while a combination of "bobyqa" (an alternative derivative-free
method) and "Nelder_Mead" is the default for glmer. To use the
nlminb optimizer as in the old version of lme4, you can use
optimizer="optimx" with control=list(method="nlminb") (you will
need the optimx package to be installed and loaded). See
lmerControl for details.
Families in GLMMs are no longer restricted to built-in/hard-
coded families; any family described in family, or following that
design, is usable (although there are some hard-coded families, which
will be faster).
[gn]lmer now produces objects of class merMod rather than
class mer as before.
the structure of the Zt (transposed random effect
design matrix) as returned by getME(.,"Zt"), and the
corresponding order of the random effects vector
(getME(.,"u")) have changed. To retrieve Zt
in the old format, use do.call(Matrix::rBind,getME(.,"Ztlist")).
the package checks input more thoroughly for
non-identifiable or otherwise problematic cases: see
lmerControl for fine control of the test behaviour.
A general-purpose getME accessor method allows
extraction of a wide variety of components of a mixed-model
fit. getME also allows a vector of objects to be returned as
a list of mixed-model components. This has been backported to
be compatible with older versions of lme4 that still produce mer
objects rather than merMod objects. However, backporting is incomplete;
some objects are only extractable in newer versions of lme4.
Optimization information (convergence codes, warnings, etc.)
is now stored in an @optinfo slot.
bootMer provides a framework for obtaining parameter confidence
intervals by parametric bootstrapping.
plot.merMod provides diagnostic plotting
methods similar to those from the nlme package
(although missing augPred).
A predict.merMod method gives predictions;
it allows an effect-specific choice of conditional prediction or prediction at the
population level (i.e., with random effects set to zero).
Likelihood profiling for lmer and glmer results (see
link{profile-methods}).
Confidence intervals by likelihood profiling (default),
parametric bootstrap, or Wald approximation (fixed effects only):
see confint.merMod
nAGQ=0, an option to do fast (but inaccurate) fitting of GLMMs.
Using devFunOnly=TRUE allows the user to extract a deviance
function for the model, allowing further diagnostics/customization of
model results.
The internal structure of [gn]lmer is now more modular, allowing
finer control of the different steps of argument checking; construction
of design matrices and data structures; parameter estimation; and construction
of the final merMod object (see ?modular).
the formula, model.frame, and terms
methods return full versions (including random effect terms and
input variables) by default, but a fixed.only argument
allows access to the fixed effect submodel.
glmer.nb provides an embryonic negative
binomial fitting capability.
Adaptive Gaussian quadrature (AGQ) is not available for multiple and/or non-scalar random effects.
Posterior variances of conditional models for non-scalar random effects.
Standard errors for predict.merMod results.
Automatic MCMC sampling based on the fit turns out to be very difficult
to implement in a way that is really broadly reliable and robust; mcmcsamp
will not be implemented in the near future. See
pvalues for alternatives.
"R-side" structures (within-block correlation and heteroscedasticity) are not on the current timetable.
In a development version, prior weights were not being used properly in the calculation of the residual standard deviation, but this has been fixed. Thanks to Simon Wood for pointing this out.
In a development version, the step-halving component of the penalized iteratively reweighted least squares algorithm was not working, but this is now fixed.
In a development version, square RZX matrices would lead to a
pwrssUpdate did not converge in 30 iterations error. This has been fixed
by adding an extra column of zeros to RZX.
Previous versions of lme4 provided
the mcmcsamp function, which efficiently generated
a Markov chain Monte Carlo sample from the posterior
distribution of the parameters, assuming flat (scaled
likelihood) priors. Due to difficulty in constructing a
version of mcmcsamp that was reliable even in
cases where the estimated random effect variances were
near zero (e.g.
https://stat.ethz.ch/pipermail/r-sig-mixed-models/2009q4/003115.html),
mcmcsamp has been withdrawn (or more precisely,
not updated to work with lme4 versions >=1.0).
Calling glmer with the default gaussian family
redirects to lmer, but this is deprecated
(in the future glmer(...,family="gaussian") may
fit a LMM using the penalized iteratively reweighted least squares
algorithm). Please call lmer directly.
Calling lmer with a family argument redirects
to glmer; this is deprecated. Please call glmer directly.
The underlying algorithms and representations for all the mixed-effects models fit by this package have changed - for the better, we hope. The class "mer" is a common mixed-effects model representation for linear, generalized linear, nonlinear and generalized nonlinear mixed-effects models.
ECME iterations are no longer used at all, nor are analytic gradients. Components named 'niterEM', 'EMverbose', or 'gradient' can be included in the 'control' argument to lmer(), glmer() or nlmer() but have no effect.
PQL iterations are no longer used in glmer() and nlmer(). Only the Laplace approximation is currently available. AGQ, for certain classes of GLMMs or NLMMs, is being added.
The 'method' argument to lmer(), glmer() or nlmer() is deprecated. Use the 'REML = FALSE' in lmer() to obtain ML estimates. Selection of AGQ in glmer() and nlmer() will be controlled by the argument 'nAGQ', when completed.
The representation of mixed-effects models has been dramatically changed to allow for smooth evaluation of the objective as the variance-covariance matrices for the random effects approach singularity. Beta testers found this representation to be more robust and usually faster than previous versions of lme4.
The mcmcsamp function uses a new sampling method for the variance-covariance parameters that allows recovery from singularity. The update is not based on a sample from the Wishart distribution. It uses a redundant parameter representation and a linear least squares update.
CAUTION: Currently the results from mcmcsamp look peculiar and are probably incorrect. I hope it is just a matter of my omitting a scaling factor but I have seen patterns such as the parameter estimate for some variance-covariance parameters being the maximum value in the chain, which is highly unlikely.
The 'verbose' argument to lmer(), glmer() and nlmer() can be used instead of 'control = list(msVerbose = TRUE)'.