Autoload Manual
Table of Contents
[in package AUTOLOAD]
1 Links and Systems
Here is the official repository and the HTML documentation for the latest version.
-
- Version: 0.1.0
- Description: An ASDF autoloading facility. See Autoload Manual.
- Licence: MIT, see COPYING.
- Author: Gábor Melis
- Mailto: mega@retes.hu
- Homepage: https://github.com/melisgl/autoload
- Bug tracker: https://github.com/melisgl/autoload/issues
- Source control: GIT
- Depends on: closer-mop, mgl-pax-bootstrap
-
- Description: Parts of the Autoload library that depend on
mgl-paxare in this system to avoid the circular dependencies that would arise becausemgl-paxdepends onautoload. Note thatmgl-pax/navigateandmgl-pax/documentdepend on this system, which renders most of this an implementation detail. - Depends on: autoload, dref, mgl-pax, named-readtables, pythonic-string-reader
- Description: Parts of the Autoload library that depend on
2 Introduction
Libraries often choose to limit dependencies, even if it means sacrificing features or duplicating code, to minimize
compilation time,
memory usage in deployment, and
the risk of breakage through dependencies.
This library mitigates the first two issues by loading heavy dependencies on demand. The core idea is
(defmacro autoload (name asdf-system)
`(defun ,name (&rest args)
(asdf:load-system ,asdf-system)
(apply ',name args)))
Suppose we have a library called my-lib that autoloads
my-lib/full. In my-lib, we could use autoload
as
(autoload foo "my-lib/full")
and have
(defun foo (x)
"doc"
(1+ x))
in my-lib/full.
However, manually keeping the loaddefs (e.g. the autoload form
above) in sync with the definitions is fragile, so we introduce the
defun/auto autodef to mark autoloaded functions in the
my-lib/full system:
(defun/auto foo (x)
"doc"
(1+ x))
To generate loaddefs, we add a few lines to the system definitions:
(asdf:defsystem "my-lib"
:defsystem-depends-on ("autoload")
:class "autoload:autoload-system"
:auto-depends-on ("my-lib/full")
:auto-loaddefs "loaddefs.lisp"
:components ((:file "loaddefs")
...))
(asdf:defsystem "my-lib/full"
:defsystem-depends-on ("autoload")
:class "autoload:autoload-system"
:components (...))
Then, the loaddefs can be extracted:
(extract-loaddefs "my-lib")
=> ((autoload foo "my-lib/full" :arglist "(x)" :docstring "doc"))
This is implemented by loading the :auto-depends-on of my-lib and
recording defun/autos. extract-loaddefs is a low-level utility used
by record-loaddefs, which writes its results to the
system's :auto-loaddefs, "loaddefs.lisp" in the above example. So,
all we need to do is call record-loaddefs to regenerate the loaddefs
file:
(record-loaddefs "my-lib")
To prevent the loaddefs file from getting out of sync with the
definitions, asdf:test-system calls check-loaddefs by default.
ASDF, and by extension Quicklisp, doesn't know about the declared
:auto-depends-on, so (ql:quickload "my-lib") does not install the
autoloaded dependencies. They can be installed manually with
(autodeps "my-lib" :installer #'ql:quickload)
If all the autoloaded dependencies are installed, one can eagerly load them to ensure that autoloading is not triggered later (e.g. in deployment):
(map nil #'asdf:load-system (autodeps "my-lib"))
Other Features
Autoloading is not only for Functions:
Autoloading Classes at the time of their first instantiation is supported.
Variables can be marked for early definition and have their initial values assigned if the initial value form provably doesn't have dependencies. If that's not the case, subject to platform support, the definition in the loaded system injects a global binding even in the presence of local bindings.
Multiple Packages can have their final states and interdependencies reconstructed before loading their systems even if they were mutated operations like
importandexport.
3 Basics
-
A loaddef is a preliminary definition that serves as a stand-in until the fully-realized implementation is loaded. Accessing it may or may not load a system. See
loaddef-function-p,loaddef-class-p,loaddef-variable-pandloaddef-package-p.
-
An autodef (e.g.
(defun/auto name ...)) performs the job of its plain counterpart (defun). In addition, it marks the definition (ofnameas a function) for Automatically Generating Loaddefs and, at the time of the first such autodef, it signals anautoload-warningifnamehas not been declared as a loaddef (has never beenloaddef-function-p). Seedefun/auto,defclass/auto,defvar/autoanddefpackage/auto.
3.1 Loading Systems
Function and class loaddefs trigger the loading of asdf:systems.
Unlike normal ASDF dependencies (declared in :depends-on), autoload
dependencies (which may be declared in :auto-depends-on) are allowed
to be circular. The rules for loading are as follows.
It is an
autoload-errorif loading is triggered during compile time or during aloadof either a source file or a compiled file. This is to prevent infinite autoload recursion.It is an
autoload-errorif the system does not exist.The system is loaded under
with-compilation-unit:overridetandwith-standard-io-syntaxbut with*print-readably*nil. Other non-portable measures may be taken to standardize the dynamic environment. Errors signalled during the load are not handled or resignalled by the Autoload library.It is an
autoload-errorif the loaddef is not replaced by a normal definition or deleted by the loaded system, that is, when it remains a loaddef (e.g. in terms ofloaddef-function-p).
3.2 Conditions
[condition] autoload-error error
Signalled for some failures during Loading Systems.
[condition] autoload-warning simple-warning
See
autoload, autodef and:auto-depends-onfor when this is signalled.
3.3 Functions
[macro] autoload name system-name &key (arglist nil) docstring
This is the loaddef for autodef
defun/auto.Define a function stub with
nameand returnname. The arguments are not evaluated. Ifnamehas anfdefinitionand it is notloaddef-function-p, then this does nothing and returnsnil.The stub first loads
system-name, then itapplys the functionnameto the arguments originally provided to the stub.The stub is not defined at compile time, which matches the required semantics of
defun.nameisdeclaimed withftypefunctionandnotinline.arglistwill be installed as the stub's arglist if specified and it's supported on the platform (currently only SBCL). Ifarglistis a string, the effective value ofarglistis read from it. If the read fails, anautoload-warningis signalled and processing continues as ifarglisthad not been provided.Arglists are for interactive purposes only. For example, they are shown by SLIME autodoc and returned by
dref:arglist.docstring, if non-nil, will be the stub's docstring. Ifnil, then a generic docstring that says what system it autoloads will be used.
When
autoloadis macroexpanded during the compilation or loading of anautoload-system, it signals anautoload-warningifsystem-nameis not among those declared in:auto-depends-on.
[function] loaddef-function-p name
See if an
autoloadfornamewas established, and since then it has not been redefined (e.g. withdefun/auto,defun) or madefmakunbound.
[macro] defun/auto name lambda-list &body body
This is the autodef for the loaddef
autoload.Like
defun, but also silence redefinition warnings.namemay be of the form(definer name). In that case, instead ofdefun,defineris used to establish the underlying function binding.Loaddef: The corresponding loaddef is an
autoloadform.extract-loaddefswithprocess-arglisttinstallslambda-listas thearglist. Ifprocess-arglistisnil, thenarglistwill not be passed toautoload.
[macro] defgeneric/auto name lambda-list &body options
A shorthand for
(defun/auto(defgeneric name) ...).
3.4 Classes
[macro] autoload-class class-name system-name &key docstring (metaclass 'standard-class)
This is the loaddef for autodef
defclass/auto.Define a dummy class with
class-nameand arrange forsystem-nameto be loaded when the class or any of its subclasses are instantiated. Returns the class object. The arguments are not evaluated. Ifclass-namedenotes aclassand it is notloaddef-class-p, then it does nothing and returnsnil.When
autoload-classis macroexpanded during the compilation or loading of anautoload-system, it signals anautoload-warningifsystem-nameis not among those declared in:auto-depends-on.docstring, if non-nil, will be the stub's docstring. Ifnil, then a generic docstring that says what system it autoloads will be used.metaclassis a symbol denotingstandard-classor a subclass of it. Also, classes with this metaclass must be allowed to inherit from standard classes. In MOP terms,closer-mop:validate-superclassmust return true when called with an instance ofmetaclassand an instance ofstandard-class.
The dummy class is also defined at compile time to approximate the semantics of
defclass. It class hasmetaclasswith a single superclass and no slots. These are visible through introspection (e.g. viacloser-mop:class-direct-superclasses), which does not trigger autoloading.Note that
initialize-instance:aroundmethods specialized on a subclass ofclass-namemay run twice in the context of themake-instancethat triggers autoloading.
[function] loaddef-class-p name
See if an
autoload-classfornamewas established, and since then it has not been redefined (e.g. withdefclass/auto,defclass) or deleted (with(setf (find-class ...) nil)). Subclasses do not inherit this property.
[macro] defclass/auto name direct-superclasses direct-slots &rest options
This is the autodef for the loaddef
autoload-class.Like
defclass.namemay be of the form(definer name). In that case, instead ofdefclass,defineris used to establish the underlying class definition.Loaddef: The corresponding loaddef is an
autoload-classform. Note that the metaclass of the classnamemust already be defined when the loaddef is evaluated.
3.5 Variables
[function] loaddef-variable-p name
See if a loaddef was generated from a
defvar/autoforname, but this autodef has not been evaluated.
[macro] defvar/auto var &optional (val nil) doc
This is an autodef with no public loaddef. See below.
Unlike
defvar, this works with the global binding on Lisps that support it (currently Allegro, CCL, ECL, SBCL). This is to handle the case when a system that usesdefvarwith a default value is autoloaded while that variable is locally bound:;; Some base system only foreshadows *X*. (declaim (special *x*)) (let ((*x* 1)) ;; Imagine that the system that defines *X* is autoloaded here. (defvar/auto *x* 2) *x*) => 1Loaddef: The corresponding loaddef is not public and must be generated. The generated loaddef declaims the variable special and maybe sets its initial value and docstring. If the initial value form in
defvar/autois detected as a simple constant form, then it is evaluated and its value is assigned to the variable as indefvar. Simple constant forms are strings, numbers, characters, keywords, constants in the CL package, andquoted nested lists containing any of the previous or any symbol from theclpackage and other packages for which loaddefs have been generated in the sameextract-loaddefscall (seedefpackage/auto)).In case the global binding of
varhas been set between the corresponding loaddef and its first autodef,valis evaluated for side effect.
3.6 Packages
[function] loaddef-package-p name
See if a loaddef was generated from a
defpackage/autoforname, but this autodef has not been evaluated nor has the package been deleted.
[macro] defpackage/auto name &rest options
This is an autodef with no public loaddef. See below.
Unlike
defpackage, if the package is already defined,defpackage/autoextends it additively. The additivity means that instead of replacing the package definition or signalling errors on redefinition, it expands into individual package-altering operations such asshadow,use-packageandexport. This allows the package state to be built incrementally, but it also means that the(definer name)syntax cannot be supported.defpackage/autois idempotent.In addition,
defpackage/autodeviates fromdefpackagein the following ways.The default
:uselist is empty.:sizeis not supported.Implementation-specific extensions such as
:local-nicknamesare not supported. Useadd-package-local-nicknameafter thedefpackage/auto.
Loaddef: The corresponding loaddef is not public and must be generated. As in the expansion of
defpackage/autoitself, the generated operations are additive.The generated loaddef reconstructs the package states as they exist after all
:auto-depends-onsystems are loaded. Thus, manual modifications after thedefpackage/autodefinition (e.g. by additionalexports) are reflected in the loaddef.To handle circular dependencies, the loaddefs of all autodef packages and those passed in the
packagesargument toextract-loaddefsare generated in an interleaved manner. First, all packages are created, then their state is reconstructed in phases followingdefpackage.Any reference to non-existent packages (e.g. in
:use) or symbols in non-existent packages (e.g.:import-from) is silently skipped when the loaddef is evaluated.
Instead of
defpackage/auto, one may use, for example,defpackageoruiop:define-packageand arrange for Automatically Generating Loaddefs for the package by listing it in:packagesof:auto-loaddefs.
4 ASDF Integration
[class] autoload-system asdf/system:system
Inheriting from this class in an
asdf:defsystemform enables the features documented in the reader methods. Consider the following example.(asdf:defsystem "my-system" :defsystem-depends-on ("autoload") :class "autoload:autoload-system" :auto-depends-on ("dyndep") :auto-loaddefs "loaddefs.lisp" :components ((:file "package") (:file "loaddefs") ...))With the above,
It is an error if an
autoloadrefers to a system other thandyndep.(record-loaddefs"my-system")will updateloaddefs.lisp.(asdf:test-system "my-system")checks thatloaddefs.lispis up-to-date.
If the package definitions are also generated with
record-loaddefs(e.g. because there is adefpackage/autoindyndepor:auto-loaddefsspecifies:packages), then we can do without thepackage.lispfile:(asdf:defsystem "my-system" :defsystem-depends-on ("autoload") :class "autoload:autoload-system" :auto-depends-on ("dyndep") :auto-loaddefs ("loaddefs.lisp" :packages #:my-pkg) :components ((:file "loaddefs") ...))
[class] autoload-cl-source-file asdf/lisp-action:cl-source-file
This is the
:default-component-classofautoload-system. ASDF Integration relies on source files belonging to this class. When combining autoload with another ASDF extension that has its ownasdf:cl-source-filesubclass, define a new class that inherits from both, and use that as:default-component-class.
[reader] system-auto-depends-on autoload-system (:auto-depends-on = nil)
This is the list of the names of systems that this system may autoload. The names are canonicalized with
asdf:coerce-name. It is anautoload-warningif a loaddef refers to a system not listed here. This is also used byextract-loaddefsand affects the checks performed by Loading Systems.
[reader] system-auto-loaddefs autoload-system (:auto-loaddefs = nil)
When non-
nil, this specifies arguments for Automatically Generating Loaddefs. It may be a single pathname designator or a list of the form(loaddefs-file &key (process-arglist t) (process-docstring t) packages (test t))loaddefs-filedesignates the pathname whererecord-loaddefswrites the extracted loaddefs. The pathname is relative toasdf:system-source-directoryofsystemand isopened with:if-exists:supersede.process-arglist,process-docstringandpackagesare passed on byrecord-loaddefstoextract-loaddefs.If
test, thencheck-loaddefsis run byasdf:test-system.
Conditions signalled while ASDF is compiling or loading the file given have a
record-loaddefsrestart.
[function] autodeps system &key (cross-autoloaded t) installer
Return the list of system names that may be autoloaded by
systemor any of its direct or indirect dependencies. This recursively visits systems in the dependency tree, traversing both normal (:depends-on) and autoloaded (:auto-depends-on) dependencies. It works even ifsystemis not anautoload-system.cross-autoloadedcontrols whether systems only reachable fromsystemvia intermediate autoloaded dependencies are visited. Thus, ifcross-autoloadedisnil, then the returned list is the first boundary of autoloaded systems.If
installeris non-nil, it is called when an autoloaded system that is not installed (i.e.asdf:find-systemfails) is visited.installeris passed a single argument, the name of the system to be installed. It may or may not install the system.
If an autoloaded system is not installed (i.e.
asdf:find-systemfails, even afterinstallerhad a chance), then its dependencies are unknown and cannot be traversed. Note that autoloaded systems that are not installed are still visited and included in the returned list.The following example makes sure that all autoloaded dependencies (direct or indirect) of
my-systemare installed:(autodeps "my-system" :installer #'ql:quickload)
4.1 Automatically Generating Loaddefs
[function] extract-loaddefs system &key (process-arglist t) (process-docstring t) packages
List the loaddef forms of the autodef definitions in
:auto-depends-onofsystem.There is rarely a need to call this function directly, as
record-loaddefsandcheck-loaddefsprovide ASDF Integration.Note that this is an expensive operation, as it loads or reloads the direct dependencies listed in
:auto-depends-onone by one withasdf:load-system:forcetto find the autodefs.See the individual autodefs for descriptions of the generated loaddefs.
- If
process-docstring, then the docstrings extracted from autodef definitions will be associated with the definition.
Note that if a definition is not made with an autodef, then
extract-loaddefswill not detect it. For such functions, loaddefs must be written manually.- If
[function] write-loaddefs loaddefs stream
Write
loaddefstostreamso they can beloaded or included in anasdf:defsystem.
[function] record-loaddefs system
extract-loaddefsfromsystemandwrite-loaddefs. The arguments of these functions are taken fromsystem's:auto-loaddefs.As
extract-loaddefsloads the direct autoloaded dependencies, compiler warnings (e.g. about undefined specials and functions) may occur that go away once the generated loaddefs are in place. The easiest way to trigger this is to callrecord-loaddefsbefore these dependencies have been loaded. In this case, temporarily emptying the loaddefs file and fixing these warnings is recommended.record-loaddefsmay also be used as a condition handler, in which case it invokes therecord-loaddefsrestart.
[function] check-loaddefs system &key (errorp t)
In the
autoload-systemsystem, check that both recorded and manual autoload declarations are correct.If
:auto-loaddefsis specified, check that the file generated byrecord-loaddefsis up-to-date.Check that all manual (non-generated) loaddefs in
systemare resolved (e.g. no longerloaddef-function-p) by loading:auto-depends-on.
If
errorp, then signal an error if a check fails or the loaddefs file cannot be read. If:auto-loaddefsis specified, then therecord-loaddefsrestart is provided.If
errorpisnil, then instead of signalling an error, returnnil.This function is called automatically by
asdf:test-opon anautoload-systemmethod if:auto-loaddefshas:testt.
-
Provided by
check-loaddefsand also when the compilation of the loaddefs file declared in:auto-loaddefsfails. The functionrecord-loaddefscan be used as a condition handler to invoke this restart.