The use and distribution terms for this software are covered by the Common Public License 1.0, which can be found in the file CPL.TXT at the root of this distribution. By using this software in any fashion, you are agreeing to be bound by the terms of this license. You must not remove this notice, or any other, from this software.
eq
on the Lisp sideensure-typed-ref
, which makes a remote reference
its most fully derived type in Lisp, works in place, using change-class
new
macro)We are going to try using SourceForge facilities for all communication regarding Foil, so please use the project tracker and forums there.
(compile-file "/dev/foil/foil.lisp") (load "/dev/foil/foil") (use-package :foil) ;this is specific to LispWorks' sockets support (require "comm") ;create a foreign VM (setf *fvm* (make-instance 'foreign-vm :stream (comm:open-tcp-stream "localhost" 13579))) ;;;;;;;;; if Java ;;;;;;;;;;; ;create a wrapper for dialog class (def-foil-class "javax.swing.JOptionPane") ;use the wrapper (use-package "javax.swing") ;show it (joptionpane.showmessagedialog nil "Hello World from Lisp") ;;;;;;;;; if CLI ;;;;;;;;;;;; ;create a wrapper for dialog class (def-foil-class "System.Windows.Forms.MessageBox") ;use the wrapper (use-package "System.Windows.Forms") ;show it (messagebox.show "Hello World from Lisp")
Typically you wouldn't define single class wrappers by hand, but would instead pre-generate wrappers for entire packages using
get-library-classnames
and dump-wrapper-defs-to-file
. See foil-java.lisp
and
foil-cli.lisp
for examples.
foil.lisp
, and is
runtime server independent. Foil is designed to be very portable, and is 99% pure Common Lisp. A complete port requires some
facility for weak hash tables and object finalization.
Foil is built upon the notion of interactions with one or more foreign VMs, instances of the JVM or CLR, running the Foil libraries, in another process on the same or another machine. The connection to a specific VM is via one or more bidirectional streams. Note that the instantiation of the foreign VM and the establishment of the streams is outside the scope of this Lisp API. It is presumed you might utilize one of the supplied runtime servers, creating stream connections via sockets or pipes with the API provided by your Lisp implementation. Many scenarios are possible, including embedding the Foil support libraries into your existing Java or C# application, multiple streams to different threads in the same VM, etc.
A foreign VM is represented by an instance of the foreign-vm
class. Each instance has a primary default stream
over which communication will occur. The special variable *fvm*
represents the default VM to which any unqualified
Foil calls will be directed, and can be bound in a specific context, thus allowing for multiple VMs.
Note - instance property/method calls will always be routed to the VM hosting that instance.
Foreign VMs maintain their own foreign reference pools, type caches etc, and objects from one VM cannot be passed to another, even if they are both Java or CLI. However, in multi-thread, multi-stream scenarios, references are valid across threads in the same VM, and the runtime server implementations are thread safe.
A simple startup scenario would look like this:
- First, outside of Lisp, start the Java or CLI Foil runtime server supplied with Foil, running on port 13579, then:
(load "/dev/foil/foil") (use-package :foil) (require "comm") ;LispWorks-specific socket library (setf *fvm* (make-instance 'foreign-vm :stream (comm:open-tcp-stream "localhost" 13579))) ;use Foil
*fvm*
This must be bound to an instance of foreign-vm
. Default: nil
Direct use of this other than during initial setup is not recommended, use instead with-vm
or
with-vm-of
*thread-fvm*
If set, this thread is waiting on a callback from this VM. Default: nil
This is only used for advanced multi-thread scenarios
*thread-fvm-stream*
If this thread is waiting on a callback (i.e. *thread-fvm*
is bound), and (eql *fvm* *thread-fvm*)
,
use this stream instead of the primary default stream for the VM. Default: nil
This is only used for advanced multi-thread scenarios
foreign-vm
Manages a foreign VM. Requires the initarg :stream
be set to a bidirectional stream with an instance of the
Foil runtime services on the other end.
(with-vm vm &body body)
Causes the body to be evaluated in a context where *fvm*
is bound to vm
(with-vm-of this &body body)
Causes the body to be evaluated in a context where *fvm*
is bound to the source VM of this
fref
class. The Foil API and the runtime server cooperate to ensure that objects referenced by Lisp are kept alive
in the foreign runtime, and that when no longer referenced by Lisp, they become available for collection in the foreign VM.
frefs
maintain their source VM, an ID and revision counter for this purpose. In addition, frefs
can
cache hash codes, types, and values that have been marshalled. Only a single fref
will be created for each remote object,
thus any 2 frefs
that reference the same object are themselves eq
.
fref
Reference to a foreign object. fref
is the superclass of all of the Foil classes generated to mirror the
foreign hierarchy.
(fref-vm fref)
->The foreign-vm from which this reference originated
(fref-id fref)
->An integer ID, unique within a VM
(fref-type fref)
->A Class or Type fref
This will only be set if the type has been marshalled or get-type
has been called on this fref.
(fref-hash fref)
->int
This will only be set if the hash code has been marshalled or hash
has been called on this fref.
(fref-val fref)
-> A Lisp object representing the value of the object
This will only be set if the value has been marshalled.
(ensure-typed-reference fref)
-> fref
, whose class may have been changed
Given a generic Foil fref, determines the full type of the object and
uses change-class
to convert the fref to that type. Since we don't want to always incur the cost of
type determination, the wrapper-generated API functions return generic references. Use this function to convert to
a typed reference corresponding to the full actual type of the object when desired:
CL-USER 42 > (setf string-class (get-type-for-name "java.lang.String")) #}1 CL-USER 43 > (type-of string-class) FREF CL-USER 44 > (ensure-typed-ref string-class) #}1 CL-USER 45 > (type-of string-class) CLASS.
(def-foil-class full-class-name) -> unspecified
Given the package-qualified, case-correct name of a Java/CLI class as a string, will generate wrapper functions for its public constructors, fields, properties and methods.
The core API for generating interfaces to Java/CLI is the def-foil-class
macro. This
macro will, at expansion time, use Java/CLI reflection to find all of the public
constructors, fields, properties and methods of the given class and generate functions to
access them.
(def-foil-class "java.lang.ClassName")
you get several symbols/functions:
|java.lang|
(note case)classname.
(note the dot is part of the name)(classname.new &rest args) -> fref
, which
returns a foreign-reference (fref) to the newly created object. Note that the constructor function,
and therefore everything built upon it, can take the actual arguments to the Java/CLI ctor, followed by
zero or more property initializers, which take the form:(window.new parent :width 200 :height 200)
make-new
, ultimately
calling classname.new
, specialized on (the value of) the class-symbol
(classname.fieldname [instance]) -> field value
(setf classname.fieldname [instance])
*classname.fieldname*
If the type of the field is primitive, the field value will be converted to a native Lisp value. If it is a Java/CLI String, it will be converted to a Lisp string. Otherwise, a foreign reference to the Java/CLI object is returned. Similarly, when setting, Lisp values will be accepted for primitives, Lisp strings for Strings, or foreign references for reference types.
(classname.propertyname [instance] [args]) -> property value
(setf classname.propertyname [instance] [args])
*classname.propertyname*
(classname.methodname &rest args) -> return-value
The same argument and return value conversions are performed as are for fields. The function documentation string describes the method signature(s) from the Java/CLI perspective.
fref
.
An instance of this class will be
returned by ensure-typed-ref
, at which point the entire hierarchy will
consist of finalized standard-classes.
(get-library-classnames jar-or-assembly-filename &rest packages)
-> list-of-strings
Returns a list of class name strings. Packages should be strings of the form "java/lang" or "System/IO" for recursive lookup and "java/util/" or "System/IO/" (note trailing slash) for non-recursive.
(dump-wrapper-defs-to-file filename classnames) ->
filename
Given a list of classnames (say from get-library-classnames
), writes
the consolidated expansions of calls to def-foil-class
to a file:
(dump-wrapper-defs-to-file "/lisp/java-lang.lisp" (get-library-classnames "/j2sdk1.4.2_01/jre/lib/rt.jar " "java/lang/")) (compile-file "/lisp/java-lang") (load "/lisp/java-lang") (use-package "java.lang") ;Wrappers for all of java.lang are now availableThis is the recommended way to access entire library packages. In particular, it has the advantage that the dumped code does not require a foreign runtime to either compile or load.
classname.new
described above), the following, built upon the same,
add some additional capabilites and ease of use:
(make-new classname. &rest args) -> fref
Allows for definition of before/after methods on constructors. Calls classname.new
ctor.
The new macro expands into a call to this.
(new class-spec args &body body) -> fref
class-spec -> class-sym | (class-sym this-name)
class-sym -> classname.
args -> as per ctors and make-new
make-new
generic function,
then runs the body replacing all top-level calls of the form
(.anything whatever)
with
(classname.anything new-object whatever)
If this-name
is supplied it will be bound to the newly-allocated object and available
to the body (note - but not to the args!)
Example:
(new shell. (*display* :text "SWT Apropos" :layout (gridlayout.new 1 t )) (.setsize 800 600) (.setlocation 100 100))Expands into:
(LET ((#:G2249 (MAKE-NEW SHELL. *DISPLAY* :TEXT "SWT Apropos" :LAYOUT (GRIDLAYOUT.NEW 1 T)))) (PROGN (SHELL.SETSIZE #:G2249 800 600) (SHELL.SETLOCATION #:G2249 100 100)) #:G2249)
(equals fref1 fref2) -> boolean
Portable Object.equals/Equals
(instance-of fref type) -> boolean
Portable instanceof/is
(to-string fref) -> string
Portable Object.toString/ToString
(hash fref &key rehash) -> int
Portable Object.hasCode/GetHashCode. Note: will cache the value on the fref. If already cached, will return that, unless :rehash is t.
(get-type fref) -> Class or Type fref
Portable Object.getClass/GetType. Note: will cache the value on the fref. Note also that obtaining the exact type of
the object is completely independent of the coercion of the fref to its corresponding Lisp type
(see ensure-typed-ref
)
(iref indexable-obj &rest indexes) -> a value
Calls the default indexer for the object. CLI only. Settable.
(make-new-vector type length &rest inits) -> array fref
Creates a foreign vector of specified type and length. There can be fewer inits than the length, in which case the remaining values take the default.
(vref vector index) -> value
Returns the value at the index. Settable.
(vlength vector index) -> int
Returns the length of the vector.
In most cases argument matching and conversion should be transparent. Lisp strings can be passed where Strings are required,
Lisp numbers where int float etc are required. t
and nil
can be passed where booleans are required etc.
nil
can be passed for null
.
Some Foil APIs (e.g. make-new-vector) require type
arguments, and unless specified otherwise,
any of the following are acceptable:
classname.
:boolean|:byte|:char|:double|:float|:int|:long|:short
"package.qualified.ClassName"
string, case-sensitive.
This is least efficient and should only be used in dynamic scenarios.
Occasionally it may be necessary to provide a hint as to the intended type of a numeric argument in order to force resolution to a particular overload.
(box type val)
Type must be a primitive designator keyword - :boolean|:byte|:char|:double|:float|:int|:long|:short
Produces an object that when passed to a Foil-generated function will be interpreted by the runtime as that type.
Note that silent truncation may occur.
It is also possible to create vectors in-line as arguments, which will avoid multiple round-trips vs.
calling make-new-vector
in place. Note this is only good for ephemeral vectors, as there is no way to retain a
reference to the newly-created vector.
(box-vector type &rest vals)
Produces an object that when passed to a Foil-generated function will be an array of type with the supplied values.
CL-USER 102 > (vref (box-vector string. "a" "b" "c" "d") 2) ;only one round-trip "c"
(get-type-for-name full-class-name)
Returns a Class/Type instance corresponding to name. Portable Class.forName/Type.GetType.
(full-class-name class-symbol)
Returns the package qualified name string corresponding to the class symbol:
CL-USER 104 > (full-class-name string.) "java.lang.String"
(handle-proxy-call method-symbol proxy &rest args))
The proxy infrastructure routes all callbacks to this generic function. If a proxy p implements an interface i, and the foreign
VM ends up invoking i.foo on p, it will map to a call to handle-proxy-call
with the first 2 arguments
of 'i.foo and p, followed by the actual arguments to the invocation. So, callback handlers can be defined by specializing
methods on the method-symbol, the proxy object, or both. The unspecialized method spews out "unhandled proxy call" to
standard output and returns nil to the foreign VM. Any unhandled errors that occur on the Lisp side during a callback turn into
exceptions on the runtime server.
(make-new-proxy arg-marshall-flags arg-marshall-depth &rest interface-types) -> proxy fref
Creates and returns a proxy object that implements the given interface types or delegate type. arg-marshall-flags
and
arg-marshall-depth
will be used to marshall the arguments to the callback. No handlers are defined by this function.
(new-proxy proxy arg-marshall-flags arg-marshall-depth &rest interface-defs) -> proxy fref
Creates and returns a proxy object that implements the given interface types or delegate.
proxy -> a symbol
interface-def -> (interface-name method-defs+)
interface-name -> classname. (must name an interface or delegate type)
method-def -> (method-name (args*) body)
method-name -> symbol (without classname)
The symbol proxy
will be bound to the proxy instance in the body of the method implementations.
(new-proxy p +MARSHALL-ID+ 0 (keylistener. (keyreleased (event) (when (eql *SWT.CR* (keyevent.character event)) (gob)))))
Foil supports an extensible marshalling system which allows the values of reference/composite types to be returned in addition to, or even instead of, the references themselves. Used appropriately, this can substantially reduce the number of round trips between processes and avoid significant 'chatter' overhead.
Marshalling comes into play whenever a reference type is returned from a Foil function. With certain settings, it is possible
to return any or all of a reference, its hash code, its type, and its value (and the same for any of its
value's reference members), to a specific depth. The nature and depth of the marshalling is governed by two special variables on the Lisp side -
*marshalling-flags*
and *marshalling-depth*
.
The format of marshalled values is determined by the runtime servers, and both the Java and CLI servers provided with Foil have facilities for adding new marshallers for specific types. The way an object's value is marshalled is a function of its type.
Class or Types will always marshall the string representing the packageQualifiedTypeName, ignoring *marshalling-depth*
By default, the following marshalling will be performed when requested, i.e. *marshalling-depth*
> 0
*marshalling-depth*
Default: 0
A depth of 0 means no values are marshalled, a setting of 1 means
that values will be marshalled for the returned object (if it is a reference), but not any nested references.
When > 1 nested reference types will marshall, to that depth of nesting.
*marshalling-flags*
Either +MARSHALL-NO-IDS+
, or the logical or-ing of +MARSHALL-ID+
and zero or more of:
+MARSHALL-HASH+
and +MARSHALL-TYPE+
Default: +MARSHALL-ID+
A setting of +MARSHALL-NO-IDS+
means that no frefs will be returned, and thus no references will be held
on the VM side. If *marshalling-depth*
is 0 then nil will be returned. If *marshalling-depth*
> 0
then the value will be returned instead of the fref.
Otherwise, +MARSHALL-ID+
must be set, and frefs will be returned for reference types.
If *marshalling-depth*
> 0,
then the marshalled values will be in the fref-val
slot. If +MARSHALL-HASH+
is set then
the object's hash code will be calculated and stored in the fref-hash
slot. Similarly, if
+MARSHALL-TYPE+
is set then
the object's class/type will be determined and stored in the fref-type
slot.
(with-marshalling (depth &rest flags) &body body)
Evaluates the body in a context in which the *marshalling-depth*
is set to depth and
*marshalling-flags*
to the logior
of flags.
(marshall fref)
Explicitly marshalls the object with the current *marshalling-flags*
and *marshalling-depth*
settings,
and returns the marshalled object (which may be the same fref, but with additional data in its type/hash/val slots)
CL-USER 79 > (setf string-class (get-type-for-name "java.lang.String")) #}1 CL-USER 88 > (class.getpackage string-class) #}12 CL-USER 90 > (pprint (with-marshalling (1 +MARSHALL-NO-IDS+) (class.getpackage string-class))) ((:IMPLEMENTATIONTITLE . "Java Runtime Environment") (:IMPLEMENTATIONVENDOR . "Sun Microsystems, Inc.") (:IMPLEMENTATIONVERSION . "1.4.2_05") (:NAME . "java.lang") (:SEALED) (:SPECIFICATIONTITLE . "Java Platform API Specification") (:SPECIFICATIONVENDOR . "Sun Microsystems, Inc.") (:SPECIFICATIONVERSION . "1.4"))
Foil includes 2 complete implementations of the runtime server portion of the protocol, one for Java/JVM, the other for C#/CLI. The implementations are 100% Java/Managed, and use only standard libraries. Both will run the protocol over one or more TCP sockets specified on the command line, or, if none specified, via standard IO.
Project files are included for Eclipse and Visual Studio. All of the Java code is in com/richhickey/foil and the stand-alone server is in RuntimeServer.Main. The CLI implementation is in 2 projects, one for the Foil library itself - FoilCLI, and the other for the stand-alone server - FoilCLISvr.
After building, you can invoke the Java server as follows:
java -cp . com.richhickey.foil.RuntimeServer 13579
Make sure the classpath includes the libraries and .jars you will want to use via Foil.
After building, you can invoke the CLI server as follows:
foilclisvr 13479
Returnable messages:
(:cref member-type tref|"packageQualifiedTypeName" "memberName")
Where member-type is an integer representing one of:
Note that both Java and the CLI support overloading, so a single member name might map to multiple overloads. The resolution of the overloading must occur in the runtime server at the time of invocation, i.e. any of the overloads may be called through the same cref.
Returns -> A reference to a callable thing is returned in the standard return format (see below).
(:ret #{:ref ...})
(:new tref marshall-flags marshall-value-depth-limit (args ...) property-inits ...)
where property-inits is a set of :keyword-style-name value
pairs
(:call cref marshall-flags marshall-value-depth-limit target args ...)
Example:
(:call #}101 1 0 2 #}17 "fred")
Where cref is an cref that has been obtained via :cref, or, only in the case of calls to Lisp,
a symbol that names a function.
marshalling-flags is an integer representing a bitwise-or'ing of:
(:ret value)
(:proxy-call ...)
(:err "error description" "stack trace")
#{:box typename value}
Where typename is one of :byte :int :long :short :float :double
N.B. silent truncation may occur
Return values should never be boxed
#{:vector "packageQualifiedTypeName"|tref|:int(etc) value ...}
#{:ref id rev :type a-ref :hash an-int :val marshalled-value}
A reference (obtained previously) is passed back to its host like this:
#}123
The host will look up the object with that id and pass it along to the call.
(:err "error description" "stack trace")
if an exception occurred while processing the request. Unless the exception originated in the reflection API, it is preferred that the stack trace be of the inner (reflection-invoked) call.
(:free refid refrev ...)
-> nil
a list of id/rev pairs is passed. Allows one or more refs to be GC-ed
on the hosting side. It is an error to refer to these refids again.
It is anticipated that runtime servers will provide for user-installable marshallers, associated with types, that will render the value of an object of that type on a stream in a form readable by Lisp. By default at least the following marshallers should be provided:
In addition to marshalling returns during calls, the value of an object reference can be explicitly marshalled:
(:marshall ref marshall-flags marshall-value-depth-limit)
-> Lisp-readable-value
Hash values
(:hash ref)
-> int
Object equality
(:equals ref ref)
-> t|nil, per Object.Equals
ToString
(:str ref)
-> "string value"
(:tref "packageQualifiedTypeName")
-> tref(:type-of ref)
-> tref(:is-a ref tref)
-> t|nil
(:bases tref|"packageQualifiedTypeName")
-> (:ret ("packageQualifiedTypeName" ...)) ;most-derived to least-derived
(:members tref|"packageQualifiedTypeName")
-> (:ret ( (:ctors doc-string ...)
(:methods ((:name string)
(:static bool)
(:doc doc-string)) ...)
(:fields ((:name string)
(:static bool)
(:doc doc-string)) ...)
(:properties ((:name string)
(:static bool)
(:get-doc doc-string)
(:set-doc doc-string)) ...)))
(:proxy marshall-flags marshall-value-depth-limit interface-trefs ...)
-> (:ret proxy-ref)
Creates a proxy object that implements the given interface(s). When any of the object's methods are called, sends a :proxy-call message of the form:
(:proxy-call method-symbol proxy-ref args ...)
where the proxy-ref is the same one originally returned from the :proxy message, and the args are marshalled with the flags and depth requested in the :proxy message. method-symbol has the form
|package.name|::classname.methodname
note this means that the Lisp names are not independent, hmmm...
I'd like to thank my good friend Eric Thorsen for his hard work on the CLI/C# port.
I hope you find Foil useful. It is my sincere intent that it enhance the utility and interoperability of Common Lisp. I welcome comments and code contributions.
Rich Hickey, February, 2005