New patches: [demonstrate differing behavior between foreign-dyn expansions Stephen Compall **20060301194122 - change expand-type-to-foreign-dyn for foreign-typedef to not short-circuit expand-type-to-foreign when falling back (see paste 17379) - add tests showing that you can change the foreign-dyn expansion semantics by providing an expand-to-foreign method that falls back ] < > { hunk ./src/early-types.lisp 456 (expand-to-foreign-dyn value var body (name type))) ;; If there is to-foreign _expansion_ we use that. ((applicablep #'expand-to-foreign value (name type)) - `(let ((,var ,(expand-to-foreign value (name type)))) + `(let ((,var ,(expand-type-to-foreign value type))) ,@body)) ;; Else... (t *runtime-translator-form*))) hunk ./tests/misc-types.lisp 192 (deftest misc-types.expand.4 (expand-expect-int-sum (callback expand-int-sum)) t) + +(defctype translate-tracker :int) + +(declaim (special .fto-called.)) + +(defmethod free-translated-object (value (type-name (eql 'translate-tracker)) + param) + (declare (ignore value param)) + (setf .fto-called. t)) + +(defctype expand-tracker :int) + +(defmethod free-translated-object (value (type-name (eql 'expand-tracker)) + param) + (declare (ignore value param)) + (setf .fto-called. t)) + +(eval-when (:compile-toplevel :load-toplevel :execute) + (defmethod expand-to-foreign (value (type-name (eql 'expand-tracker))) + (declare (ignore value)) + *runtime-translator-form*)) + +(defcfun ("abs" ttracker-abs) :int + (n translate-tracker)) + +(defcfun ("abs" etracker-abs) :int + (n expand-tracker)) + +;; free-translated-object must be called when there is no etf +(deftest misc-types.expand.5 + (let ((.fto-called. nil)) + (ttracker-abs -1) + .fto-called.) + t) + +;; free-translated-object must not be called when there is an etf, but +;; they answer *runtime-translator-form* +(deftest misc-types.expand.6 + (let ((.fto-called. nil)) + (etracker-abs -1) + .fto-called.) + nil) + } [remove *runtime-translator-form* from the interface Stephen Compall **20060327173632 - Remove *runtime-translator-form* from the public interface, in favor of (call-next-method). - Describe this change in the manual, including some text from the option 1 patch. - Be clear that ALLOC-PARAM is thrown out by %expand-type-to-foreign. ] < > { hunk ./doc/cffi-manual.texinfo 2035 @noindent Much better. -The third generic function in this interface is -@code{expand-to-foreign-dyn} which can be used when it's necessary to -allocate something with dynamic extent. Consider the following -example: +@code{expand-to-foreign-dyn}, the third generic function in this +interface, is especially useful when you can allocate something much +more efficiently if you know the object has dynamic extent. Consider +the following example: @lisp ;;; This type inherits :string's translators. hunk ./doc/cffi-manual.texinfo 2050 ,@@body)) @end lisp -To provide short-circuiting functionality similar to that provided by -@code{define-compiler-macro}'s @code{&whole} argument, @cffi{} binds -the special variable @strong{@code{*runtime-translator-form*}}. -Return it instead in cases where your method cannot generate an -appropriate replacement for it. +To short-circuit expansion and use the @code{translate-*} functions +instead, simply call the next method. Return its result in cases +where your method cannot generate an appropriate replacement for it. The @code{expand-*} methods have precedence over their @code{translate-*} counterparts and are guaranteed to be used in hunk ./doc/cffi-manual.texinfo 2057 @code{defcfun}, @code{foreign-funcall}, @code{defcvar} and -@code{defcallback}. They may or may not be used in other @cffi{} -operators that need to translate between Lisp and C data; you may only -assume that @code{expand-*} methods will probably only be called -during Lisp compilation. +@code{defcallback}. If you define a method on each of the +@code{expand-*} generic functions, you are guaranteed to have full +control over the expressions generated for type translation in these +macros. + +They may or may not be used in other @cffi{} operators that need to +translate between Lisp and C data; you may only assume that +@code{expand-*} methods will probably only be called during Lisp +compilation. @code{expand-to-foreign-dyn} has precedence over @code{expand-to-foreign} and is only used in @code{defcfun} and hunk ./doc/cffi-manual.texinfo 2069 -@code{foreign-funcall}, only making sense in those contexts. +@code{foreign-funcall}, only making sense in those contexts. If you +do not define a method on @code{expand-to-foreign-dyn}, however, +please note that this expand method for the hypothetical type +@code{my-string} is not the same as defining no method at all: + +@lisp +(defmethod expand-to-foreign (value-form (type-name (eql 'my-string))) + (call-next-method)) +@end lisp + +Without this method, your runtime @code{translate-to-foreign} method +will be called, and its result will be passed to +@code{free-translated-object}. However, if you define this method, +@code{translate-to-foreign} will still be called, but its result will +not be passed to @code{free-translated-object}. If you need to free +values with this interface, you must define an +@code{expand-to-foreign-dyn} method. @strong{Important note:} this set of generic functions is called at macroexpansion time. Methods are defined when loaded or evaluated, hunk ./src/early-types.lisp 399 (defun %expand-type-to-foreign (value type) (if (translate-p type) - `(translate-type-to-foreign ,value ,type) + `(values (translate-type-to-foreign ,value ,type)) value)) (defun %expand-type-from-foreign (value type) hunk ./src/early-types.lisp 407 `(translate-type-from-foreign ,value ,type) `(values ,value))) -;;; This special variable is bound by the various :around methods below -;;; to the respective form generated by the above %EXPAND-* functions. -;;; This way, an expander can "bail out" by returning it. +;;; This special variable is bound by the various :around methods +;;; below to the respective form generated by the above %EXPAND-* +;;; functions. This way, an expander can "bail out" by calling the +;;; next method. All 6 of the below-defined GFs have a default method +;;; that simply answers the rtf bound by the default :around method. (defvar *runtime-translator-form*) hunk ./src/early-types.lisp 414 +(defun specializedp (gf &rest args) + "Answer whether GF has more than one applicable method for ARGS." + (typep (compute-applicable-methods gf args) '(cons t cons))) + (defgeneric expand-type-to-foreign-dyn (value var body type) (:method :around (value var body type) (let ((*runtime-translator-form* hunk ./src/early-types.lisp 427 ;; If COMPUTE-APPLICABLE-METHODS only finds one method it's ;; the default one meaning that there is no to-foreign expander ;; therefore we return *RUNTIME-TRANSLATOR-FORM* instead. - (if (< 1 (length (compute-applicable-methods - #'expand-type-to-foreign (list value type)))) + (if (specializedp #'expand-type-to-foreign value type) `(let ((,var ,(expand-type-to-foreign value type))) ,@body) *runtime-translator-form*))) hunk ./src/early-types.lisp 448 (declare (ignore value type)) *runtime-translator-form*)) -(defgeneric expand-to-foreign-dyn (value var body type)) -(defgeneric expand-to-foreign (value type)) -(defgeneric expand-from-foreign (value type)) - -(defun applicablep (gf &rest args) - "Returns true if GF has any applicable methods for ARGS." - (not (null (compute-applicable-methods gf args)))) +(defgeneric expand-to-foreign-dyn (value var body type) + (:method (value var body type) + (declare (ignore value var body type)) + *runtime-translator-form*)) +(defgeneric expand-to-foreign (value type) + (:method (value type) + (declare (ignore value type)) + *runtime-translator-form*)) +(defgeneric expand-from-foreign (value type) + (:method (value type) + (declare (ignore value type)) + *runtime-translator-form*)) (defmethod expand-type-to-foreign-dyn (value var body (type foreign-typedef)) hunk ./src/early-types.lisp 462 - (cond ((applicablep #'expand-to-foreign-dyn value var body (name type)) - (expand-to-foreign-dyn value var body (name type))) - ;; If there is to-foreign _expansion_ we use that. - ((applicablep #'expand-to-foreign value (name type)) - `(let ((,var ,(expand-type-to-foreign value type))) - ,@body)) - ;; Else... - (t *runtime-translator-form*))) + (if (or (specializedp #'expand-to-foreign-dyn + value var body (name type)) + (not (specializedp #'expand-to-foreign value (name type)))) + (expand-to-foreign-dyn value var body (name type)) + ;; If there is to-foreign _expansion_, but not to-foreign-dyn + ;; expansion, we use that. + `(let ((,var ,(expand-type-to-foreign value type))) + ,@body))) (defmethod expand-type-to-foreign (value (type foreign-typedef)) hunk ./src/early-types.lisp 472 - (if (applicablep #'expand-to-foreign value (name type)) - (expand-to-foreign value (name type)) - *runtime-translator-form*)) + (expand-to-foreign value (name type))) (defmethod expand-type-from-foreign (value (type foreign-typedef)) hunk ./src/early-types.lisp 475 - (if (applicablep #'expand-from-foreign value (name type)) - (expand-from-foreign value (name type)) - *runtime-translator-form*)) + (expand-from-foreign value (name type))) ;;; User interface for converting values from/to foreign using the ;;; type translators. Something doesn't feel right about this, makes hunk ./src/early-types.lisp 499 (defun free-converted-object (value type param) (free-type-translated-object value type param)) + hunk ./src/package.lisp 108 #:expand-to-foreign-dyn #:expand-to-foreign #:expand-from-foreign - #:*runtime-translator-form* ;; Foreign globals. #:defcvar hunk ./tests/misc-types.lisp 212 (eval-when (:compile-toplevel :load-toplevel :execute) (defmethod expand-to-foreign (value (type-name (eql 'expand-tracker))) (declare (ignore value)) - *runtime-translator-form*)) + (call-next-method))) (defcfun ("abs" ttracker-abs) :int (n translate-tracker)) } Context: [small manual fixes Stephen Compall **20060316115505 - Add convert-to-foreign, convert-from-foreign, and free-converted-object to the master menu. - (load-foreign-library): Add note about specifying directories in paths. - (defcfun): Typo in syntax. ] [Lispworks: use regression-test instead of rt. Luis Oliveira **20060316114554] [SCL update + new cffi-sys primitive Luis Oliveira **20060316114028 - New cffi-sys primitive: canonicalize-symbol-name-case. Use this instead of read-from-string. Also new tests for this. - cffi-scl.lisp: correct the evaluation order of %MEM-REF and %MEM-SET. Have 'foreign-symbol-pointer return 'nil if the symbol is not found. - uffi-compat.lisp: better support for SCL. - defcfun.lisp (defcfun.undefined): package at time of eval may differ from the compile time package. Patch courtesy of Douglas Crosher. ] [when (and ecl (not dffi)) cffi:load-foreign-library doesn't work Luis Oliveira **20060316112818 - mention this in the manual. - make l-f-l signal an error on non-dffi ecl platforms. ] [Remove known issues sub-section from the manual. Luis Oliveira **20060228193623 - Listing the expected failures in the manual is too much trouble. Removed those. - Also, renamed "The Scieneer Common Lisp" to "Scieneer CL" for consistency. ] [TODO item about a pointer type, suggested by Jörg Höhle Luis Oliveira **20060228191642] [More testing Luis Oliveira **20060227194854 - Make defcfun.undefined an expected failure for SBCL on non linkage-table platforms. - New file: tests/misc.lisp. - Added a couple of tests for cffi-features. ] [TODO item about our use of EVAL Luis Oliveira **20060227194813] [manual: convert-* interface, optimizing translators overview optimized Stephen Compall **20060301043349 - (Wrapper generators): Add comment about the 30-90% figure. - Reword `Pointers' to be shorter/clearer. - Promote `Optimizing Type Translators' to a full section, a few other things here, most conspicuously expanding on the note about expand-* method definition time. - (Tutorial-Conclusion): Not a first draft anymore. - Document convert-to-foreign, convert-from-foreign, and free-converted-object. ] [manual: strings, foreign allocation, add `Wrapper generators' Stephen Compall **20060228162638 - Explain the difference between mem-aref and mem-ref by analogy with C operators. - Use ::= instead of = in with-foreign-object's syntax. - (Strings): Explain that it is portable code. - (Other Types): Don't use metasyntactic variables in Lisp examples. - Add `Wrapper generators' section on Verrazano et al. - (Foreign Type Translators): Use Please note: instead of Note: to suppress makeinfo warning. ] [Try to improve the wording in foreign-alloc's description.. Luis Oliveira **20060225155627] [Minor change to foreign-alloc's documentation Luis Oliveira **20060225041058 - make it slightly clearer that count can be omitted when initial-contents is supplied. - add reference to with-foreign-object. ] [foreign-alloc changes Luis Oliveira **20060225034634 - bugfix, foreign-alloc doesn't need to call translate-type-to-foreign explicitly since mem-aref already does. Bug reported by Greg Pfeil. - new keyword argument: NULL-TERMINATE-P. - new regression tests for the bug described above and new tests for the new keyword argument. - document new argument. ] [More tests Luis Oliveira **20060224192231 - A couple of new tests involving defcfun/foreign-funcall/defcallback and lots of doubles and floats. These were written to figure out what exactly was going on with the CALLBACKS.BFF.[12] failures. ] [Fix cffi-lispworks bug in foreign-funcall. Luis Oliveira **20060224131735 - Add same declarations to define-foreign-funcallable as those used for define-foreign-callable. - Makes FUNCALL.FLOAT pass on linux/x86. ] [SCL port, courtesy of Douglas Crosher Luis Oliveira **20060223061757 - Makefile: new test-scl target. Add SCL's fasl file extensions to the clean target. - Remove SCL TODO item. - New file: cffi-scl.lisp. - New primitive type :long-double. Since it's only supported by SCL, it's not worth adding a no-long-double feature just yet. - New tests for :long-double. - Add information about SCL and the new :long-double type to the user manual. ] [Update TODO item about fixnums. Luis Oliveira **20060223181831] [Implement and use my_llabs instead of llabs in tests Luis Oliveira **20060223081406] [Make CLISP not throw an error on undefined functions Luis Oliveira **20060220044813 - Catch the error in %foreign-funcall and throw a warning instead. - Regression test: DEFCFUN.UNDEFINED. (CMUCL fails) ] [Fix LOOP indentation in backends. James Bielman **20060217204457 - Thanks for Luís for showing me how to configure Emacs to get this right. ] [Fix and add tests for %MEM-REF and %MEM-SET evaluation order. James Bielman **20060216083448 - Some minor reformatting of LOOP forms to pacify cl-indent. - Add new regression tests to check the evaluation order of %MEM-REF and %MEM-SET. - Add the necessary ONCE-ONLY forms to the compiler macros. - Have the CFFI-TESTS package use CFFI-SYS for testing primitives. ] [Fix typo. extend -> extent Luis Oliveira **20060216051544] [Minor change to tests/defcfun.lisp Luis Oliveira **20060216050735 - Remove unnecessary (setf (mem-ref s :char) 0) forms. ] [CFFI Manual update Luis Oliveira **20060216045547 - Update known issues (callback.bff.[12] failures). - defbitfield, foreign-bitfield-symbols and foreign-bitfield-value. - Fix typo in tutorial. defenum -> defcenum. - New section "Other Types" documenting :string, :boolean and :wrapper. - Add note about the translate-* methods not being meant to be called directly and suggesting convert-* instead. - Document the macroexpansion-time translators in a new "Optimizing Type Translators" sub-section. - Document defcenum's base-type option. - Document defcfun's varargs support. ] [New functions convert-to/from-foreign and free-converted-object Luis Oliveira **20060216045335 - These functions basically export the functionality of translate-type-to-foreign, translate-type-from-foreign and free-translated-object. - TODO: document these. ] [New TODO item: defcfun compiler macros Luis Oliveira **20060216045240] [Implement compiler macros for %MEM-REF and %MEM-SET in LispWorks. James Bielman **20060215222559 - Use FLI:FOREIGN-TYPED-AREF if available and dereferencing a number. - Avoid calling INC-POINTER when the offset is a multiple of the type size allowing direct use of the INDEX argument to FOREIGN-TYPED-AREF. - Fall back to open-coding the call to FLI:DEREFERENCE otherwise. ] [Delete LispWorks/Linux .ufsl files on make clean. James Bielman **20060215213442] [Use C limits for foreign floats and doubles instead of Lisp's. James Bielman **20060215201514 - Export 'float_min', 'float_max', 'double_min', and 'double_max' from the libtest shared library with the values of FLT_MIN, FLT_MAX, DBL_MIN, and DBL_MAX, respectively. - Use *FLOAT-MIN*, *FLOAT-MAX*, *DOUBLE-MIN*, and *DOUBLE-MAX* as test values for foreign floats/doubles instead of the Lisp constants -POSITIVE--FLOAT. ] [Oops, fix tests. Luis Oliveira **20060215174316 - MISC-TYPES.BOOLEAN.2 was using the wrong ff name. - MISC-TYPES.EXPAND.* need the expand-* methods defined at macroexpansion-time. ] [Change +REQUIRED-DLL-VERSION+ to *REQUIRED-DLL-VERSION*. James Bielman **20060215171631] [Remove completed TODO item about building 32 and 64 bit libtest. James Bielman **20060215091337] [A few more tests Luis Oliveira **20060215163551 - Tests for the new macroexpansion-time type translator interface: MISC-TYPES.EXPAND.[1234] - New tests: DEREF.LONG-LONG and DEREF.UNSIGNED-LONG-LONG. - Tests for mem-ref with non-constant type arguments: DEREF.NONCONST.* - Fix some comments in tests/memory.lisp. - New test: MISC-TYPES.BOOLEAN.2 (accepting typedefs to integer types) ] [Minor test changes Luis Oliveira **20060215030152 - Use long instead of int in the big C functions in libtest.c. - Mark callbacks.bff.[12] as expect failures for a couple of lisps. ] [Make (%callback 'non-existant-callback) signal an error Luis Oliveira **20060215025906 - Make %callback signal an error for non-existing callbacks. (allegro, sbcl and openmcl) - Regression test: callbacks.non-existant ] [Implement %MEM-REF and %MEM-SET compiler macros for Allegro CL. James Bielman **20060215025257] [Ignore RETTYPE in Allegro CL %DEFCALLBACK. James Bielman **20060215022651] [Fix float varargs promotion. Luis Oliveira **20060214042928 - Convert floats to doubles in foreign-funcall-varargs. Test DEFCFUN.VARARGS.FLOAT now passes. ] [Preliminary random tester. Luis Oliveira **20060214042733 - Push random-tester.lisp. This was used to generate the BFF tests, but is still not automated. - Update respective TODO item. ] [MORE TESTS Luis Oliveira **20060214032830 - New tests: callbacks.funcall.2, callbacks.bff.[12], defcfun.bff.[12]. - Re-enable defcfun.varargs.double. - Use #'float instead of #'coerce in the defcfun.varargs.* tests. - Fix dll version in libtest.c. ] [Change defconstant to defparameter in bindings.lisp Luis Oliveira **20060214002839] [Macroexpansion-time translators Luis Oliveira **20060214001931 - New interface. expand-to-foreign-dyn, expand-to-foreign and expand-from-foreign. - defcfun/foreign-funcall/defcallback/defcvar use this as well as the compiler macros and setf-expanders for mem-ref, mem-aref, and foreign-slot-value. - Also, parse ignore declarations and use them to avoid translating such types (thus avoiding breaking the ignore declaration). ] [Implement simple version check for libtest DLL Jack Unrue **20060212182709] [Push CFFI-FEATURES:X86-64 on Allegro/amd64. James Bielman **20060211003123] [Removed outdated comment from the colorize script. Luis Oliveira **20060210220055] [Use slanted instead of italic for VAR in style.css Luis Oliveira **20060210195248] [use lispcmt for code comments instead of @r Stephen Compall **20060210160507 - Add ROMANCOMMENTS option to manual. - Change all instances of @r to @lispcmt. ] [make Texinfo @r and colorize play nice Stephen Compall **20060210035038 - Comment out master menu entry for explain-foreign-slot-value so manual builds again. - Add @r to all Lisp comments in manual. - (Tutorial-easy_setopt): Revert previous @dots{}; it is literal `...', not an ellipsis. Oops. - In colorize-lisp-examples.lisp, remove the output generated by @r from Lisp blocks. Also shove around the process-file code some. ] [Remove @r{} in manual's code comments Luis Oliveira **20060210021117 (at least until the colorize script can cope with these) ] [More minor doc changes. Luis Oliveira **20060210005707 - @emph -> @var. - change var's style to something more sensible in the CSS file. - remove (setf callback) from the docs. - fix some typos. ] [Make :boolean accept typedefs to integer types. Luis Oliveira **20060210003650] [Minor documentation changes Luis Oliveira **20060209225109 - Have the Makefile's clean target delete some more of the temporary texinfo files. - Comment out reference to explain-foreign-slot-value. ] [reference prose in Pointers and Strings manual chapters Stephen Compall **20060209183938 - (Tutorial-easy_setopt): Use @dots{}. - (seealso): Remove old comment about @code formatting. - Pointers chapter: Describe the theory of foreign data, dealing with pointers per se, and expand on the allocation description. - (Strings): A short paragraph by way of introduction. ] [Remove note about unsupported FOREIGN-FUNCALL on Lispworks. James Bielman **20060206185439] [TAG 0.9.0 James Bielman **20060204082912] Patch bundle hash: 2ab5fe36c1e1e9e4066c8caebcc09d934695f803