Pymacs version 0.24-beta1
Extending Emacs with Python
| Author: | François Pinard |
|---|
| Email: | pinard@iro.umontreal.ca |
|---|
| Copyright: | © Progiciels Bourbeau-Pinard inc., Montréal 2003, 2008 |
|---|
Pymacs is a powerful tool which, once started from Emacs, allows two-way
communication between Emacs Lisp and Python. Pymacs aims to employ
Python as an extension language for Emacs rather than the other way
around, and this asymmetry is reflected in some design choices. Within
Emacs Lisp code, one may load and use Python modules. Python functions
may themselves use Emacs services, and handle Emacs Lisp objects kept in
Emacs Lisp space.
The goals are to write naturally in both languages, debug with ease,
fall back gracefully on errors, and allow full cross-recursion.
It is very easy to install Pymacs, as neither Emacs nor Python need to
be compiled nor relinked. Emacs merely starts Python as a subprocess,
and Pymacs implements a communication protocol between both processes.
Report problems, documentation flaws, or suggestions to François Pinard:
The main Pymacs site conveys the Pymacs documentation (you are reading
its Pymacs manual right now) and distributions:
I expect average Pymacs users to have a deeper knowledge of Python
than Emacs Lisp. People have widely varying approaches in writing
.emacs files, as far as Pymacs is concerned:
- Some can go and write almost no Emacs Lisp, yet a bit is still
necessary for establishing a few loading hooks. For many simple
needs, one can do a lot without having to learn much.
- On the other hand, for more sophisticated usages, people cannot
really escape knowing the Emacs Lisp API to some extent, because they
should be programming-wise familiarity with what is a buffer, a point,
a mark, etc. and what are the allowed operations on those.
While Pymacs examples are no substitute for a careful reading of the
Pymacs manual, the contemplation and study of others' nice works may
well enligthen and deepen your understanding. A few examples are
included within the Pymacs distribution, each as a subdirectory of the
contrib/ directory, and each haviing its own README
file. These are listed below, easiest examples first:
- Paul Winkler's example
- Fernando Pérez' examples
- Giovanni Giorgi's files
- A reformatter for boxed comments
A few more substantial examples of Pymacs usage have been brought to my
attention, and are available externally (listed here in no particular
order):
- pymdev -- A Python Emacs Development Module:
- Ropemacs -- Features like refactoring and code-assists:
- Bicycle Repair Man -- A Refactoring Tool for Python:
- Emacs Freex -- A personal wiki on steroids:
The QaTeX project was influenced by Pymacs, according to its author:
You are welcome writing to or joining the following mailing list, where
there are a few people around likely to give you feedback:
If you have no fear of wider crowds :-), there still is:
There are other Web sites specifically about Pymacs. Giovanni Giorgi
has one of them:
You should make sure that both Emacs and Python are usable, whatever the
directory happens to be the current one. This is particularly important
at the time Emacs launches Python under the scene, as Python ought to be
found then started. On most systems, this means setting the search path
correctly.
The following notes, for MS Windows, have been provided by Greg Detre.
After Start / Run / Cmd, type python. If this works
wherever you are, then your Python installation directory is already
in your system's PATH environment variable. If that's not the
case, follow the instructions here to add it:
http://www.computerhope.com/issues/ch000549.htm
You may have to add the directory containing the Python scripts that
you want to run through Pymacs to your PYTHONPATH variable,
in the same fashion as above. You can test this by running Python,
and then:
import sys
sys.path
or just:
import my_python_scripts
from somewhere besides your scripts directory.
The environment variable PYMACS_PYTHON is usually left unset or
empty, in which case python is implied. It has the purpose of
naming the Python interpreter program to be called for starting the
Pymacs helper. It may be set to give the full path of the executable if
the Python program exists at some location outside the program search
path. It may also be given when the interpreter name is different,
for exemple when the Python version is part of the program name.
The similar environment variable PYMACS_EMACS is usually left unset
or empty, in which case emacs is implied. It has the purpose of
naming the Emacs editor, yet this is only meaningful for the validation
(see next section). For normal Pymacs usage, Emacs is launched by the
user long before Pymacs is itself started, and consequently, there
is absolutely no need to tell Pymacs which Emacs is needed. For the
validation suite however, it may be set to give the full path of the
executable if the Emacs program exists at some location outside the
program search path. It may also be given when the editor name is
different, for example when the Emacs version is part of the program
name, or when this is a different editor (like the value xemacs to
call XEmacs).
To know, before installing Pymacs, if it would work on your system,
try the validation suite by running make check. The suite is
fairly elementary, but nevertheless, it is able to detect some common
show stoppers. As a convenience for those who want to quickly try
various Emacs and Python combinations, make check emacs=SOME_EMACS
python=SOME_PYTHON temporarily overrides the environment variables
PYMACS_EMACS and PYMACS_PYTHON. For example, make check
emacs=xemacs runs the validation suite using xemacs for an editor.
The remaining of this section may be safely be skipped, for mere Pymacs
installation.
I did not base the validation suite on Junit (the Python unit testing
framework is a re-implementation of it), but on Codespeak's pylib
py.test, which is much simpler, and still very powerful. The
pylib project is driven by Holge Kregel, but attracted some
Python brains, like Armin Rigo (known for Psyco, among other things --
I think his lsprof has also been added to Python 2.5 under the
name cProfile). This gang addresses overdone/heavy methods in
Python, and do them better. Even py.test is a bit more complex
that I would want, and has (or at least had) flaws on the Unicode side,
so I rewrote my own, as a simple single file. I merely translated it
from French to English, to make it more distributable within Pymacs.
It has not been fruitful, trying to use Emacs stdin and stdout for
communicating expressions to evaluate and getting back results from
within the validation suite. After some fight, I reluctantly put this
avenue aside. Currently, the suite writes problems in files, for Emacs
to read, and Emacs writes replies in files, for the suite to check.
Busy waiting (with small sleep added in the loops) is used on both
sides. This is all too heavy, and it slows down the suite. Hopefully,
the suite is not run often, this is not a real problem.
Pymacs is a small package. Putting the documentation and administrative
files aside, there is one Python file and one Emacs Lisp file to it, to
be installed in turn. Always start with the Python file.
For the Python part
At the top-level of the Pymacs distribution, then execute python
setup.py install. First, the script copies a few source files while
presetting the version strings in them. Second, it installs the
Python package through the Python standard Distutils tool. To get
an option reminder, do python setup.py install --help. Consult
the Distutils documentation if you need more information about this.
That's all to it. To check that pymacs.py is properly
installed, start an interactive Python session and type from Pymacs
import lisp: you should not receive any error.
For the Emacs part
This is usually done by hand now. First select some directory along
the list kept in your Emacs load-path, for which you have
write access, and copy file pymacs.el in that directory.
If you want speed, you should ideally byte-compile this file. To do
so, go to that directory, launch Emacs, then give the command M-x
byte-compile-file RET pymacs.el RET. If for some reason you intend
to such commands often, you could create a little script to do so.
Here is an example of such a script, assuming here that you use Emacs
and want to install in directory ~/share/emacs/lisp/:
#!/bin/bash
cp pymacs.el ~/share/emacs/lisp/
emacs -batch -eval '(byte-compile-file "~/share/emacs/lisp/pymacs.el")'
You should be done now. To check that pymacs.el is properly
installed, return to your usual directories, start Emacs and give
it the command M-x load-library RET pymacs RET: you should not
receive any error.
Some features from previous Pymacs releases have been dropped:
- There used to be a script for installing the Emacs Lisp file. As it
was difficult to get it right in all circumstances; the script grew
an interactive mode and lot of options. This is just not worth the
complexity, so this script is now gone.
- Examples were all installed automatically, but at least for some of
them, this was more pollution than help. You may browse the contents of
the contrib/ directory to learn about available examples.
The .emacs file is not given in the distribution, you likely
have one already in your home directory. You need to add these lines:
(autoload 'pymacs-apply "pymacs")
(autoload 'pymacs-call "pymacs")
(autoload 'pymacs-eval "pymacs" nil t)
(autoload 'pymacs-exec "pymacs" nil t)
(autoload 'pymacs-load "pymacs" nil t)
;;(eval-after-load "pymacs"
;; '(add-to-list 'pymacs-load-path YOUR-PYMACS-DIRECTORY"))
If you plan to use a special directory to hold your own Pymacs code in
Python, which should be searched prior to the usual Python import search
path, then uncomment the last two lines (by removing the semi-colons)
and replace YOUR-PYMACS-DIRECTORY by the name of your special
directory. If the file ~/.emacs does not exist, merely create
it with the above lines. You are now all set to use Pymacs.
To check this, start a fresh Emacs session, and type M-x
pymacs-eval RET. Emacs should prompt you for a Python expression.
Try repr(2L**111) RET. The mini buffer should display
"2596148429267413814265248164610048L". M-x pymacs-load RET should
prompt you for a Python module name. Reply os RET RET (the second
RET is for accepting the default prefix. This should have the
effect of importing the Python os module within Emacs. Typing
M-: (os-getcwd) RET should echo the current directory in the message
buffer, as returned by the os.getcwd Python function.
Pymacs has been initially developed on Linux, Python 1.5.2, and Emacs
20, and currently on Python 2.5, Emacs 22.1 and XEmacs 21.5. It is
expected to work out of the box on many flavours of Unix, MS Windows and
Mac OSX, and also on many version of Python, Emacs and XEmacs.
From Pymacs 0.23 and upwards, Python 2.2 or better is likely needed,
and for the Pymacs proper, I rely on testers or users for portability
issues. However, the validation suite itself requires Python 2.4 or
better, someone might choose to contribute the back porting.
Pymacs uses Emacs weak hash tables. It can run without them, but then,
complex Python objects transmitted to Emacs will tie Python memory
forever. It should not be a practical problem in most simple cases.
Some later versions of Emacs 20 silently create ordinary tables when
asked for weak hash tables. Older Emacses do not have hash tables.
The Pymacs Python package holds a single pymacs.py file
(and the mandatory __init__.py). Programmers might elect, but
are not required, to install their own Pymacs applications either as
sub-modules or sub-packages on Pymacs.
Whenever Emacs Lisp calls Python functions giving them arguments, these
arguments are Emacs Lisp structures that should be converted into Python
objects in some way. Conversely, whenever Python calls Emacs Lisp
functions, the arguments are Python objects that should be received
as Emacs Lisp structures. We need some conventions for doing such
conversions.
Conversions generally transmit mutable Emacs Lisp structures as mutable
objects on the Python side, in such a way that transforming the object
in Python will effectively transform the structure on the Emacs Lisp
side (strings are handled a bit specially however, see below). The
other way around, Python objects transmitted to Emacs Lisp often loose
their mutability, so transforming the Emacs Lisp structure is not
reflected on the Python side.
Pymacs sticks to standard Emacs Lisp, it explicitly avoids various Emacs
Lisp extensions. One goal for many Pymacs users is taking some distance
from Emacs Lisp, so Pymacs is not overly pushing users deeper into it.
Emacs Lisp nil and the equivalent Emacs Lisp () yield Python
None. Python None, Python False and the Python
empty list [] are returned as nil in Emacs Lisp. Notice
the assymetry, in that three different Python objects are mapped into
a single Emacs Lisp object. So, neither False nor [] are
likely produced by automatic conversions from Emacs Lisp to Python.
Emacs Lisp t yields Python True. Python True is
returned as t in Emacs Lisp.
Emacs Lisp numbers, either integer or floating, are converted in
equivalent Python numbers. Emacs Lisp characters are really numbers
and yield Python numbers. In the other direction, Python numbers are
converted into Emacs Lisp numbers, with the exception of long Python
integers and complex numbers.
Emacs Lisp strings are usually converted into equivalent Python strings.
As Python strings do not have text properties, these are not reflected.
This may be changed by setting the pymacs-mutable-strings
option: if this variable is not nil, Emacs Lisp strings are
then transmitted opaquely. Python strings are always converted into
Emacs Lisp strings. Unicode strings are produced on the Python side for
Emacs Lisp multi-byte strings, but only when they do not fit in ASCII,
otherwise Python narrow strings are produced. Conversely, Emacs Lisp
multi-byte strings are produced for Python Unicode strings, but only
when they do not fit ASCII, otherwise Emacs Lisp uni-byte strings are
produced. Currently, Pymacs behaviour is undefined for users wandering
outside the limits of Emacs' utf-8 coding system.
Emacs Lisp symbols yield lisp[STRING] notations on the Python
side, where STRING names the symbol. In the other direction,
Python lisp[STRING] corresponds to an Emacs Lisp symbol printed
with that STRING which, of course, should then be a valid Emacs
Lisp symbol name. As a convenience, lisp.SYMBOL on the Python side
yields an Emacs Lisp symbol with underscores replaced with hyphens;
this convention is welcome, as Emacs Lisp programmers commonly prefer
using dashes, where Python programmers use underlines. Of course, this
lisp.SYMBOL notation is only usable when the SYMBOL is a
valid Python identifier, while not being a Python keyword.
The case of strings has been discussed in the previous section.
Proper Emacs Lisp lists, those for which the cdr of last cell
is nil, are normally transmitted opaquely to Python. If
pymacs-forget-mutability is set, or if Python later asks for
these to be expanded, proper Emacs Lisp lists get converted into Python
lists, if we except the empty list, which is always converted as Python
None. In the other direction, Python lists are always converted
into proper Emacs Lisp lists.
Emacs Lisp vectors are normally transmitted opaquely to Python.
However, if pymacs-forget-mutability is set, or if Python
later asks for these to be expanded, Emacs Lisp vectors get converted
into Python tuples. In the other direction, Python tuples are always
converted into Emacs Lisp vectors.
Remember the rule: Round parentheses correspond to square brackets!.
It works for lists, vectors, tuples, seen from either Emacs Lisp or
Python.
The above choices were debatable. Since Emacs Lisp proper lists
and Python lists are the bread-and-butter of algorithms modifying
structures, at least in my experience, I guess they are more naturally
mapped into one another, this spares many casts in practice. While in
Python, the most usual idiom for growing lists is appending to their
end, the most usual idiom in Emacs Lisp to grow a list is by cons'ing
new items at its beginning:
(setq accumulator (cons 'new-item accumulator))
or more simply:
(push 'new-item accumulator)
So, in case speed is especially important and many modifications
happen in a row on the same side, while order of elements ought to
be preserved, some (nreverse ...) on the Emacs Lisp side or
.reverse() on the Python side side might be needed. Surely, proper
lists in Emacs Lisp and lists in Python are the normal structure for
which length is easily modified.
We cannot so easily change the size of a vector, the same as it is a bit
more of a stunt to modify a tuple. The shape of these objects is
fixed. Mapping vectors to tuples, which is admittedly strange, will
only be done if the Python side requests an expanded copy, otherwise an
opaque Emacs Lisp object is seen in Python. In the other direction,
whenever an Emacs Lisp vector is needed, one has to write
tuple(python_list) while transmitting the object. Such
transmissions are most probably to be unusual, as people are not going
to blindly transmit whole big structures back and forth between Emacs
and Python, they would rather do it once in a while only, and do only
local modifications afterwards. The infrequent casting to tuple
for getting an Emacs Lisp vector seems to suggest that we did a
reasonable compromise.
In Python, both tuples and lists have O(1) access, so there is no real
speed consideration there. Emacs Lisp is different: vectors have
O(1) access while lists have O(N) access. The rigidity of Emacs Lisp
vectors is such that people do not resort to vectors unless there
is a speed issue, so in real Emacs Lisp practice, vectors are used
rather parsimoniously. So much, in fact, that Emacs Lisp vectors are
overloaded for what they are not meant: for example, very small vectors
are used to represent X events in key-maps, programmers only want to
test vectors for their type, or users just like bracketed syntax. The
speed of access is hardly an issue then.
When a Python function is called from Emacs Lisp, the function arguments
have already been converted to Python types from Emacs Lisp types and
the function result is going to be converted back to Emacs Lisp.
Several Emacs Lisp objects do not have Python equivalents, like for
Emacs windows, buffers, markers, overlays, etc. It is nevertheless
useful to pass them to Python functions, hoping that these Python
functions will operate on these Emacs Lisp objects. Of course, the
Python side may not itself modify such objects, it has to call for
Emacs services to do so. Emacs Lisp handles are a mean to ease this
communication.
Whenever an Emacs Lisp object may not be converted to a Python object,
an Emacs Lisp handle is created and used instead. Whenever that Emacs
Lisp handle is returned into Emacs Lisp from a Python function, or
is used as an argument to an Emacs Lisp function from Python, the
original Emacs Lisp object behind the Emacs Lisp handle is automatically
retrieved.
Emacs Lisp handles are either instances of the internal Lisp
class, or of one of its subclasses. If OBJECT is an Emacs
Lisp handle, and if the underlying Emacs Lisp object is an Emacs
Lisp sequence, then whenever OBJECT[INDEX], OBJECT[INDEX] =
VALUE and len(OBJECT) are meaningful, these may be used to
fetch or alter an element of the sequence directly in Emacs Lisp
space. Also, if OBJECT corresponds to an Emacs Lisp function,
OBJECT(ARGUMENTS) may be used to apply the Emacs Lisp function over
the given arguments. Since arguments have been evaluated the Python
way on the Python side, it would be conceptual overkill evaluating them
again the Emacs Lisp way on the Emacs Lisp side, so Pymacs manage to
quote arguments for defeating Emacs Lisp evaluation. The same logic
applies the other way around.
Emacs Lisp handles have a value() method, which merely returns
self. They also have a copy() method, which tries to open
the box if possible. Emacs Lisp proper lists are turned into Python
lists, Emacs Lisp vectors are turned into Python tuples. Then,
modifying the structure of the copy on the Python side has no effect on
the Emacs Lisp side.
For Emacs Lisp handles, str() returns an Emacs Lisp representation
of the handle which should be eq to the original object if
read back and evaluated in Emacs Lisp. repr() returns a Python
representation of the expanded Emacs Lisp object. If that Emacs Lisp
object has an Emacs Lisp representation which Emacs Lisp could read
back, then repr() value is such that it could be read back and
evaluated in Python as well, this would result in another object which
is equal to the original, but not necessarily eq.
The same as Emacs Lisp handles are useful for handling Emacs Lisp
objects on the Python side, Python handles are useful for handling
Python objects on the Emacs Lisp side.
Many Python objects do not have direct Emacs Lisp equivalents, including
long integers, complex numbers, modules, classes, instances and surely a
lot of others. When such are being transmitted to the Emacs Lisp side,
Pymacs use Python handles. These are automatically recovered into the
original Python objects whenever transmitted back to Python, either as
arguments to a Python function, as the Python function itself, or as the
return value of an Emacs Lisp function called from Python.
The objects represented by these Python handles may be inspected or
modified using the basic library of Python functions. For example, in:
(pymacs-exec "import re")
(setq matcher (pymacs-eval "re.compile('PATTERN').match"))
(pymacs-call matcher ARGUMENT)
the setq line above could be decomposed into:
(setq compiled (pymacs-eval "re.compile('PATTERN')")
matcher (pymacs-call "getattr" compiled "match"))
This example shows that one may use pymacs-call with
getattr as the function, to get a wanted attribute for a Python
object.
Function (pymacs-exec TEXT) gets TEXT executed as a Python
statement, and its value is always nil. So, this function may
only be useful because of its possible side effects on the Python side.
This function may also be called interactively:
M-x pymacs-exec RET TEXT RET
Function (pymacs-eval TEXT) gets TEXT evaluated as a Python
expression, and returns the value of that expression converted back to
Emacs Lisp.
This function may also be called interactively:
M-x pymacs-eval RET TEXT RET
Function (pymacs-call FUNCTION ARGUMENT...) will get Python to
apply the given FUNCTION over zero or more ARGUMENT.
FUNCTION is either a string holding Python source code for a
function (like a mere name, or even an expression), or else, a Python
handle previously received from Python, and hopefully holding a callable
Python object. Each ARGUMENT gets separately converted to Python
before the function is called. pymacs-call returns the resulting
value of the function call, converted back to Emacs Lisp.
Function (pymacs-apply FUNCTION ARGUMENTS) will get Python to
apply the given FUNCTION over the given ARGUMENTS.
ARGUMENTS is a list containing all arguments, or nil
if there is none. Besides arguments being bundled together
instead of given separately, the function acts pretty much like
pymacs-call.
Function (pymacs-load MODULE PREFIX) imports the Python
module into Emacs Lisp space. MODULE is the name of the
file containing the module, without any .py or .pyc
extension. If the directory part is omitted in MODULE, the
module will be looked into the current Python search path. Dot notation
may be used when the module is part of a package. Each top-level
function in the module produces a trampoline function in Emacs Lisp
having the same name, except that underlines in Python names are
turned into dashes in Emacs Lisp, and that PREFIX is uniformly
added before the Emacs Lisp name (as a way to avoid name clashes).
PREFIX may be omitted, in which case it defaults to base name
of MODULE with underlines turned into dashes, and followed by a
dash.
Note that pymacs-load has the effect of declaring the module
variables and methods the Emacs Lisp side, but it does not declare
anything on the Python side. Of course, Python imports the module
before making it available for Emacs, but there is no Pymacs ready
variable on the Python side holding that module. If you need to import
MODULE in a variable on the Python side, the proper incantation
is (pymacs-exec "import MODULE"). And of course, that this latter
statement does not declare anything on the Emacs Lisp side.
Whenever pymacs_load_hook is defined in the loaded
Python module, pymacs-load calls it without arguments,
but before creating the Emacs view for that module. So, the
pymacs_load_hook function may create new definitions or even add
interaction attributes to functions.
The return value of a successful pymacs-load is the module
object. An optional third argument, noerror, when given and not
nil, will have pymacs-load to return nil instead
of raising an error, if the Python module could not be found.
When later calling one of these trampoline functions, all provided
arguments are converted to Python and transmitted, and the function
return value is later converted back to Emacs Lisp. It is left to
the Python side to check for argument consistency. However, for an
interactive function, the interaction specification drives some checking
on the Emacs Lisp side. Currently, there is no provision for collecting
keyword arguments in Emacs Lisp.
This function may also be called interactively:
M-x pymacs-load RET MODULE RET PREFIX RET
We do not expect that pymacs-exec, pymacs-eval,
pymacs-call or pymacs-apply will be much used, if
ever, in most Pymacs applications. In practice, the Emacs Lisp side
of a Pymacs application might call pymacs-load a few times for
linking into the Python modules, with the indirect effect of defining
trampoline functions for these modules on the Emacs Lisp side, which can
later be called like usual Emacs Lisp functions.
These imported functions are usually those which are of interest for the
user, and the preferred way to call Python services with Pymacs.
Users could alter the inner working of Pymacs through a few variables,
these are all documented here. Except for pymacs-load-path,
which should be set before calling any Pymacs function, the value of
these variables can be changed at any time.
Users might want to use special directories for holding their Python
modules, when these modules are meant to be used from Emacs. Best is to
preset pymacs-load-path, nil by default, to a list of
these directory names. (Tilde expansions and such occur automatically.)
Here is how it works. The first time Pymacs is needed from Emacs,
a Pymacs helper is automatically started as an Emacs subprocess, and
given as arguments all strings in the pymacs-load-path list.
These arguments are added at the beginning of sys.path,
or moved at the beginning if they were already on sys.path.
So in practice, nothing is removed from sys.path.
The *Pymacs* buffer, within Emacs, holds a trace of transactions
between Emacs and Python. When pymacs-trace-transit is
nil, the buffer only holds the last bi-directional transaction
(a request and a reply). In this case, it gets erased before each and
every transaction. If that variable is t, all transactions are
kept. This could be useful for debugging, but the drawback is that
this buffer could grow big over time, to the point of diminishing Emacs
performance. As a compromise, that variable may also be a cons cell
of integers (KEEP . LIMIT), in which case the buffer is reduced to
approximately KEEP bytes whenever its size exceeds LIMIT
bytes, by deleting an integral number of lines from its beginning. The
default setting for pymacs-trace-transit is (5000 . 30000).
The default behaviour of Pymacs is to transmit Emacs Lisp objects to
Python in such a way that they are fully modifiable from the Python
side, would it mean triggering Emacs Lisp functions to act on them.
When pymacs-forget-mutability is not nil, the behaviour
is changed, and the flexibility is lost. Pymacs then tries to expand
proper lists and vectors as full copies when transmitting them on the
Python side. This variable, seen as a user setting, is best left to
nil. It may be temporarily overridden within some functions,
when deemed useful.
There is no corresponding variable from objects transmitted to Emacs
from Python. Pymacs automatically expands what gets transmitted.
Mutability is preserved only as a side-effect of not having a natural
Emacs Lisp representation for the Python object. This asymmetry is on
purpose, yet debatable. Maybe Pymacs could have a variable telling that
mutability is important for Python objects? That would give Pymacs
users the capability of restoring the symmetry somewhat, yet so far, in
our experience, this has never been needed.
Strictly speaking, Emacs Lisp strings are mutable. Yet, it does not
come naturally to a Python programmer to modify a string in-place, as
Python strings are never mutable. When pymacs-mutable-strings
is nil, which is the default setting, Emacs Lisp strings are
transmitted to Python as Python strings, and so, loose their mutability.
Moreover, text properties are not reflected on the Python side. But
if that variable is not nil, Emacs Lisp strings are rather
passed as Emacs Lisp handles. This variable is ignored whenever
pymacs-forget-mutability is set.
Emacs needs to protect itself a bit, in case the Pymacs service program,
which handles the Python side of requests, would not start correctly, or
maybe later die unexpectedly. So, whenever Emacs reads data coming from
that program, it sets a time limit, and take some action whenever that
time limit expires. All times are expressed in seconds.
The pymacs-timeout-at-start variable defaults to 30 seconds,
this time should only be increased if a given machine is so heavily
loaded that the Pymacs service program has not enough of 30 seconds to
start, in which case Pymacs refuses to work, with an appropriate message
in the mini buffer.
The two remaining timeout variables almost never need to be changed
in practice. When Emacs is expecting a reply from Python, it might
repeatedly check the status of the Pymacs service program when that
reply is not received fast enough, just to make sure that this program
did not die. The pymacs-timeout-at-reply variable, which
defaults to 5, says how many seconds to wait without checking, while
expecting the first line of a reply. The pymacs-timeout-at-line
variable, which defaults to 2, says how many seconds to wait without
checking, while expecting a line of the reply after the first.
When the Pymacs helper dies, all useful Python objects it might contain
also die with it. However, if the death occurs unexpectedly, instead of
normally at the end of the Emacs session, there might now exist dangling
references in Emacs Lisp space towards those vanished Python objects.
Pymacs could not do much without a Pymacs helper, and likely, a new
one will soon be created within the same Emacs session, and brand new
Python objects may be created within that new helper. Now, and this is
a bit technical, all references are transmitted in form of object slot
numbers. As a consequence, the new Pymacs helper should be careful at
never allocating a new Python object using a slot number of a useful
vanished object, as this might possibly create fatal confusion.
There is not enough information for the new Pymacs helper to recreate
the useful objects which disappeared. However, there is enough
machinery to recover all their slot numbers, and then, all these slots
are initialized with so-called zombies. If Emacs later calls a
vanished Python object, this merely awakes its zombie, which will then
make some noise, then fall asleep again. The noise has the form of a
diagnostic within the *Messages* buffer, sometimes visible in the
mini-buffer as well when the mini-buffer is not simultaneously used for
some other purpose.
Zombies get more dreadful if pymacs-dreadful-zombies is set to a
non-nil value. In this case, calling a vanished Python object
raises an error that will eventually interrupt the current computation.
Such a behaviour might be useful for debugging purposes, or for making
sure that no call to a vanished Python object goes unnoticed.
In previous Pymacs releases, zombies were always dreadful, under the
assumption that calling a vanished object is a real error. However, it
could cause irritation in some circumstances, like when associated with
frequently triggered Emacs Lisp hook functions. That's why that, by
default, zombies have been finally turned into more innocuous beings!
For Python modules meant to be used from Emacs and which receive nothing
but Emacs nil, numbers or strings, or return nothing but Python
None, numbers or strings, then Pymacs requires little or no
setup. Otherwise, use from Pymacs import lisp at the start of your
module. If you need more Pymacs features, like the Let class,
then write from Pymacs import lisp, Let.
The Pymacs helper runs Python code to serve the Emacs side, and it is
blocked waiting until Emacs sends a request. Until the Pymacs helper
returns a reply, Emacs is blocked in turn, yet fully listening to serve
eventual Python sub-requests, etc. So, either Emacs or the Pymacs
helper is active at a given instant, but never both at once.
Unless Emacs has sent a request to the Pymacs helper and is expecting
a reply, it is just not listening to receive Python requests. So, any
other Python thread may not asynchronously use Pymacs to get Emacs
services. The design of the Python application should be such that the
communication is always be channelled from the main Python thread.
When Pymacs starts, all process signals are inhibited on the Python
side. Yet, SIGINT gets re-enabled while running user functions.
If the user elects to reactivate some other signal in her Python code,
she should do so as to not damage or severe the communication protocol.
lisp is a special object which has useful built-in magic. Its
attributes do nothing but represent Emacs Lisp symbols, created on the
fly as needed (symbols also have their built-in magic).
As special cases, lisp.nil or lisp["nil"] are the same
as None, and lisp.t or lisp["t"] are the same as
True. Otherwise, both lisp.SYMBOL and lisp[STRING]
yield objects of the internal Symbol type. These are genuine
Python objects, that could be referred to by simple Python variables.
One may write quote = lisp.quote, for example, and use quote
afterwards to mean that Emacs Lisp symbol. If a Python function
received an Emacs Lisp symbol as an argument, it can check with ==
if that argument is lisp.never or lisp.ask, say. A Python
function may well choose to return some symbol, like lisp.always.
In Python, writing lisp.SYMBOL = VALUE or lisp[STRING] = VALUE
does assign VALUE to the corresponding symbol in Emacs Lisp
space. Beware that in such cases, the lisp. prefix may not be
spared. After result = lisp.result, one cannot hope that a later
result = 3 will have any effect in the Emacs Lisp space: this would
merely change the Python variable result, which was a reference to a
Symbol instance, so it is now a reference to the number 3.
The Symbol class has value() and copy() methods. One
can use either lisp.SYMBOL.value() or lisp.SYMBOL.copy()
to access the Emacs Lisp value of a symbol, after conversion to
some Python object, of course. However, if value() would have
given an Emacs Lisp handle, lisp.SYMBOL.copy() has the effect of
lisp.SYMBOL.value().copy(), that is, it returns the value of the
symbol as opened as possible.
A symbol may also be used as if it was a Python function, in which case
it really names an Emacs Lisp function that should be applied over the
following function arguments. The result of the Emacs Lisp function
becomes the value of the call, with all due conversions of course.
As Emacs Lisp uses dynamic bindings, it is common that Emacs Lisp
programs use let for temporarily setting new values for some
Emacs Lisp variables having global scope. These variables recover their
previous value automatically when the let gets completed, even
if an error occurs which interrupts the normal flow of execution.
Pymacs has a Let class to represent such temporary
settings. Suppose for example that you want to recover the value of
lisp.mark() when the transient mark mode is active on the Emacs Lisp
side. One could surely use lisp.mark(True) to force reading the
mark in such cases, but for the sake of illustration, let's ignore that,
and temporarily deactivate transient mark mode instead. This could be
done this way:
try:
let = Let()
let.push(transient_mark_mode=None)
... USER CODE ...
finally:
let.pop()
let.push() accepts any number of keywords arguments. Each keyword
name is interpreted as an Emacs Lisp symbol written the Pymacs way, with
underlines. The value of that Emacs Lisp symbol is saved on the Python
side, and the value of the keyword becomes the new temporary value for
this Emacs Lisp symbol. A later let.pop() restores the previous
value for all symbols which were saved together at the time of the
corresponding let.push(). There may be more than one let.push()
call for a single Let instance, they stack within that instance.
Each let.pop() will undo one and only one let.push() from the
stack, in the reverse order or the pushes.
A single call to let.pops() automatically does all pending
let.pop() at once, in the correct reverse order. When the
Let instance disappears, either because the programmer does
del let or let = None, or just because the Python let
variable goes out of scope, let.pops() gets executed under the
scene, so the try/finally statement may be omitted in
practice. For this omission to work flawlessly, the programmer should
be careful at not keeping extra references to the Let instance.
The constructor call let = Let() also has an implied initial
.push() over all given arguments, given there is any, so the
explicit let.push() may be omitted as well. In practice, this sums
up and the above code could be reduced to a mere:
let = Let(transient_mark_mode=None)
... USER CODE ...
Be careful at assigning the result of the constructor to some Python
variable. Otherwise, the instance might disappear immediately after
having been created, restoring the Emacs Lisp variable much too soon.
Any variable to be bound with Let should have been bound in
advance on the Emacs Lisp side. This restriction usually does no kind
of harm. Yet, it will likely be lifted in some later version of Pymacs.
The Let class has other methods meant for some macros which are
common in Emacs Lisp programming, in the spirit of let bindings.
These method names look like push_* or pop_*, where Emacs Lisp
macros are save-*. One has to use the matching pop_* for
undoing the effect of a given push_* rather than a mere .pop():
the Python code is clearer, this also ensures that things are undone in
the proper order. The same Let instance may use many push_*
methods, their effects nest.
push_excursion() and pop_excursion() save and restore
the current buffer, point and mark. push_match_data() and
pop_match_data() save and restore the state of the last regular
expression match. push_restriction() and pop_restriction() save
and restore the current narrowing limits. push_selected_window() and
pop_selected_window() save and restore the fact that a window holds
the cursor. push_window_excursion() and pop_window_excursion()
save and restore the current window configuration in the Emacs display.
As a convenience, let.push() and all other push_* methods return
the Let instance. This helps chaining various push_* right
after the instance generation. For example, one may write:
let = Let().push_excursion()
if True:
... USER CODE ...
del let
The if True: (use if 1: with older Python releases, some people
might prefer writing if let: anyway), has the only goal of indenting
USER CODE, so the scope of the let variable is made very
explicit. This is purely stylistic, and not at all necessary. The last
del let might be omitted in a few circumstances, for example if the
excursion lasts until the end of the Python function.
Pymacs offers a device for evaluating a raw Emacs Lisp expression, or a
sequence of such, expressed as a string. One merely uses lisp
as a function, like this:
lisp("""
...
POSSIBLY-LONG-SEQUENCE-OF-LISP-EXPRESSIONS
...
""")
The Emacs Lisp value of the last or only expression in the sequence
becomes the value of the lisp call, after conversion back to
Python.
Emacs functions have the concept of user interaction for completing the
specification of their arguments while being called. This happens only
when a function is interactively called by the user, it does not happen
when a function is directly called by another. As Python does not have
a corresponding facility, a bit of trickery was needed to retrofit that
facility on the Python side.
After loading a Python module but prior to creating an Emacs view
for this module, Pymacs decides whether loaded functions will be
interactively callable from Emacs, or not. Whenever a function has
an interaction attribute, this attribute holds the Emacs
interaction specification for this function. The specification is
either another Python function or a string. In the former case, that
other function is called without arguments and should, maybe after
having consulted the user, return a list of the actual arguments to be
used for the original function. In the latter case, the specification
string is used verbatim as the argument to the (interactive ...)
function on the Emacs side. To get a short reminder about how this
string is interpreted on the Emacs side, try C-h f interactive
within Emacs. Here is an example where an empty string is used to
specify that an interactive has no arguments:
from Pymacs import lisp
def hello_world():
"`Hello world' from Python."
lisp.insert("Hello from Python!")
hello_world.interaction = ''
Versions of Python released before the integration of PEP 232 do not
allow users to add attributes to functions, so there is a fall-back
mechanism. Let's presume that a given function does not have an
interaction attribute as explained above. If the Python module
contains an interactions global variable which is a dictionary,
if that dictionary has an entry for the given function with a value
other than None, that function is going to be interactive on the
Emacs side. Here is how the preceding example should be written for an
older version of Python, or when portability is at premium:
from Pymacs import lisp
interactions = @{@}
def hello_world():
"`Hello world' from Python."
lisp.insert("Hello from Python!")
interactions[hello_world] = ''
One might wonder why we do not merely use lisp.interactive(...)
from within Python. There is some magic in the Emacs Lisp interpreter
itself, looking for that call before the function is actually entered,
this explains why (interactive ...) has to appear first in an Emacs
Lisp defun. Pymacs could try to scan the already compiled
form of the Python code, seeking for lisp.interactive, but as the
evaluation of lisp.interactive arguments could get arbitrarily
complex, it would a real challenge un-compiling that evaluation into
Emacs Lisp.
An interactive function may be bound to a key sequence.
To translate bindings like C-x w, say, one might have to know a
bit more how Emacs Lisp processes string escapes like \C-x or
\M-\C-x in Emacs Lisp, and emulate it within Python strings, since
Python does not have such escapes. \C-L, where L is an upper case
letter, produces a character which ordinal is the result of subtracting
0x40 from ordinal of L. \M- has the ordinal one gets by adding
0x80 to the ordinal of following described character. So people can
use self-inserting non-ASCII characters, \M- is given another
representation, which is to replace the addition of 0x80 by prefixing
with `ESC', that is 0x1b.
So \C-x in Emacs is 'x18' in Python. This is easily found, using
an interactive Python session, by giving it: chr(ord('X') - ord('A') +
1). An easier way would be using the kbd function on the Emacs
Lisp side, like with lisp.kbd('C-x w') or lisp.kbd('M-<f2>').
To bind the F1 key to the helper function in some
module:
lisp.global_set_key((lisp.f1,), lisp.module_helper)
(item,) is a Python tuple yielding an Emacs Lisp vector.
lisp.f1 translates to the Emacs Lisp symbol f1. So, Python
(lisp.f1,) is Emacs Lisp [f1]. Keys like [M-f2] might
require some more ingenuity, one may write either (lisp['M-f2'],) or
(lisp.M_f2,) on the Python side.
Initially, the Pymacs communication protocol was rather simple deep
down, merely using evaluation on arrival on both sides. All the rest
was recursion trickery over that simple idea. But the magic was fragile
to interruption requests, so the protocol has been revisited a bit so
each message action could be recognized before evaluation is attempted.
The idea (not fully implemented yet) is to make the protocol part immune
to interruptions, but to allow evaluations themselves to be interrupted.
- As it is more easy to generate than to parse, and also because Emacs
has a Lisp parser and Python has a Python parser, Emacs generates
Python code when preparing a message to the Pymacs helper, and Python
generates Emacs Lisp expressions when preparing a message for Emacs.
- Messages are exchanged in strictly alternating directions (from
Python to Emacs, from Emacs to Python, etc.), the first message being
sent by the Pymacs helper just after it started, identifying the
current Pymacs version.
- Messages in both directions have a similar envelope. Each physical
message has a prefix, the message contents, and a newline. The
prefix starts with either < or > to mark the directionality,
immediately followed by the decimal expression of the contents length
counted in characters, immediately followed by a single horizontal
tab. The count excludes the prefix, but includes the newline.
- In each direction, messages are made up of two elements: an action
keyword and a single argument (yet the argument may sometimes be
complex). As a special case, memory cleanup messages, from Python
to Emacs, use four elements: the atom free, a list of slot
numbers to free, and then the real action and argument. This is
because the cleanup is delayed and piggy-backed over some other
message from Python to Emacs.
- For Emacs originated messages, the action and the argument are
separated by a space. For Python originated messages, the action and
the argument are made into a Lisp list.
- Most actions in the following table are available in both
directions, unless noted. The first three actions start a new level
of Pymacs evaluation, the remaining actions end the current level.
- eval requests the evaluation of its expression argument.
- exec requests the execution of its statement argument (this may
only be received on the Python side).
- expand requests the opening of an Emacs Lisp structure (this may
only be received on the Emacs side).
- return represents the normal reply to a request, the argument
holds the value to be returned (nil in case of exec).
- raise represents the error reply to a request, the argument
then holds a diagnostic string.
Emacs and Python are two separate processes (well, each may use more
than one process). Pymacs implements a simple communication protocol
between both, and does whatever needed so the programmers do not have
to worry about details. The main debugging tool is the communication
buffer between Emacs and Python, which is named *Pymacs*. By
default, this buffer gets erased before each transaction. To make good
debugging use of it, first set pymacs-trace-transit to either
t or to some (KEEP . LIMIT). As it is sometimes helpful to
understand the communication protocol, it is briefly explained here,
using an artificially complex example to do so. Consider:
(pymacs-eval "lisp('(pymacs-eval \"repr(2L**111)\")')")
"2596148429267413814265248164610048L"
Here, Emacs asks Python to ask Emacs to ask Python for a simple bignum
computation. Note that Emacs does not natively know how to handle big
integers, nor has an internal representation for them. This is why I
use the repr function, so Python returns a string representation
of the result, instead of the result itself. Here is a trace for this
example. Imagine that Emacs stands on the left and that Python stands
on the right. The < character flags a message going from Python to
Emacs, while the > character flags a message going from Emacs to
Python. The number gives the length of the message, including the end
of line. (Acute readers may notice that the first number is incorrect,
as the version number gets replaced in the example while this manual is
being produced.)
<22 (version "0.24-beta1")
>43 eval lisp('(pymacs-eval "repr(2L**111)")')
<45 (eval (progn (pymacs-eval "repr(2L**111)")))
>19 eval repr(2L**111)
<47 (return "2596148429267413814265248164610048L")
>45 return "2596148429267413814265248164610048L"
<47 (return "2596148429267413814265248164610048L")
Python evaluation is done in the context of the Pymacs.pymacs
module, so for example a mere reply really means
Pymacs.pymacs.reply. On the Emacs Lisp side, there is no concept of
module name spaces, so we use the pymacs- prefix as an attempt to
stay clean. Users should ideally refrain from naming their Emacs Lisp
objects with a pymacs- prefix.
reply and pymacs-reply are special functions meant to
indicate that an expected result is finally transmitted. error
and pymacs-error are special functions that introduce
a string which explains an exception which recently occurred.
pymacs-expand is a special function implementing the copy()
methods of Emacs Lisp handles or symbols. In all other cases, the
expression is a request for the other side, that request stacks until a
corresponding reply is received.
Part of the protocol manages memory, and this management generates some
extra-noise in the *Pymacs* buffer. Whenever Emacs passes a
structure to Python, an extra pointer is generated on the Emacs side to
inhibit garbage collection by Emacs. Python garbage collector detects
when the received structure is no longer needed on the Python side, at
which time the next communication will tell Emacs to remove the extra
pointer. It works symmetrically as well, that is, whenever Python
passes a structure to Emacs, an extra Python reference is generated to
inhibit garbage collection on the Python side. Emacs garbage collector
detects when the received structure is no longer needed on the Emacs
side, after which Python will be told to remove the extra reference.
For efficiency, those allocation-related messages are delayed, merged
and batched together within the next communication having another
purpose.
Variable pymacs-trace-transit may be modified for controlling
how and when the *Pymacs* buffer, or parts thereof, get erased.
If cross-calls between Emacs Lisp and Python nest deeply, an error will
raise successive exceptions alternatively on both sides as requests
unstack, and the diagnostic gets transmitted back and forth, slightly
growing as we go. So, errors will eventually be reported by Emacs. I
made no kind of effort to transmit the Emacs Lisp back trace on the
Python side, as I do not see a purpose for it: all debugging is done
within Emacs windows anyway.
On recent Emacses, the Python back trace gets displayed in the
mini-buffer, and the Emacs Lisp back trace is simultaneously shown
in the *Backtrace* window. One useful thing is to allow to
mini-buffer to grow big, so it has more chance to fully contain the
Python back trace, the last lines of which are often especially useful.
Here, I use:
(setq resize-mini-windows t
max-mini-window-height .85)
in my .emacs file, so the mini-buffer may use 85% of the screen,
and quickly shrinks when fewer lines are needed. The mini-buffer
contents disappear at the next keystroke, but you can recover the Python
back trace by looking at the end of the *Messages* buffer. In
which case the ffap package in Emacs may be yet another friend!
From the *Messages* buffer, once ffap activated, merely
put the cursor on the file name of a Python module from the back trace,
and C-x C-f RET will quickly open that source for you.
I found useful to automatically pymacs-load some Python files
whenever they get saved from Emacs. This can be decided on a per-file
or per-directory basis. To get a particular Python file to be reloaded
automatically on save, add the following lines at the end:
# Local Variables:
# pymacs-auto-reload: t
# End:
Here is an example of automatic reloading on a per-directory basis.
The code below assumes that Python files meant for Pymacs are kept in
~/share/emacs/python:
(defun fp-maybe-pymacs-reload ()
(let ((pymacsdir (expand-file-name "~/share/emacs/python/")))
(when (and (string-equal (file-name-directory buffer-file-name)
pymacsdir)
(string-match "\\.py\\'" buffer-file-name))
(pymacs-load (substring buffer-file-name 0 -3)))))
(add-hook 'after-save-hook 'fp-maybe-pymacs-reload)
The Pymacs helper is a Python program which accepts options and arguments.
The available options, which are only meant for debugging, are:
|
-d FILE |
Debug the protocol to FILE |
|
-s FILE |
Trace received signals to FILE |
- The -d option saves a copy of the communication protocol in the
given file, as seen from the Pymacs helper. The file should be fairly
identical to the contents of the *Pymacs* buffer within Emacs.
- The -s option monitors most signals received by the Pymacs helper
and logs them in the given file. Each log line merely contains a signal
number, possibly followed by a star if the interruption was allowed in.
Besides logging, signals are usually ignored.
The arguments list directories to be added at the beginning of the
Python module search path, and whenever Emacs launches the Pymacs
helper, the contents of the Emacs Lisp pymacs-load-path variable
is turned into this argument list.
The Pymacs helper options may be set through the PYMACS_OPTIONS
environment variable. For example, one could execute something like:
export PYMACS_OPTIONS='-d /tmp/pymacs-debug -s /tmp/pymacs-signals'
in a shell (presuming bash here) and start Emacs from that shell.
Then, when Emacs will launch the Pymacs helper, the above options will
be obeyed.
Memory may leak in some theoretical circumstances (I say theoretical,
because no one ever reported this as being an actual problem). As
Richard Stallman once put it (2002-08):
I wonder, though, can this [memory management] technique fully handle
cycles that run between Lisp and Python? Suppose Lisp object A refers
to Python object B, which refers to Lisp object A, and suppose nothing
else refers to either one of them. Will you succeed in recognizing
these two objects as garbage?
I once hungered for a Python-extensible editor, so much so that I
pondered the idea of dropping Emacs for other avenues, but found nothing
much convincing. Moreover, looking at all Lisp extensions I'd made
for myself, and considering all those superb tools written by others,
all of which are now part of my computer life, it would have been a
huge undertaking for me to reprogram these all in Python. So, when I
began to see that something like Pymacs was possible, I felt strongly
motivated! :-)
Pymacs draws on previous work of Cedric Adjih that enabled
the running of Python as a process separate from Emacs.
See http://www.crepuscule.com/pyemacs/, or write Cedric at
mailto:adjih-pam@crepuscule.com. Cedric presented pyemacs to me
as a proof of concept. As I simplified that concept a bit, I dropped
the e in pyemacs :-). Cedric also previously wrote patches for
linking Python right into XEmacs, but abandoned the idea, as he found
out that his patches were unmaintainable over the evolution of both
Python and XEmacs.
As Brian McErlean independently and simultaneously wrote a tool
similar to this one, we decided to merge our projects. In an amusing
coincidence, he even chose pymacs as a name. Brian paid
good attention to complex details that escaped my courage, so his
help and collaboration have been beneficial. You may reach Brian at
mailto:brianmce@crosswinds.net.
The initial throw at Pymacs has been written on 2001-09-05, and releases
in the 0.x series followed in a rapid pace for a few months, and Pymacs
became stable. Since then, it did not need to move much, as bugs are
not found often. Yet, in my opinion, some missing features should be
addressed before we dare some 1.0 release.
Pymacs has been fairly stable since the early versions. I surely used
it a great deal, constantly, magically, in my daily works, to the point
of forgetting that was it there all the time. It was fairly complete,
at least for my own needs, and did not move much anymore.
Some time later, someone begged me to consider Vim, and not only Emacs,
for some tools I was then writing. Looking at Vim more closely, I
discovered that it is a worth editor, with Python nicely integrated,
enough for me to switch. In a Web article (which many enjoyed, as
they told me), I detailed my feelings on these matters.
My viewpoint is that Pymacs, maybe after an initial flurry of a bit more
than a dozen releases, soon became stable in its history. Reported
bugs or suggestions were minor, there was not enough new material to
warrant other releases. Nevertheless, when I switched from Emacs to
Vim in my day-to-day habits, I felt that Pymacs needed a more credible
maintainer than me. Syver Enstad, who was an enthusiastic user and
competent contributor, was kind enough to accept the duty (2003-10).
Some more bugs and suggestions flowed in since then, but Syver did not
elect to make any new release, and this did not bother me. Syver then
became unavailable, to the point I could not contact him in years. I
would loathe to see myself interfering with an official maintainer,
but when I decided to return to some moderate Emacs usage, and because
of the long silence, I considered resuming Pymacs maintenance as well
(2007-11). Then, I dived into it for real (2008-01).
Giovanni Giorgi once (2007-03) wanted to expand on Pymacs and publish
it on his own, and later felt like maintaining it whole (late 2007-12).
I rather suggested an attempt at collaborative maintenance, and this
experiment is still going on...
Emacs Lisp is deeply soldered into Emacs internals. Vim has its own
language, which people sometimes call Vimscript, similarly tightened
into Vim. My feeling is that Emacs Lisp allows for a more intimate
handling of edit buffers and external processes than Vimscript does, yet
this intimacy has a price in complexity, so all totalled, they may be
perceived as comparable for most practical purposes.
Pymacs allows customising Emacs with Python instead of Emacs Lisp, and
then runs Python as a process external to Emacs, with a communication
protocol between both processes. Python may be built into Vim, and then
both Python and Vim use a single process. The same as Pymacs gives
access to almost all of Emacs Lisp, Python within Vim gives access to
almost all of Vimscript, but with a much smaller overhead than Pymacs.
Pymacs is not Emacs Lisp, and Python in Vim is not Vimscript either,
tweaks are needed in both cases for accessing some of the underlying
scripting facilities. Pymacs is rather elegant, Python in Vim is rather
clean. Python itself is both elegant and clean, but one strong point of
Python for me is the legibility, which builds deeper roots on the clean
side than on the elegant side. All in all, despite I know how debatable
it can be, I guess I now have a prejudice towards Python in Vim.
I figured out a simple way to have the same Python source usable both
within Pymacs or Vim. However, Emacs is byte oriented, while Vim is
line oriented. In a few Pymacs applications of mine, I internally
toggle between line orientation and byte orientation, keeping both for
speed most probably, while I see things would be a bit simpler (and
maybe slower) if I was pushing myself on the line-oriented side. Each
of Emacs and Vim have their own logic and elegance, and it is probable
that we loose overall if we try to emulate one with the other.
The idea traversed me to convert all the few Pymacs examples so they
work both for Pymacs and Vim, and through the documentation, publicise
how people writing Python extensions could write them for both editors
at once. Yet, while doing so, one has to stretch either towards Emacs
or Vim, and I guess I would favour Vim over Emacs when the time comes to
evaluate efficiency-related choices.
I also thought about writing a Pymacs module for running Python scripts
already written for Vim, by offering a compatibility layer. The
complexity of this might be unbounded, I should study actual Python
scripts for Vim before knowing better if this is thinkable or not.
Gerd Möllman, who was maintaining Emacs at the time of Pymacs birth and
development, retrofitted (2001-09) the idea of a post-gc-hook
from XEmacs, as a way to facilitate memory management within Pymacs.
Richard Stallman once suggested (2001-10) that Pymacs be distributed
within Emacs, and while discussing the details of this, I underlined
small technical difficulties about Emacs installing the Python parts,
and the need of a convention about where to install Python files meant
for Pymacs. As Richard felt, at the time, very overwhelmed with other
duties, no decision was taken and the integration went nowhere.
After Gerd resigned as an Emacs maintainer, someone from the Emacs
development team wrote again (2002-01) asking information about how
to integrate Pymacs. It was easy for me to write a good and thorough
summary, after all these discussions with Richard. And that's the end
of the story: I never heard of it again. :-)
Doug Bagley's shoot out project compares the relative speed of many
popular languages, and this might interest Pymacs users. The first URL
points to the original, the second points to a newer version oriented
towards Win32 systems, the third is more recent but Debian-oriented:
I've not heard of any Python to Lisp compiler. Lisp may be slow or fast
depending on how one uses it, and how much one uses declarations. Some
Lisp systems have really excellent compilers, that give very fast code
when properly hinted.
Python itself may be slow or fast, once again depending on how one
uses it. With the proper bend, one can develop the habit of writing
Python which shows honest speed. And there is always Pyrex, which is
Python complemented with explicit declarations (a bit like some Lisp
implementations), and which can buy a lot of speed.
This is quite likely that one can have fast programs while using Python,
or a mix of Python and Pyrex (or even Psyco sometimes), that is, within
Python paradigms, without feeling any need of resorting to Lisp.
If Python looks like being slow while being used with Emacs, the problem
probably lies in Emacs-Python communication which Pymacs implements.
One has to learn how to do the proper compromises for having less
communications. (In that regard, Vim and Python are really linked
together, so Python in Vim is likely faster than Pymacs for someone who
does not pay special attention to such matters.)
Ali Gholami Rudi also writes (2008-02):
Well, there seems to be lots of overhead when transferring large
strings. Transferring them requires:
- escaping characters in the strings
- putting them in *Pymacs* buffer
- sending the region to Python process
- evaluating the Python string in Python-side (involves compiling)
In my experiments, transferring a ~5k-line file takes more than a
second on a relatively new computer (data from rope-dev).
Improving that probably requires a new protocol that does not
use Python eval and has an optional debug buffer. Probably few
applications need to transfer large strings to Python but if they do,
it is quite slow.
All in all, speed may sometimes become a real issue for Pymacs. I once
wrote within http://pinard.progiciels-bpi.ca/opinions/editors.html :
While Pymacs is elegant in my opinion, one cannot effectively use
Pymacs (the Python part) without knowing at least the specification
of many Lisp functions, and I found that it requires some doing for a
Pymacs developer to decouple the Emacs interaction part from the purer
algorithmic part in applications. Moreover, if you do not consider
speed issues, they bite you.
Some people suggested important internal Pymacs changes. In my opinion,
new bigger features are better implemented in a careful way, first as
examples or contributions, and moved closer to internal integration
depending on how users use or appreciate them. For now, Pymacs should
concentrate at doing its own humble job well, and resist bloat.
Before Pymacs closes to some version 1.0, some specifications should be
revisited, user suggestions pondered, porting matters documented. The
test suite should grow up, we should collect more examples. Pymacs
should aim seamless integration with .el files and with
transparent autoload (my little tries were not so successful).
On the Python side, Pymacs might fake primitives like getindex
and putindex, better support iterators and some newer Python
features, and at least consider Python 3.0.
Pymacs is not much geared towards Python threads. It is not clear yet if
it would be reasonably tractable to better support them.
|
|
|