mercurial needs better end-of-line support
One real world issue with source control systems is the handling of end-of-line characters in text files. Currently Mercurial pretty much punts. The hg book says:
Note: The Windows version of Mercurial does not automatically convert line endings between Windows and Unix styles. If you want to share work with Unix users, you must do a little additional configuration work. XXX Flesh this out.
NOTE: the tempfile mechanism is recommended for Windows systems,
where the standard shell I/O redirection operators often have
strange effects. In particular, if you are doing line ending
conversion on Windows using the popular dos2unix and unix2dos
programs, you *must* use the tempfile mechanism, as using pipes will
corrupt the contents of your files.
Tempfile example:
[encode]
# convert files to unix line ending conventions on checkin
**.txt = tempfile: dos2unix -n INFILE OUTFILE
[decode]
# convert files to windows line ending conventions when writing
# them to the working dir
**.txt = tempfile: unix2dos -n INFILE OUTFILE
However (1) unix2dos and dos2unix are generally not available on Windows machines and (2) if dos2unix isn't available the "encoding" here will silently wipe out your file to empty content on checkin.
How is Mozilla handling this in their hg repository? Is it mandated that new files added on Windows use Unix line endings or is some kind of conversion for Windows attempted?
3 commentspatch for "custom action" for mozilla updater
Here is a patch (against a slightly out of date updater.cpp on the Mozilla 1.8 branch) that would add support for a:
customaction "relative-path-to-executable"
action in the "update.manifest" for a partial or full update (.mar file) for the Mozilla update system.
I'm just chucking this up here quickly for lack of a better place to put it right now. Komodo uses the Mozilla update system and will possibly need this patch at some point.
Limitations:
- It ignores the return value of the spawned executable.
- It doesn't support arguments to the executable.
an intro to Komodo extensions
Komodo uses the Mozilla extension mechanism -- same .xpi files as Firefox to install an extension, same kind of bundle content in an extension. However, Komodo adds a number of "hooks" that can be used to customize Komodo with an extension (see the end of this post).
In Komodo 4.2 (currently in beta) we've been working at improving the extension story. Part of my work there has been to improve the tools for building them. To that end Komodo 4.2 now includes a sort of "SDK" with a few tools:
- koext
- A tool for building and generating stubs for Komodo extensions. A recently added a (very brief) intro to using koext to Komodo's extension forum.
- luddite
- A tool for working with Komodo's UDL (User-Defined Languages) system. The UDL system (new in Komodo 4.0) provides a way to define lexers for new languages. Lexers are used mainly for syntax coloring, but can also be used by Komodo Code Intelligence system for provide autocomplete and calltips. Eric wrote up a long intro to UDL a while back. UDL currently isn't for the faint of heart, but it provides an execellent system for robust lexing of code languages -- in particular it supports *multi-language* code (e.g. JavaScript in HTML, Ruby in RHTML, CSS in Django HTML).
- codeintel
- A tool to help writing a language support for Komodo's Code Intelligence system. I'll write more on this later.
These tools are all works in progress but they are used internally as part of normal Komodo development, so should be usable for Komodo extension authors.
Komodo's koext tool briefly describes all the current Komodo extension "hooks":
$ koext help hooks
Many parts of Komodo's functionality can be extended with a
Komodo extension. We call those "hooks" here. The following is
a list of all extension hooks that Komodo currently supports.
The "source tree files" sections below are conventions for
placement of sources files. If you use these conventions, then
`koext build' will automatically be able to build your extension
properly.
chrome
Chrome is the collective term for XUL (content), JavaScript
(content), CSS (skin), images (skin) and localized files
(locale, typically DTDs) that can be used to extend the
Komodo UI. This works in Komodo extensions in exactly the
same way as any other Mozilla-base application (such as
Firefox). See `koext help chrome' for some tips.
source tree files:
chrome.manifest
content/ # XUL overlays, dialogs and JavaScript
skin/ # CSS
locale/ # localized files (typically DTDs)
XPCOM components
XPCOM components are placed here. These can be written in
Python or JavaScript. (C++-based components are possible
as well, but currently the Komodo SDK does not support
building them.)
source files:
components/
*.idl # interface definitions
*.py # PyXPCOM components
*.js # JavaScript XPCOM components
templates
A file hierarchy under here maps into Komodo's "New File"
dialog. For example, "templates/Common/Foo.pl" will result
in a new Perl file template called "Foo" in the "Common"
folder of the "New File" dialog.
source files:
templates/
lexers
Komodo User-Defined Languages (UDL) system provides a
facility for writing regular expression, state-based lexers
for new languages (including for multi-lang languages).
".lexres" files are built from ".udl" source files with
the "luddite" tool (in this SDK). See `koext help udl' and
Komodo's UDL documentation for more details.
source files:
udl/
*-mainlex.udl # a .lexres will be build for each of these
*.udl # support files to be included by
# "*-mainlex.udl" files
XML catalogs
An extension can include an XML catalog (and associates
schemas) defining namespace to schema mappings for XML
autocomplete.
source files:
catalog.xml # Note: This may move to xmlcatalogs/...
API catalogs
An extension can include API catalogs to provide autocomplete
and calltips for 3rd party libraries. An API catalog is a CIX
file (an XML dialect) that defines the API of a
library/project/toolkit.
source files:
apicatalogs/ # .cix files here will be included
# in the API catalog list in the
# "Code Intelligence" prefs panel
Python modules
An extension can supply Python modules by placing then in
the "pylib" directory of the extension. This "pylib" directory
will be appended to Komodo's Python runtime sys.path.
source files:
pylib/
codeintel
An extension can provide the Code
Intelligence logic (for autocomplete and calltips, for
"Jump to Definition" and for the Code Browser in Komodo IDE)
for new languages.
source files:
pylib/ # lang_*.py files here are picked up
# by the codeintel system.
6
comments
open komodo and the code
Yesterday we (ActiveState) announced Open Komodo, an open-source project seeded with much of the core of Komodo Edit and Komodo IDE with the goals of produce a platform/framework for and (codename Komodo Snapdragon) an IDE for client-side open web development.
That's a mouthful. Shane and David have done a good job giving some wider perspective on what the Open Komodo project could mean (if all goes well). David went so far as to invent new language to make his points.
Some quick thoughts from a coder's perspective:
The source will be available in a Mercurial repository in (quoting Shane paraphrasing Mike Shaver) "Two F**king Months!". Early November -- or earlier if we can.
Komodo is a Mozilla-based application with the added heavy use of PyXPCOM for much of the core logic. That means the app comes together like this:
- Get a slightly tweaked mozilla build (C++, JavaScript, XUL).
- Get a slightly tweaked Python build (C).
- Add a bunch of core logic (Python). For example, the guts of Komodo's Find/Replace system is written in Python -- using Python's unicode-aware regular expression engine.
- Add Komodo chrome (XUL, JavaScript, CSS, DTDs).
What this means is that to work on and add significant functionality to Komodo, all you tend to need to know is XUL, JavaScript and Python. From early on in Komodo's development we've felt that this is one of Komodo's aces in the hole: developing in the dynamic languages is so much faster. I remember David Ascher making the comment way back that if Subversion had been written in Python, it would have been ready years sooner. And now two of the primary DVCS, Mercurial and Bazaar, are written in Python.
Komodo uses the same extension mechanisms as Firefox. It is easy to build a .xpi to add functionality to Komodo. We really hope that a community of Komodo extension authors will develop.
Komodo builds and runs on Windows, Linux and Mac OS X. Given some work there is little reason the Open Komodo code base couldn't be made to run well on Solaris, BSD, etc.
If any of this sounds interesting to you as an open-source tinkerer, then give Komodo Edit or Komodo IDE a try. The first app that will come out of the Open Komodo project (Komodo Snapdragon) will look and feel a lot like them.
In subsequent posts, and especially once the source code repository is up, I plan to blog here about Komodo's internals.
3 commentsmini-mick cometh
Mini-mick (a.k.a. Ewan Mick) arrived about three weeks ago. I am so happy to have him and proud of my wife Alli.
In some ways, having a child is like a little boy's dream come true: farts and poos are acceptable entertainment!
Posts
